WebAssembly is aliveeeee 🔥 Gemini Nano will be included in Chrome itself with WebGPU + Wasm.

Phi 3 running on your browser!

100% local, powered by WebGPU & Rust 🦀

一个有趣的WebGPU的实验性项目，一个迷你的网页解谜游戏。竟然完整的开源了~

github.com/neuroprod/webg…

Introducing Phi-3 WebGPU, a private and powerful AI chatbot that runs locally in your browser, powered by 🤗 Transformers.js and onnxruntime-web!

🔒 On-device inference: no data sent to a server
⚡️ WebGPU-accelerated (> 20 t/s)
📥 Model downloaded once and cached

Try it out! 👇

Phi-3 WebGPU： 在浏览器中本地运行Phi-3

🔒 所有数据处理都在本地完成，确保隐私
⚡️ WebGPU加速：提升处理速度 超过20 t/s
📥 首次下载后，模型将被缓存，无需重复下载

跑了一下，英文回答还可以，中文幻觉特别严重。

体验地址：huggingface.co/spaces/Xenova/……

This is interesting.

Phi-3 WebGPU: a private and powerful AI chatbot that runs 100% locally in your browser

conversations are not sent to a server for processing... built with Transformers.js and onnxruntime-web

huggingface.co/spaces/Xenova/…

with WebGPU you can run llama3 8b on a website in Chrome at 30 tokens per second
fully local/offline with model cached in browser

When WebGPU support arrives for all browsers, the local LLM ecosystem will change forever:

- Experiment with models locally without sending any data to servers or downloading new apps 🤯
- Have easy support of models (vision models, LLMs, etc) through browser extensions
- Very…

ローカルLLMを利用する一つの形。
ブラウザで開いているけど、サーバー側でGPUによる推論をしているのではなく、開いている私の端末で推論が行われる。

【Phi-3 - WebGPU】…

Phi-3 running at 42 tokens per second 100% locally in your browser! 🤯⚡️

What speed do you get? Try it out yourself! 👇
huggingface.co/spaces/Xenova/…

My guy Xenova getting a shoutout on stage of the developer keynote during the on device section about WebGPU

The always brilliant Xenova released an experimental WebGPU powered demo of phi-3 running directly in the browser. My consumer but well-specced workstation got ~42 tokens per second!

The age of usable on-device inference is rapidly approaching!

This is a webGPU experimental work, creator Kris Temmerman has open-sourced all the code for this project, it's awesome.

foodforfish.org
github.com/neuroprod/webg…

やりたいことがWebGPUでできそうなのでチュートリアルやってみた。ピンポンバッファの設定が結構楽になってるかも

I played with WebGPU and built Conway's Game of Life by following a tutorial.

codelabs.developers.google.com/your-first-web…

#webgpu

Got the WebGPU renderer in Three.js working to create a basic Lorenz attractor! However, it goes off the rails after about 30s, I think due to floating point inconsistencies. Suspect I need to rethink my approach, but trying to find a parallel solution is breaking my brain.

I got Llama 3 running in my browser using only my GPU with my Wi-Fi switched OFF completely client-side

WebGPU is a new feature in browsers where JS can use the GPU of the device and apparently you can run LLMs on it too, and it's fast!

My end goal for 🧠…

WebGPU enthusiasts
> so you're saying there's a chance

Demo: huggingface.co/spaces/Xenova/…

#GoogleIO WASM and WebGPU are the best partners, both to release local performance. The wa programming language is also designed for wasm, and we are currently improving webgpu support. The picture on the right is a demo of webgpu written in wa language. Chrome for Developers WebAssemblyWeekly