experimental-phi3-webgpu

experimental-phi3-webgpu

Run Microsoft’s excellent Phi-3 model directly in your browser, using WebGPU so didn’t work in Firefox for me, just in Chrome.

It fetches around 2.1GB of data into the browser cache on first run, but then gave me decent quality responses to my prompts running at an impressive 21 tokens a second (M2, 64GB).

I think Phi-3 is the highest quality model of this size, so it’s a really good fit for running in a browser like this.

Via …

ai browser browsers cache chrome data experimental firefox generativeai homebrewllms llms microsoft phi phi-3 prompts quality responses running the browser think tokens webassembly webgpu work