[D] Discussing Apple's Deployment of a 3 Billion Parameter AI Model on the iPhone 15 Pro - How Do They Do It?

Hey everyone,

So, I've been working with running the Phi-3 mini locally, and honestly, it's been a bit of a ok . Despite all the tweaks and structured prompts in model files, it was normal, especially considering the laggy response times on a typical GPU setup. I was recently checking Apple's recent on -device model, they've got a nearly 3 billion parameter AI model running on an iPhone 15 Pro!

It's a forward in what's possible with AI on mobile …

ai model apple billion deployment files hey iphone iphone 15 pro machinelearning normal phi phi-3 prompts running