all AI news
GGUF, the long way around
Feb. 29, 2024, 9:39 p.m. |
Simon Willison's Weblog simonwillison.net
Vicki Boykis dives deep into the GGUF format used by llama.cpp, after starting with a detailed description of how PyTorch models work and how they are traditionally persisted using Python pickle.
Pickle lead to safetensors, a format that avoided the security problems with downloading and running untrusted pickle files.
Llama.cpp introduced GGML, which popularized 16-bit (as opposed to 32-bit) quantization and bundled metadata and tensor data in a single file.
GGUF fixed some design flaws …
ai cpp files format generativeai llama llms python pytorch running security work
More from simonwillison.net / Simon Willison's Weblog
We can have a different web
1 day, 9 hours ago |
simonwillison.net
Introducing the Claude Team plan and iOS app
1 day, 20 hours ago |
simonwillison.net
Jobs in AI, ML, Big Data
AI Research Scientist
@ Vara | Berlin, Germany and Remote
Data Architect
@ University of Texas at Austin | Austin, TX
Data ETL Engineer
@ University of Texas at Austin | Austin, TX
Lead GNSS Data Scientist
@ Lurra Systems | Melbourne
Senior Machine Learning Engineer (MLOps)
@ Promaton | Remote, Europe
Robotics Technician - 3rd Shift
@ GXO Logistics | Perris, CA, US, 92571