[R] Beyond Language Models: Byte Models are Digital World Simulators - Microsoft Research Asia 2024 - bGPT - Exceptional capabilities in simulating CPU behaviour, with an accuracy exceeding 99.99% in executing various operations! Could help combat th

Paper: [https://arxiv.org/abs/2402.19155](https://arxiv.org/abs/2402.19155)

Paper Page with **code and weights**: [https://byte-gpt.github.io/](https://byte-gpt.github.io/)

Abstract:

>**Traditional deep learning often overlooks bytes, the basic units of the digital world, where all forms of information and operations are encoded and manipulated in binary format**. Inspired by the success of next token prediction in natural language processing, we introduce **bGPT**, a model with **next byte prediction** to simulate the digital world. bGPT matches specialized models in performance across various modalities, including text, audio, and images, and offers new …

abstract basic binary deep learning digital digital world format forms information language language processing machinelearning natural natural language natural language processing next operations performance prediction processing success token units world