[R] DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligence - DeepSeek-AI 2024 - SOTA open-source coding model that surpasses GPT-3.5 and Codex while being unrestricted in research and commercial use!

Paper: [https://arxiv.org/abs/2401.14196](https://arxiv.org/abs/2401.14196)

Github: [https://github.com/deepseek-ai/DeepSeek-Coder](https://github.com/deepseek-ai/DeepSeek-Coder)

Models: [https://huggingface.co/deepseek-ai](https://huggingface.co/deepseek-ai)

Abstract:

>The rapid development of large language models has revolutionized code intelligence in software development. However, the predominance of closed-source models has restricted extensive research and development. To address this, we introduce the DeepSeek-Coder series, a range of open-source code models with sizes from 1.3B to 33B, trained from scratch on 2 trillion tokens. These models are pre-trained on a high-quality project-level code corpus and employ a fill-in-the-blank task with a 16K window to …

abstract code code intelligence coder deepseek development intelligence language language models large language large language models machinelearning research research and development series software software development tokens