Tele-FLM Technical Report

arXiv:2404.16645v1 Announce Type: new
Abstract: Large language models (LLMs) have showcased profound capabilities in language understanding and generation, facilitating a wide array of applications. However, there is a notable paucity of detailed, open-sourced methodologies on efficiently scaling LLMs beyond 50 billion parameters with minimum trial-and-error cost and computational resources. In this report, we introduce Tele-FLM (aka FLM-2), a 52B open-sourced multilingual large language model that features a stable, efficient pre-training paradigm and enhanced factual judgment capabilities. Tele-FLM demonstrates superior multilingual …

abstract applications array arxiv beyond billion capabilities computational cost cs.ai cs.cl error however language language models language understanding large language large language models llms minimum parameters report resources scaling technical type understanding