Meta Open-Sources MEGALODON LLM for Efficient Long Sequence Modeling

Researchers from Meta, University of Southern California, Carnegie Mellon University, and University of California San Diego recently open-sourced MEGALODON, a large language model (LLM) with an unlimited context length. MEGALODON has linear computational complexity and outperforms a similarly-sized Llama 2 model on a range of benchmarks.

ai anthony benchmarks california carnegie mellon carnegie mellon university complexity computational context context length deep learning generative-ai language language model large language large language model large language models linear llama llama 2 llama 2 model llm meta ml & data engineering modeling neural networks researchers university university of california university of southern california