Researchers from NVIDIA Introduce Retro 48B: The Largest LLM Pretrained with Retrieval before Instruction Tuning

Researchers from Nvidia and the University of Illinois at Urbana Champaign introduce Retro 48B, a significantly larger language model than previous retrieval-augmented models like Retro (7.5B parameters). Retro 48B is pre-trained with retrieval on an extensive corpus, leading to improved perplexity. The encoder in InstructRetro can be ablated, suggesting that continued retrieval-augmented pre-training enhances the […]

The post Researchers from NVIDIA Introduce Retro 48B: The Largest LLM Pretrained with Retrieval before Instruction Tuning appeared first on MarkTechPost.

ai shorts applications artificial intelligence editors pick encoder illinois language language model large language model llm machine learning nvidia perplexity researchers retrieval staff tech news technology university