Microsoft’s Orca Learns From Complex Explanation Traces of GPT-4 to Significantly Enhance Smaller Models

In a new paper Orca: Progressive Learning from Complex Explanation Traces of GPT-4, a Microsoft research team introduces Orca, a 13-billion parameter model that learns explanation traces; step-by-step thought processes; and complex instructions from GPT-4 to significantly boosts SOTA instruction-tuned models.

ai artificial intelligence deep-neural-networks gpt gpt-4 machine learning machine learning & data science microsoft microsoft research ml paper processes progressive learning research research team sota team technology thought traces