Model Merging and Mixtures of Experts // Maxime Labonne // AI in Production Conference

// Abstract
Model merging has recently become extremely popular in the open-source community. The idea of merging several fine-tuned models, or combining them into a Mixture of Experts (MoE), led to new state-of-the-art LLMs. This talk introduces the main concepts around model merging and how to implement it using the mergekit library. It provides a notebook to create your own models and directly upload them on the Hugging Face Hub.

// Bio
Maxime Labonne is a seasoned Machine Learning Scientist …

abstract art become community concepts conference experts llms maxime labonne merging mixture of experts moe popular production state talk them