Prompt-prompted Mixture of Experts for Efficient LLM Generation

arXiv:2404.01365v1 Announce Type: new
Abstract: With the development of transformer-based large language models (LLMs), they have been applied to many fields due to their remarkable utility, but this comes at a considerable computational cost at deployment. Fortunately, some methods such as pruning or constructing a mixture of experts (MoE) aim at exploiting sparsity in transformer feedforward (FF) blocks to gain boosts in speed and reduction in memory requirements. However, these techniques can be very costly and inflexible in practice, as …

arxiv cs.ai cs.cl cs.lg experts llm mixture of experts prompt type