Birbal: An efficient 7B instruct-model fine-tuned with curated datasets

arXiv:2403.02247v1 Announce Type: new
Abstract: LLMOps incur significant costs due to hardware requirements, hindering their widespread accessibility. Additionally, a lack of transparency in model training methods and data contributes to the majority of models being non-reproducible. To tackle these challenges, the LLM Efficiency Challenge was introduced at NeurIPS Workshop, aiming to adapt foundation models on a diverse set of tasks via fine-tuning on a single GPU (RTX 4090 or A100 with 40GB) within a 24-hour timeframe. In this system description …

abstract accessibility arxiv challenge challenges costs cs.cl data datasets efficiency hardware llm llmops neurips requirements training transparency type workshop