Inferring the Phylogeny of Large Language Models and Predicting their Performances in Benchmarks

arXiv:2404.04671v1 Announce Type: cross
Abstract: This paper introduces PhyloLM, a method applying phylogenetic algorithms to Large Language Models to explore their finetuning relationships, and predict their performance characteristics. By leveraging the phylogenetic distance metric, we construct dendrograms, which satisfactorily capture distinct LLM families (across a set of 77 open-source and 22 closed models). Furthermore, phylogenetic distance predicts performances in benchmarks (we test MMLU and ARC), thus enabling a time and cost-effective estimation of LLM capabilities. The approach translates genetic concepts …

abstract algorithms arxiv benchmarks construct cs.cl cs.lg explore families finetuning language language models large language large language models llm paper performance performances q-bio.pe relationships set type