Small But Funny: A Feedback-Driven Approach to Humor Distillation

arXiv:2402.18113v1 Announce Type: new
Abstract: The emergence of Large Language Models (LLMs) has brought to light promising language generation capabilities, particularly in performing tasks like complex reasoning and creative writing. Consequently, distillation through imitation of teacher responses has emerged as a popular technique to transfer knowledge from LLMs to more accessible, Small Language Models (SLMs). While this works well for simpler tasks, there is a substantial performance gap on tasks requiring intricate language comprehension and creativity, such as humor generation. …

abstract arxiv capabilities creative cs.ai cs.cl distillation emergence feedback funny humor knowledge language language generation language models large language large language models light llms popular reasoning responses small tasks through transfer type writing