iScore: Visual Analytics for Interpreting How Language Models Automatically Score Summaries

arXiv:2403.04760v1 Announce Type: cross
Abstract: The recent explosion in popularity of large language models (LLMs) has inspired learning engineers to incorporate them into adaptive educational tools that automatically score summary writing. Understanding and evaluating LLMs is vital before deploying them in critical learning environments, yet their unprecedented size and expanding number of parameters inhibits transparency and impedes trust when they underperform. Through a collaborative user-centered design process with several learning engineers building and deploying summary scoring LLMs, we characterized fundamental …

abstract analytics arxiv cs.ai cs.cy cs.hc cs.lg educational engineers environments language language models large language large language models llms summary them tools type understanding visual visual analytics vital writing