May 10, 2024, 4:46 a.m. | Adian Liusie, Vatsal Raina, Yassir Fathullah, Mark Gales

cs.CL updates on arXiv.org arxiv.org

arXiv:2405.05894v1 Announce Type: new
Abstract: LLM-as-a-judge approaches are a practical and effective way of assessing a range of text tasks, aligning with human judgements especially when applied in a comparative assessment fashion. However, when using pairwise comparisons to rank a set of candidates the computational costs scale quadratically with the number of candidates, which can have practical limitations. This paper introduces a Product of Expert (PoE) framework for efficient LLM Comparative Assessment. Here individual comparisons are considered experts that provide …

abstract arxiv assessment computational costs cs.cl experts fashion framework however human judge llm practical product scale set tasks text type

Doctoral Researcher (m/f/div) in Automated Processing of Bioimages

@ Leibniz Institute for Natural Product Research and Infection Biology (Leibniz-HKI) | Jena

Seeking Developers and Engineers for AI T-Shirt Generator Project

@ Chevon Hicks | Remote

Software Engineer for AI Training Data (School Specific)

@ G2i Inc | Remote

Software Engineer for AI Training Data (Python)

@ G2i Inc | Remote

Software Engineer for AI Training Data (Tier 2)

@ G2i Inc | Remote

Manager, Business Intelligence

@ Revlon | New York City, United States