Tur[k]ingBench: A Challenge Benchmark for Web Agents

arXiv:2403.11905v1 Announce Type: cross
Abstract: Recent chatbots have demonstrated impressive ability to understand and communicate in raw-text form. However, there is more to the world than raw text. For example, humans spend long hours of their time on web pages, where text is intertwined with other modalities and tasks are accomplished in the form of various complex interactions. Can state-of-the-art multi-modal models generalize to such complex domains?
To address this question, we introduce TurkingBench, a benchmark of tasks formulated as …

abstract agents arxiv benchmark challenge chatbots cs.ai cs.cl cs.cv cs.hc example form however humans raw spend tasks text type web world