J-CRe3: A Japanese Conversation Dataset for Real-world Reference Resolution

arXiv:2403.19259v1 Announce Type: new
Abstract: Understanding expressions that refer to the physical world is crucial for such human-assisting systems in the real world, as robots that must perform actions that are expected by users. In real-world reference resolution, a system must ground the verbal information that appears in user interactions to the visual information observed in egocentric views. To this end, we propose a multimodal reference resolution task and construct a Japanese Conversation dataset for Real-world Reference Resolution (J-CRe3). Our …

abstract arxiv conversation cs.cl dataset human information interactions japanese reference resolution robots systems type understanding verbal world