Enhancing Human-Computer Interaction in Chest X-ray Analysis using Vision and Language Model with Eye Gaze Patterns

arXiv:2404.02370v1 Announce Type: new
Abstract: Recent advancements in Computer Assisted Diagnosis have shown promising performance in medical imaging tasks, particularly in chest X-ray analysis. However, the interaction between these models and radiologists has been primarily limited to input images. This work proposes a novel approach to enhance human-computer interaction in chest X-ray analysis using Vision-Language Models (VLMs) enhanced with radiologists' attention by incorporating eye gaze data alongside textual prompts. Our approach leverages heatmaps generated from eye gaze data, overlaying them …

abstract analysis arxiv computer cs.ai cs.cl cs.cv diagnosis however human human-computer interaction images imaging language language model medical medical imaging novel patterns performance ray tasks type vision work x-ray