GigaPevt: Multimodal Medical Assistant

arXiv:2402.16654v1 Announce Type: cross
Abstract: Building an intelligent and efficient medical assistant is still a challenging AI problem. The major limitation comes from the data modality scarceness, which reduces comprehensive patient perception. This demo paper presents the GigaPevt, the first multimodal medical assistant that combines the dialog capabilities of large language models with specialized medical models. Such an approach shows immediate advantages in dialog quality and metric performance, with a 1.18\% accuracy improvement in the question-answering task.

abstract arxiv assistant building capabilities cs.ai cs.cl cs.hc data demo dialog intelligent language language models large language large language models major medical multimodal paper patient perception type