Retrieval Augmented Chest X-Ray Report Generation using OpenAI GPT models. (arXiv:2305.03660v1 [cs.CL])

We propose Retrieval Augmented Generation (RAG) as an approach for automated
radiology report writing that leverages multimodally aligned embeddings from a
contrastively pretrained vision language model for retrieval of relevant
candidate radiology text for an input radiology image and a general domain
generative model like OpenAI text-davinci-003, gpt-3.5-turbo and gpt-4 for
report generation using the relevant radiology text retrieved. This approach
keeps hallucinated generations under check and provides capabilities to
generate report content in the format we desire leveraging the …

arxiv automated embeddings general generative gpt gpt-3 gpt-3.5 image language language model openai openai gpt radiology ray report retrieval retrieval augmented generation text vision writing x-ray