Which Modality should I use -- Text, Motif, or Image? : Understanding Graphs with Large Language Models

arXiv:2311.09862v2 Announce Type: replace
Abstract: Our research integrates graph data with Large Language Models (LLMs), which, despite their advancements in various fields using large text corpora, face limitations in encoding entire graphs due to context size constraints. This paper introduces a new approach to encoding a graph with diverse modalities, such as text, image, and motif, coupled with prompts to approximate a graph's global connectivity, thereby enhancing LLMs' efficiency in processing complex graph structures. The study also presents GraphTMI, a …

abstract arxiv constraints context cs.cl cs.si data encoding face fields graph graph data graphs image language language models large language large language models limitations llms motif paper research text type understanding