Multi-view Content-aware Indexing for Long Document Retrieval

arXiv:2404.15103v1 Announce Type: new
Abstract: Long document question answering (DocQA) aims to answer questions from long documents over 10k words. They usually contain content structures such as sections, sub-sections, and paragraph demarcations. However, the indexing methods of long documents remain under-explored, while existing systems generally employ fixed-length chunking. As they do not consider content structures, the resultant chunks can exclude vital information or include irrelevant content. Motivated by this, we propose the Multi-view Content-aware indexing (MC-indexing) for more effective long …

abstract arxiv cs.cl document documents however indexing question question answering questions retrieval systems type view words