BooookScore: A systematic exploration of book-length summarization in the era of LLMs

arXiv:2310.00785v3 Announce Type: replace-cross
Abstract: Summarizing book-length documents (>100K tokens) that exceed the context window size of large language models (LLMs) requires first breaking the input document into smaller chunks and then prompting an LLM to merge, update, and compress chunk-level summaries. Despite the complexity and importance of this task, it has yet to be meaningfully studied due to the challenges of evaluation: existing book-length summarization datasets (e.g., BookSum) are in the pretraining data of most public LLMs, and existing …

abstract arxiv book breaking complexity context context window cs.ai cs.cl cs.lg document documents exploration importance language language models large language large language models llm llms merge prompting summarization summarizing tokens type update