From News to Summaries: Building a Hungarian Corpus for Extractive and Abstractive Summarization

arXiv:2404.03555v1 Announce Type: new
Abstract: Training summarization models requires substantial amounts of training data. However for less resourceful languages like Hungarian, openly available models and datasets are notably scarce. To address this gap our paper introduces HunSum-2 an open-source Hungarian corpus suitable for training abstractive and extractive summarization models. The dataset is assembled from segments of the Common Crawl corpus undergoing thorough cleaning, preprocessing and deduplication. In addition to abstractive summarization we generate sentence-level labels for extractive summarization using sentence …

abstract arxiv building cs.cl data datasets gap however hungarian languages openly paper summarization training training data type