TextEE: Benchmark, Reevaluation, Reflections, and Future Challenges in Event Extraction

arXiv:2311.09562v2 Announce Type: replace
Abstract: Event extraction has gained considerable interest due to its wide-ranging applications. However, recent studies draw attention to evaluation issues, suggesting that reported scores may not accurately reflect the true performance. In this work, we identify and address evaluation challenges, including inconsistency due to varying data assumptions or preprocessing steps, the insufficiency of current evaluation frameworks that may introduce dataset or data split bias, and the low reproducibility of some previous approaches. To address these challenges, …

abstract applications arxiv attention benchmark challenges cs.cl data evaluation event extraction future identify performance reflections studies true type work