Comparison of Current Approaches to Lemmatization: A Case Study in Estonian

arXiv:2404.15003v1 Announce Type: new
Abstract: This study evaluates three different lemmatization approaches to Estonian -- Generative character-level models, Pattern-based word-level classification models, and rule-based morphological analysis. According to our experiments, a significantly smaller Generative model consistently outperforms the Pattern-based classification model based on EstBERT. Additionally, we observe a relatively small overlap in errors made by all three models, indicating that an ensemble of different approaches could lead to improvements.

abstract analysis arxiv case case study classification classification model comparison cs.cl current generative lemmatization observe study type word