Assessing the Efficacy of Grammar Error Correction: A Human Evaluation Approach in the Japanese Context

arXiv:2402.18101v1 Announce Type: new
Abstract: In this study, we evaluated the performance of the state-of-the-art sequence tagging grammar error detection and correction model (SeqTagger) using Japanese university students' writing samples. With an automatic annotation toolkit, ERRANT, we first evaluated SeqTagger's performance on error correction with human expert correction as the benchmark. Then a human-annotated approach was adopted to evaluate Seqtagger's performance in error detection using a subset of the writing dataset. Results indicated a precision of 63.66% and a recall …

abstract annotation art arxiv context cs.cl detection error error correction evaluation grammar grammar error correction human japanese performance samples s performance state students study tagging toolkit type university writing