Web: https://www.reddit.com/r/LanguageTechnology/comments/ulrhc4/how_do_i_check_if_two_connl_scores_are/

May 9, 2022, 1:39 p.m. | /u/omicronorcimo

Natural Language Processing reddit.com

I have an evaluation batch of 6 items. I ran this on both my system output and a human annotator's results against the gold labels.

The human average CoNLL scores are

1

0.8475281455

0.5003718249

0.7957433734

0.4920068314

0.7339975187

and the system average CoNLL scores are

0.8526315789

0.7730431201

0.7143441494

0.6639100561

0.7212088202

0.8165547658

The system mean is marginally higher at 0.7569487484 vs 0.7282746157, however the variances are 0.004943160553 and 0.04008060433 respectively. The latter variance makes me guess that I don't have enough data …

languagetechnology

More from reddit.com / Natural Language Processing

Director, Applied Mathematics & Computational Research Division

@ Lawrence Berkeley National Lab | Berkeley, Ca

Business Data Analyst

@ MainStreet Family Care | Birmingham, AL

Assistant/Associate Professor of the Practice in Business Analytics

@ Georgetown University McDonough School of Business | Washington DC

Senior Data Science Writer

@ NannyML | Remote

Director of AI/ML Engineering

@ Armis Industries | Remote (US only), St. Louis, California

Digital Analytics Manager

@ Patagonia | Ventura, California