all AI news
The Effect of Data Partitioning Strategy on Model Generalizability: A Case Study of Morphological Segmentation
April 16, 2024, 4:51 a.m. | Zoey Liu, Bonnie J. Dorr
cs.CL updates on arXiv.org arxiv.org
Abstract: Recent work to enhance data partitioning strategies for more realistic model evaluation face challenges in providing a clear optimal choice. This study addresses these challenges, focusing on morphological segmentation and synthesizing limitations related to language diversity, adoption of multiple datasets and splits, and detailed model comparisons. Our study leverages data from 19 languages, including ten indigenous or endangered languages across 10 language families with diverse morphological systems (polysynthetic, fusional, and agglutinative) and different degrees of …
abstract adoption arxiv case case study challenges clear cs.cl data data partitioning datasets diversity evaluation face language limitations multiple partitioning segmentation strategies strategy study type work
More from arxiv.org / cs.CL updates on arXiv.org
Jobs in AI, ML, Big Data
Data Engineer
@ Lemon.io | Remote: Europe, LATAM, Canada, UK, Asia, Oceania
Artificial Intelligence – Bioinformatic Expert
@ University of Texas Medical Branch | Galveston, TX
Lead Developer (AI)
@ Cere Network | San Francisco, US
Research Engineer
@ Allora Labs | Remote
Ecosystem Manager
@ Allora Labs | Remote
Founding AI Engineer, Agents
@ Occam AI | New York