all AI news
[R] Segment Any Text: A Universal Approach for Robust, Efficient and Adaptable Sentence Segmentation
June 28, 2024, 9:22 a.m. | /u/markus_583
Machine Learning www.reddit.com
**Paper:** [**https://arxiv.org/abs/2406.16678**](https://arxiv.org/abs/2406.16678)
**Code:** [https://github.com/segment-any-text/wtpsplit](https://github.com/segment-any-text/wtpsplit)
https://preview.redd.it/6frvmpc36a9d1.png?width=1849&format=png&auto=webp&s=08c9769384d63bfd3ad786b0259f1dd4a97d4bce
**Abstract:**
>Segmenting text into sentences plays an early and crucial role in many NLP systems. This is commonly achieved by using rule-based or statistical methods relying on lexical features such as punctuation. Although some recent works no longer exclusively rely on punctuation, we find that no prior method achieves all of (i) robustness to missing punctuation, (ii) effective adaptability to new …
abstract machinelearning robust segment segmentation text universal
More from www.reddit.com / Machine Learning
[D] Why do DINO models use augmentations for the teacher encoder?
1 day, 14 hours ago |
www.reddit.com
Jobs in AI, ML, Big Data
Junior Senior Reliability Engineer
@ NielsenIQ | Bogotá, Colombia
[Job - 15712] Vaga Afirmativa para Mulheres - QA (Automation), SR
@ CI&T | Brazil
Production Reliability Engineer, Trade Desk
@ Jump Trading | Sydney, Australia
Senior Process Engineer, Prenatal
@ BillionToOne | Union City and Menlo Park, CA
Senior Scientist, Sustainability Science and Innovation
@ Microsoft | Redmond, Washington, United States
Data Scientist
@ Ford Motor Company | Chennai, Tamil Nadu, India