Web: https://www.reddit.com/r/LanguageTechnology/comments/xh71yt/bertopic_for_longer_texts/

Sept. 18, 2022, 4:31 a.m. | /u/everydayislikefriday

Natural Language Processing reddit.com

I've been having great success using BERTopic to model topics from a short-document corpus. As promised, the topics are indeed way more interpretable than those extracted by other topic modeling algorithms, and the interface is easy and just works.

However, the same cannot be said of a corpus containing long documents (~30.000 characters). Given how fast the whole training phase is, I suspect they're actually being truncated (maybe this is obvious, given the well-known limitation of transformers).

So I'm trying …

languagetechnology

Research Scientists

@ ODU Research Foundation | Norfolk, Virginia

Embedded Systems Engineer (Robotics)

@ Neo Cybernetica | Bedford, New Hampshire

2023 Luis J. Alvarez and Admiral Grace M. Hopper Postdoc Fellowship in Computing Sciences

@ Lawrence Berkeley National Lab | San Francisco, CA

Senior Manager Data Scientist

@ NAV | Remote, US

Senior AI Research Scientist

@ Earth Species Project | Remote anywhere

Research Fellow- Center for Security and Emerging Technology (Multiple Opportunities)

@ University of California Davis | Washington, DC

Staff Fellow - Data Scientist

@ U.S. FDA/Center for Devices and Radiological Health | Silver Spring, Maryland

Staff Fellow - Senior Data Engineer

@ U.S. FDA/Center for Devices and Radiological Health | Silver Spring, Maryland

Senior Research Engineer, Applied Language

@ DeepMind | Mountain View, California, US

Machine Learning Engineer

@ Bluevine | Austin, TX

Lead Manager - Analytics & Data Science

@ Tide | India(Remote)

Machine Learning Engineer

@ Gtmhub | Indore, Madhya Pradesh, India