Hugging Face’s Cosmopedia Hopes To Reshape Pre-Training Data

Since the beginning of AI models, the creation of datasets for supervised and instruction-tuning of AI models relied on the painstaking process of hiring human annotators — a method not only time-consuming but also prohibitively expensive.

But it seems that Hugging Face is hoping to change all of that with Cosmopedia, a synthetic data creation tool that can cover hundreds of subjects with a duplicate content rate of less than 1%. With over 25 billion tokens and 30 million files, …

ai models artificial intelligence change data data science datasets face hiring hugging face human open-data open source pre-training process reshape synthetic training training data