Classification and Clustering of Sentence-Level Embeddings of Scientific Articles Generated by Contrastive Learning

arXiv:2404.00224v1 Announce Type: new
Abstract: Scientific articles are long text documents organized into sections, each describing aspects of the research. Analyzing scientific production has become progressively challenging due to the increase in the number of available articles. Within this scenario, our approach consisted of fine-tuning transformer language models to generate sentence-level embeddings from scientific articles, considering the following labels: background, objective, methods, results, and conclusion. We trained our models on three datasets with contrastive learning. Two datasets are from the …

abstract articles arxiv become classification clustering cs.cl documents embeddings fine-tuning generated language production research scientific text transformer type