GP-MoLFormer: A Foundation Model For Molecular Generation

arXiv:2405.04912v1 Announce Type: cross
Abstract: Transformer-based models trained on large and general purpose datasets consisting of molecular strings have recently emerged as a powerful tool for successfully modeling various structure-property relations. Inspired by this success, we extend the paradigm of training chemical language transformers on large-scale chemical datasets to generative tasks in this work. Specifically, we propose GP-MoLFormer, an autoregressive molecular string generator that is trained on more than 1.1B chemical SMILES. GP-MoLFormer uses a 46.8M parameter transformer decoder model …

abstract arxiv cs.lg datasets foundation foundation model general generative language modeling paradigm physics.chem-ph property q-bio.bm relations scale strings success tasks tool training transformer transformer-based models transformers type