all AI news
On the validity of pre-trained transformers for natural language processing in the software engineering domain. (arXiv:2109.04738v2 [cs.SE] UPDATED)
cs.LG updates on arXiv.org arxiv.org
Transformers are the current state-of-the-art of natural language processing
in many domains and are using traction within software engineering research as
well. Such models are pre-trained on large amounts of data, usually from the
general domain. However, we only have a limited understanding regarding the
validity of transformers within the software engineering domain, i.e., how good
such models are at understanding words and sentences within a software
engineering context and how this improves the state-of-the-art. Within this
article, we shed …
arxiv engineering language language processing natural natural language natural language processing processing software software engineering transformers