Simple Hack for Transformers against Heavy Long-Text Classification on a Time- and Memory-Limited GPU Service

arXiv:2403.12563v1 Announce Type: new
Abstract: Many NLP researchers rely on free computational services, such as Google Colab, to fine-tune their Transformer models, causing a limitation for hyperparameter optimization (HPO) in long-text classification due to the method having quadratic complexity and needing a bigger resource. In Indonesian, only a few works were found on long-text classification using Transformers. Most only use a small amount of data and do not report any HPO. In this study, using 18k news articles, we investigate …

abstract arxiv bigger classification colab complexity computational cs.ai cs.cl free google gpu hack hyperparameter memory nlp optimization researchers service services simple text text classification transformer transformer models transformers type