Parallel Inference of HuggingFace Transformers on CPUs

An introduction to multiprocessing predictions of large machine learning and deep learning models

The current trend in AI research is moving towards the development of increasingly larger Deep Learning models, which constantly surpass each other in terms of performance. Recent examples in Natural Language Processing (NLP) are GPT-3, XLNet or the classical BERT transformer models. While the ever-improving results are inspiring researchers enthusiastically to research even larger models, this development also has a …

hugging face nlp parallel-computing text classification transformers