Sparse Transformers | A Demo

How fast can BERT go with sparsity?

Here’s a Little Secret:

If you want to analyze how fast 19 sparse BERT models perform inference, you’ll only need a YAML file and 16GB of RAM to find out. And spoiler alert:

… they run on CPUs.

… and they’re super fast!

The latest feature from Neural Magic’s DeepSparse repo is the DeepSparse Server! And the objective of this article is to show not only how …

artificial intelligence data science deep learning demo machine learning technology transformers