[P] SimpleGEMM: Fast and minimal tensor core matrix multiplication in CUDA

Hello all! Sharing my side project here: [https://github.com/andylolu2/simpleGEMM](https://github.com/andylolu2/simpleGEMM) !

This is an *extremely* minimalistic but fast implementation of matrix multiplication in CUDA. The source code is a single, 200-line CUDA/C++ file which implements fp16 tensor core matrix multiplication, optimised for Turing (SM75) architecture. The goal is to:

1. Write a matmul kernel that does not sacrifice performance. In fact, it's faster than PyTorch/CuBLAS if you [test it on a T4 in Colab](https://colab.research.google.com/github/andylolu2/simpleGEMM/blob/master/colab/simpleGEMM.ipynb)!
2. Make it hackable for new purposes. For …

architecture code core cuda example file fp16 implementation line machinelearning matrix matrix multiplication tensor turing