[D] Mathematical aspects of tokenization

I recently made a video covering our recent work on the mathematical aspects of tokenization, specifically:
- formalization of tokenization as compression
- bounds of Byte-Pair Encoding optimality
- link between tokenization entropy and performance

I'd be very grateful for any feedback as I'm still learning how to make educational videos. Thank you!

https://youtu.be/yeEZpf4BlDA

compression educational encoding entropy feedback machinelearning performance tokenization video videos work