[D] How should I go about implementing a custom quantization method?

Hey everyone! I’ve been doing research on quantizing llms and I have a couple of custom methods that I’d like to test out. Looking at existing implementations like Tim Dettmers’ bitsandbytes makes me feel as lost as ever. Looking at llama.cpp source hasn’t helped much either. Has anyone had experience with implementing and more importantly evaluating a custom quantization method? Please share any thoughts and if you have any questions please feel free to ask. Thnaks!

cpp hey llama llms lost machinelearning quantization research test