[P] OpenGL-based inference engine

I created an OpenGL/OpenGLES based inference framework a while back which is rather GPU-agnostic and might be a good option for distributing multi-platform ML solutions for platforms ranging from Android over desktop to WebGL(2). Quite recently I added support for LLMs to that (restricted to 4-bit quantized Llama models for now).

The LLM-enabled fork can be found [here](https://github.com/mtnwrw/fyusenet) (compileable sample code inside).

Maybe someone finds this useful. Also looking for collaborators to extend the functionality.

​

android framework good gpu inference llama llama models llms machinelearning platform platforms solutions support webgl