Raven - RWKV-7B RNN's LLM Strikes Back

HF Spaces Demo: https://huggingface.co/spaces/BlinkDL/Raven-RWKV-7B

In this video I look at Raven RWKV-7B which is a very interesting LLM based on a RNN type model for inference. It allows for longer context generation and after finetuning on the Alpaca Dataset does a very good as an Instruction tuned LLM.

Github: https://github.com/BlinkDL/RWKV-LM
Overview Blog post: https://johanwind.github.io/2023/03/23/rwkv_overview.html
Basic Colab: https://colab.research.google.com/drive/1F7tZoPZaWJf1fsCmZ5tjw6sYHiFOYVWM#scrollTo=9zsbY4i90jq-
Chat Colab: https://colab.research.google.com/github/harrisonvanderbyl/rwkvstic/blob/master/notebooks/chatbot.ipynb#scrollTo=QEKvfCrZ_Axl

alpaca context dataset finetuning good inference llm look rnn type video