[D] Has anyone looked in active learning or similar techniques for LLM fine-tuning?

I was wondering if anyone has looked into data sampling or active learning techniques to fine-tune LLMs. Using PEFT methods like LoRA we can use much fewer samples for fine-tuning. But the training data still requires some sort of labels or responses for questions. I found these two datasets that seem commonly used (Alpaca and OASST1). Both seem rather small.

[Alpaca](https://huggingface.co/datasets/tatsu-lab/alpaca) has 52k instructions.
[OpenAssistant Conversations Dataset (OASST1)](https://huggingface.co/datasets/OpenAssistant/oasst1) has 160k messages that result in "in over 10,000 fully annotated conversation …

active learning data fine-tuning labels llm llms lora machinelearning questions responses sampling training training data