[D] Fine-tuning LLM on company data

Hey Redditors, I was looking into fine-tuning some open-source LLMs like Llama 2 or Falcon with our company data as a fun project. I was thinking about using some Slack channels, ZenDesk Tickets and perhaps Github/Confluence data

I was wondering two things:
1. How have your experiences been with PEFT methods in practice? Anything I should be aware of compared to regular fine-tuning?
2. Which model size would you recommend for a relatively small sized company (60 people) and how …

confluence data falcon fine-tuning fun github hey llama llama 2 llm llms machinelearning practice project slack thinking zendesk