[D] Measuring software engineering productivity before & after incorporation of LLMs into workflows

I work for a software engineering company (outsourcing), and our management wants to measure the productivity impact of Large Language Models on daily engineering work (including software engineering, data engineering, quality assurance, etc.). The end goal is to obtain some raw metrics (such as "Team X performs 30% better when using LLMs") to present to clients, intending to demonstrate that we outperform competitors who do not use LLMs.

My viewpoint is that accurately measuring this impact is challenging because LLM …

daily data data engineering engineering etc impact language language models large language large language models llms machinelearning management measuring metrics outsourcing productivity quality quality assurance raw software software engineering team work workflows