Understanding the hidden state of the middle layer of the Transformer model

I'm conducting a study on how to improve the factual accuracy of generative language models, for large language models, different Transformer layers will focus on different linguistic features. So I was thinking that if I had a 40-layer model, maybe some of those layers would contribute more to the facticity of the final output of the model. Maybe I can train a simple logistic regression model to detect how different layers behave with respect to facticity by analyzing the hidden …

accuracy features focus generative hidden language language models languagetechnology large language large language models layer state study thinking transformer transformer model understanding will