[D] How do static weights in LLMs generate such dynamic behavior?

This is a bit of a complex topic, so I feel a simple discussion post may not be the best place to fully flesh it out - but in the context of few-shot learning in LLMs (Large Language Models), we observe static/unchanged/un-updated weights being able to infer patterns and sometimes even learn complex tasks.

I was wondering why forward passing *works* *so well* \- by all means, we should've been updating weights with new information but forward passing seems to …

behavior llms machinelearning