[D] Is in-context learning outperforming supervised learning on your problems?

I think in-context learning is obviously awesome for fast prototyping, and I understand that there will be use-cases where it's a good enough solution. And obviously LLMs won't be beaten on generative tasks.

But let's say you're doing some relatively boring prediction problem, like text classification or a custom entity recognition problem, and you have a few thousand training samples. From a technical standpoint, I can't see why in-context learning should be better in this situation than training a task-specific …

cases classification context generative good llms machinelearning prediction prototyping solution supervised learning text text classification think