[R] From Words to Numbers: Your Large Language Model Is Secretly A Capable Regressor When Given In-Context Examples

**Paper**: [https://arxiv.org/abs/2404.07544](https://arxiv.org/abs/2404.07544)

**Code**: [https://github.com/robertvacareanu/llm4regression](https://github.com/robertvacareanu/llm4regression)

**Abstract**:

>We analyze how well pre-trained large language models (e.g., Llama2, GPT-4, Claude 3, etc) can do linear and non-linear regression when given in-context examples, without any additional training or gradient updates. Our findings reveal that several large language models (e.g., GPT-4, Claude 3) are able to perform regression tasks with a performance rivaling (or even outperforming) that of traditional supervised methods such as Random Forest, Bagging, or Gradient Boosting. For example, on the challenging Friedman …

abstract analyze claude claude 3 context etc examples gpt gpt-4 gradient language language models large language large language models linear linear regression llama2 machinelearning non-linear performance regression tasks training updates