From Words to Numbers: Your Large Language Model Is Secretly A Capable Regressor When Given In-Context Examples

arXiv:2404.07544v1 Announce Type: new
Abstract: We analyze how well pre-trained large language models (e.g., Llama2, GPT-4, Claude 3, etc) can do linear and non-linear regression when given in-context examples, without any additional training or gradient updates. Our findings reveal that several large language models (e.g., GPT-4, Claude 3) are able to perform regression tasks with a performance rivaling (or even outperforming) that of traditional supervised methods such as Random Forest, Bagging, or Gradient Boosting. For example, on the challenging Friedman …

abstract analyze arxiv claude claude 3 context cs.ai cs.cl etc examples gpt gpt-4 gradient language language model language models large language large language model large language models linear linear regression llama2 non-linear numbers regression training type updates words