[R] Stealing Part of a Production Language Model

We introduce the first model-stealing attack that extracts precise, nontrivial information from black-box production language models like Ope- nAI’s ChatGPT or Google’s PaLM-2. Specifi- cally, our attack recovers the embedding projec- tion layer (up to symmetries) of a transformer model, given typical API access. For under $20 USD, our attack extracts the entire projection ma- trix of OpenAI’s ada and babbage language models. We thereby confirm, for the first time, that these black-box models have a hidden dimension of 1024 …

api box chatgpt embedding google information language language model language models layer machinelearning palm part production stealing transformer transformer model usd