ChipNeMo: Domain-Adapted LLMs for Chip Design

arXiv:2311.00176v4 Announce Type: replace
Abstract: ChipNeMo aims to explore the applications of large language models (LLMs) for industrial chip design. Instead of directly deploying off-the-shelf commercial or open-source LLMs, we instead adopt the following domain adaptation techniques: domain-adaptive tokenization, domain-adaptive continued pretraining, model alignment with domain-specific instructions, and domain-adapted retrieval models. We evaluate these methods on three selected LLM applications for chip design: an engineering assistant chatbot, EDA script generation, and bug summarization and analysis. Our evaluations demonstrate that domain-adaptive …

abstract alignment applications arxiv chip chip design chipnemo commercial cs.cl design domain domain adaptation explore industrial language language models large language large language models llms pretraining retrieval tokenization type