CleanAgent: Automating Data Standardization with LLM-based Agents

arXiv:2403.08291v1 Announce Type: new
Abstract: Data standardization is a crucial part in data science life cycle. While tools like Pandas offer robust functionalities, their complexity and the manual effort required for customizing code to diverse column types pose significant challenges. Although large language models (LLMs) like ChatGPT have shown promise in automating this process through natural language understanding and code generation, it still demands expert-level programming knowledge and continuous interaction for prompt refinement. To solve these challenges, our key idea …

abstract agents arxiv challenges chatgpt code column complexity cs.ai cs.lg cs.ma data data science diverse language language models large language large language models life life cycle llm llms pandas part robust science standardization tools type types