Data Pipeline Training: Integrating AutoML to Optimize the Data Flow of Machine Learning Models

arXiv:2402.12916v1 Announce Type: new
Abstract: Data Pipeline plays an indispensable role in tasks such as modeling machine learning and developing data products. With the increasing diversification and complexity of Data sources, as well as the rapid growth of data volumes, building an efficient Data Pipeline has become crucial for improving work efficiency and solving complex problems. This paper focuses on exploring how to optimize data flow through automated machine learning methods by integrating AutoML with Data Pipeline. We will discuss …

abstract arxiv automl building complexity cs.ai cs.lg data data flow data pipeline data products data sources diversification flow growth machine machine learning machine learning models modeling pipeline products role tasks training type