Building an Efficient AI Platform for Data Preprocessing and Model Training

Lei Li, AI Platform Lead, and Zifan Ni, Senior Software Engineer from Bilibili, share how they applied Alluxio to their AI platform to increase training efficiency, as well as best practices including technical architecture and specific tuning tips. They say they came across four major challenges before adopting the AI platform, including the container crash and the need to restart the container to download the data again. They also share their best practices and best practices for training AI jobs …

ai ai platform building data data management data preprocessing machine learning ml model-training platform training