Scaling PyCaret with Spark (or Dask) through Fugue

Run PyCaret functions on each partition of data distributedly

PyCaret is a low code machine learning framework that automates a lot of parts of the machine learning pipeline. With just a few lines of code, several models can be trained on a dataset. In this post, we explore how to scale this capability by running several PyCaret training jobs in a distributed manner on Spark or Dask.

PyCaret Model Score Grid Example

First, we …

dask data science fugue machine learning pycaret scaling spark