Almost Everything You Want to Know About Partition Size of Dask Dataframes

And how to utilize it effectively in XGBoost model

Recently, my colleagues and I have been working on a big high-loaded service that utilizes the Xgboost machine learning model and Dask as the tool for distributed data processing and forecast generating. Here I would like to share findings that we have been able to maximize the use of Dask for the purpose of data preparation and ML model fitting.

What is Dask?

Dask is a library …

author big colleagues dask data dataframes data processing distributed distributed computing distributed data everything forecast image machine machine learning machine learning model parallel-computing processing programming python service tool xgboost