JAX vs PyTorch: Automatic Differentiation for XGBoost

Perform rapid loss-function prototypes to take full advantage of XGBoost’s flexibility

Motivation

Running XGBoost with custom loss functions can greatly increase classification/regression performance in certain applications. Being able to quickly test many different loss functions is key in time-critical research environments. Thus, manual differentiation is not always feasible (and sometimes even prone to human errors, or numerical instability).

Automatic differentiation allows us to automatically get the derivatives of a function, given its calculation. It …

differentiation jax loss-function machine learning pytorch xgboost