Why is it so hard to get neural networks to beat XGBoost on most small-medium tabular datasets?

I get it; XGBoost is really potent and easy to use while with DL theres a lot more that can go wrong tuning hyper parameters wise.

But i always assumed that whatever an ML model can do, a DL model with proper settings and sufficient regularization can also do as well even in low-medium size datsets (~hundreds to thoussnds of examples range).

I understand that DL models are more likely to overfit because theyre very very flexible espeically especially as …

datascience datasets easy medium networks neural networks parameters small tabular wise xgboost