Is there a methodologically sound way to derive causal insights from a predictive model (e.g. xgboost) using simulation?

Use case: I have an xgboost model that predicts customer purchase decisions (Y/N) based on 50+ features. The product team is interested in learning about the effect of some specific features (5 in total) on specific users. Running an experiment isn't possible. So, thinking, methodology-wise, would it be a good idea to run some simulations and report out of T cases S would result in a positive outcome? Also, if it's a sound approach, can you please point me to …

case customer datascience decisions experiment features insights predictive product purchase running simulation sound team total xgboost