Blind leading the blind

Recently my ML model has been under scrutiny for inaccuracy for one the sales channel predictions. The model predicts monthly proportional volume. It works great on channels with consistent volume flows (higher volume channels), not so great when ordering patterns are not consistent. My boss wants to look at model validation, that’s what was said. When creating the model initially we did cross validation, looked at MSE, and it was known that low volume channels are not as accurate. I’m …

blind boss channels consistent datascience look patterns predictions sales validation