[D] Play the game or do the science in ML publishing.

I see a lot of template vision papers out of big-techs. The recipie is always the same, make a small change to existing work, call it novel, run hyperparameter search like crazy, show results on 20 datasets. (Mind you these models most likely won’t work zero-shot, all the numbers they show are fine tuned for this dataset). These papers always get in!

I feel like this is setting very unrealistic expectation for reviewers and expect the same level of experimentatal …

big call change datasets game hyperparameter machinelearning mind novel numbers papers publishing science search show small techs template vision work zero-shot