[D] What are the heuristics for setting good baselines when experimenting?

Hello everyone. Master's student here, looking for advice about experimentation.

Let's say you want to show that some architectural change or tweak to training technique can improve model performance relative to a "vanilla" baseline. For example, [this paper](https://openaccess.thecvf.com/content_CVPR_2020/papers/Singh_Filter_Response_Normalization_Layer_Eliminating_Batch_Dependence_in_the_Training_CVPR_2020_paper.pdf) introduces a normalization technique and measures its effectiveness vs batchnorm on Imagenet and COCO. Maybe you're trying to find such an improvement somewhere else.

If you aren't trying to push the state of the art, but rather trying to show the validity …

good heuristics machinelearning