[D] Reproducing and Comparing Models from Research - Best Practices?

At my work, I'm spending a ton of time reproducing research papers and trying to apply them to our use case (medical image analysis).

Usually, that's a big hassle, and even papers-with-code often reveal that it doesn't run without extensive edits. And as soon as they run, results are only good when setting a specific random seed...

And then after applying it to our use case, I realized that the performance increase actually doesn't come from a new model, but …

analysis apply best practices big case code image machinelearning medical papers practices research research papers results spending them work