[D] Will continuous model development inevitably lead to data leakage and overfitting?

In a naive ML project one might split the dataset into train - test to train a model and get a sense of its performance, either during or after training. The problem arises when you want to improve the model continuously. The test set will no longer give an unbiased measure of model performance since you have improved your model to achieve as good a performance as possible on the test set.

So we introduce the validation set (dataset -> …

continuous data data leakage development machinelearning model development overfitting