[P] Does it make sense to train a model twice on separate datasets?

TLDR: If I use transfer learning to train a model on one general face dataset, does it make sense to then train it again on a more specific set of faces (a set more similar to my test set) or will the training of the second dataset just overwrite everything learnt from the more general dataset?

So I have the task of using a CNN for facial recognition. So I am using it for the classification of faces to different …

datasets machinelearning sense