Web: https://www.reddit.com/r/MachineLearning/comments/vfutwe/d_initialize_model_weights_based_on_a_trained/

June 19, 2022, 1:07 p.m. | /u/muwnd

Machine Learning reddit.com

Is there any existing work that explores how trained weights of a small model (e.g. Bert-base) can be used for a "smart" initialization of a larger model (bert-large) such that the training is more efficient?

I couldn't really find such work but I guess I just used the wrong search terms. How is this line of research typically called?

machinelearning model on

Machine Learning Researcher - Saalfeld Lab

@ Howard Hughes Medical Institute - Chevy Chase, MD | Ashburn, Virginia

Project Director, Machine Learning in US Health

@ ideas42.org | Remote, US

Data Science Intern

@ NannyML | Remote

Machine Learning Engineer NLP/Speech

@ Play.ht | Remote

Research Scientist, 3D Reconstruction

@ Yembo | Remote, US

Clinical Assistant or Associate Professor of Management Science and Systems

@ University at Buffalo | Buffalo, NY