June 29, 2022, 11:44 a.m. | /u/optimized-adam

Machine Learning www.reddit.com

What differences in model performance, speed, memory etc. can I expect between choosing BF16 or FP16 for mixed precision training? Is BF16 faster / consumes less memory, since I have seen people say it is "more suitable for Deep Learning". Why is that the case?

difference machinelearning mixed precision training

Data Architect

@ University of Texas at Austin | Austin, TX

Data ETL Engineer

@ University of Texas at Austin | Austin, TX

Lead GNSS Data Scientist

@ Lurra Systems | Melbourne

Senior Machine Learning Engineer (MLOps)

@ Promaton | Remote, Europe

Data Analytics & Insight Specialist, Customer Success

@ Fortinet | Ottawa, ON, Canada

Account Director, ChatGPT Enterprise - Majors

@ OpenAI | Remote - Paris