Speed up your Training with Mixed Precision on GPUs and TPUs in TensorFlow

Speed up your TensorFlow Training with Mixed Precision on GPUs and TPUs

A Simple Step-by-Step Guide

In this post, I will show you, how you can speed up your training on a suitable GPU or TPU using mixed precision bit representation. First, I will briefly introduce different floating-point formats. Secondly, I will show you step-by-step how you can implement the significant speed-up yourself using TensorFlow. A detailed documentation can be found in [1].

Outline

1. Introduction
2. Activating Mixed …

gpu gpus machine learning mixed precision tensorflow tpu tpus training