Accelerated Distributed Training with TensorFlow on Google’s TPU

Understand your Hardware to Optimize your Software

In this post I will show you the basic principles of tensor processing units (TPUs) from a hardware perspective and show you step-by-step how you can perform accelerated distributed training on a TPU using TensorFlow to train your own models.

Outline

1. Introduction
   1.1. Tensor Processing Units
   1.2. Distribution Strategies
2. Implementing Distributed Training on TPU with TensorFlow
   2.1. Hardware Detection
   2.2. Distributed Dataset
   2.3. …

distributed distributed systems google python tensorflow tpu training