First impressions: GPU + GCP Batch

Weihao and I have been working on programmatic benchmarks for DeepCell on Google Batch.

We tried Vertex AI custom training jobs but ran into an issue with service accounts. It appears that the training job ran on the expected(?) service account, but in an unexpected project. We didn't track down how to give that project's user access to BigQuery. We also figured that we may want to run the container a little closer to the metal (not a VM …

ai benchmarks cloud gcp google googlecloud gpu impressions issue job jobs programmatic project ran service training vertex