[P] Llama2 7B and 13B Chat Completion for Kaggle TPU v3-8

Hi guys, I have made some modifications to the Llama2 repository to utilize the TPU v3-8 hardware, so it can perform Llama2 7B (and even 13B) chat completion inference without graph recompilation. It is still slower than the Nvidia P100 when generating text with batch-size 1, not suitable for real-time inference but (TPU being TPU) shines well with batched text generation. I used it to generate large amount of texts for research purpose. Hope it benefits the community.

Here's the …

13b batch-size chat graph hardware inference kaggle llama2 machinelearning nvidia text tpu