[P] How to fine tune stable diffusion: how we made the text-to-pokemon model at Lambda

Here's a guide Justin Pinkney ( u/Buntworthy ) put together on how he trained the text-to-pokemon model that you've been seeing all over twitter.

[ the start sample looks like normal image, then start to get a Pokemon style, and eventually diverge from the original prompts as training continues.](https://i.redd.it/p5uwdnnp0oq91.gif)

The post: [https://lambdalabs.com/blog/how-to-fine-tune-stable-diffusion-how-we-made-the-text-to-pokemon-model-at-lambda/](https://lambdalabs.com/blog/how-to-fine-tune-stable-diffusion-how-we-made-the-text-to-pokemon-model-at-lambda/)

The model, code, and dataset are all available here:

* [Lambda Diffusers](https://github.com/LambdaLabsML/lambda-diffusers)
* [Captioned Pokémon dataset](https://huggingface.co/datasets/lambdalabs/pokemon-blip-captions)
* [Model weights in Diffusers format](https://huggingface.co/lambdalabs/sd-pokemon-diffusers)
* Hosted demo: [https://replicate.com/lambdal/text-to-pokemon](https://replicate.com/lambdal/text-to-pokemon)
* [Original …

diffusion lambda machinelearning pokemon stable diffusion text