Intro to text-image (multi-modal) ML with OpenAI's CLIP

Hi all, I created a [walkthrough](https://towardsdatascience.com/quick-fire-guide-to-multi-modal-ml-with-openais-clip-2dad7e398ac0?sk=89bb2d8b8e583ed109d8a05e00366645) (and [video](https://youtu.be/989aKUVBfbk)) demoing how to use the text and image embeddings of OpenAI's CLIP. CLIP is a multi-modal model that uses a typical text transformer for text embeddings and a vision transformer (ViT, alt version uses Resnet) for image embeddings. During pretraining CLIP learns to place (image, text) pairs into the same vector space. The result is a cool off-the-shelf model that can perform tasks across image and text data.

When I started using …

clip image languagetechnology ml openai text text-image