How to

Genshin Impact! Fine-tuning CLIP for anime search

Using Jina and Finetuner to improve search results with CLIP

Thanks to Jina AI’s Bo Wang for creating this finetuning notebook!

Today let’s build a search-anime system. We will use text as our query and get images as result. For this we would usually need to manually annotate the image with some tags, often referred to as TBIR (Text/Tag-based Image Retrieval). And for this example, we will use OpenAI CLIP.

CLIP is a powerful embedding model that outputs the similarity between text and images. And while it delivers great results, its zero-shot capability needs to be improved on domain data.

Open the notebook to learn how to finetune CLIP on Genshin Impact data.