CLIP Search powered by Jina AI

CLIP Search is a complete high-performance solution for multimodal image-text indexing and search, powered by Jina AI’s intuitive framework, eliminating the difficult engineering in deploying AI-driven services.

What is CLIP?

CLIP (doi:10.48550/arXiv.2103.00020) is a state-of-the-art neural network model for robust image and text processing. It takes as input images or texts and translates them into embeddings — vectors in a high-dimensional space laid out such that images containing similar kinds of objects, and texts describing similar objects, will be near to each other.

CLIP is an open source model, developed by OpenAI, that is integrated into Jina AI’s open source framework, making its powerful machinery for visual and textual reasoning easy to install, maintain and use.

Introducing CLIP Search

CLIP Search is a new feature of Jina’s CLIP-as-Service. In addition to calculating embeddings for images and texts, with CLIP Search, you can now:

• Index embeddings
• Filter data using indexed embeddings
• Perform queries and retrieve the results

CLIP Search also includes a gRPC backend with support for asynchronous and non-blocking calls, a Python library that provides client services, and a web front-end for immediate access to query facilities.

Features of CLIP-as-Service with CLIP Search

• Easy serialization and deserialization of objects, embeddings and queries using Jina’s intuitive DocArray library.
• A standard gRPC interface so that any kind of program can access and use CLIP Search.
• Cloud-native.
• Easy scaling.
• Out-of-the-box support for asynchronous and non-blocking calls.
• Reduces a complex ecosystem of dependencies to Jina and a few additional standard libraries.

Where to get CLIP Search?

CLIP Search installs with pip:

pip install clip-server[search]>=0.7.0

And you will need to install the CLIP client:

pip install clip-client>=0.7.0

CLIP Search is a component of Jina’s CLIP-as-Service, an open source project with code that you can view on github.

Using CLIP Search

As a component of the Jina AI framework, CLIP Search enables you to set up and use sophisticated AI with only a few lines of Python code. For code examples and full usage information, see the documentation for CLIP Search on the Jina website.

You can also experiment with the web front-end and a selection of pre-indexed data.

Learn more

Want to dig more into the Jina ecosystem? Here are some resources:

• Developer portal — tutorials, courses, videos on using Jina
• Fashion search notebook — build an image-to-image fashion search engine

Please join our rapidly growing user community on Slack.