Data Storage in the Cloud
Jina AI + Qdrant → Data Storage on the fly!
Now you can leverage Qdrant as the document store for Jina’s DocArray for faster processing and retrieval of Documents in the cloud!
Introduction
DocArray is a unique first-of-its-kind data structure for unstructured data. It is a part of the bigger Jina AI ecosystem. It can accommodate all kinds of data including text, images, audio, video, etc, and is designed to be intuitive to use with Python, so you can get started right away without any pre-requisites.
Qdrant is an open-source vector database that provides a production-ready service with a convenient API to store, search, and manage end-points. It is tailored to extended filtering support making it useful for all sorts of neural network or semantic-based matching tasks.
Set Up a Qdrant Instance
To set up a cloud storage instance using Qdrant, you need to run a Qdrant server using a docker image. To do that you can create a docker-compose.yml as follows:
You can then run docker compose up to start the instance.
Minimum Working Example
In this example, we will create a Qdrant local instance to store the Document and build a simple text search.
First, start the Qdrant service and create a DocumentArray array instance.
Now let’s index the Documents:
Now we will generate the embeddings using the BERT model:
Finally, we can query the indexed Documents and get the results:
Output: Persist Documents with Qdrant.
DocArray + Qdrant in action!
To give you a glimpse of the potential capabilities of Qdrant’s integration with Jina AI, we have created a colab notebook where you can simply create a basic fashion search engine (image-to-image search) by just using DocArray and Qdrant.
Check out the GitHub repository for the source code and the notebook.
Learning Resources
This is part-2 of three-part series about exploring different cloud document stores that can be used with Jina’s DocArray.
Venture into the exciting world of Neural Search with Jina’s Learning Bootcamp. Get certified and be a part of Jina’s Hall of Fame! 🏆
Stay tuned for more exciting updates on the upcoming products and features from Jina AI! 👋
