Open Source, Milvus, NVIDIA, Python, Milvus Lite, PyMilvus, GenAI, NVIDIA JETSON XAVIER NX, Jetson, Edge, EdgeAI, Edge VectorDB
If you read my previous article on Raspberry Pi 5 + AI Kit, then you can imagine I have been looking at the Edge for fun use cases for Milvus. I have a NVIDIA Jetson Xavier NX that I decided to install Milvus Lite on through pymilvus. This is what I used.
Latest Milvus Lite Build — 2.4.7 for AARCH64
Latest PyMilvus Build — 2.4.4
Walk Through Video
Installation is easy, taking only a few seconds and I was upgraded and ready to start running locally with no Docker footprint.
Welcome to Ubuntu 20.04.6 LTS (GNU/Linux 4.9.201-tegra aarch64) * Documentation: https://help.ubuntu.com
* Management: https://landscape.canonical.com
* Support: https://ubuntu.com/pro
This system has been minimized by removing packages and content that are
not required on a system that users do not log into.To restore this content, you can run the 'unminimize' command.Expanded Security Maintenance for Applications is not enabled.69 updates can be applied immediately.
49 of these updates are standard security updates.
To see these additional updates run: apt list --upgradable49 additional security updates can be applied with ESM Apps.
Learn more about enabling ESM Apps service at https://ubuntu.com/esmLast login: Mon Jun 17 08:27:54 2024 from 192.168.1.161
root@nvidia-desktop:/home/nvidia/nvme# source pythongo.sh
pip3 Requirement already satisfied: pymilvus in ./milvusvenv/lib/python3.11/site-packages (2.4.1)
Collecting pymilvus
Downloading pymilvus-2.4.4-py3-none-any.whl.metadata (5.4 kB)
Requirement already satisfied: setuptools>=67 in ./milvusvenv/lib/python3.11/site-packages (from pymilvus) (70.0.0)
Requirement already satisfied: grpcio<=1.63.0,>=1.49.1 in ./milvusvenv/lib/python3.11/site-packages (from pymilvus) (1.60.0)
Requirement already satisfied: protobuf>=3.20.0 in ./milvusvenv/lib/python3.11/site-packages (from pymilvus) (5.27.0)
Requirement already satisfied: environs<=9.5.0 in ./milvusvenv/lib/python3.11/site-packages (from pymilvus) (9.5.0)
Requirement already satisfied: ujson>=2.0.0 in ./milvusvenv/lib/python3.11/site-packages (from pymilvus) (5.10.0)
Requirement already satisfied: pandas>=1.2.4 in ./milvusvenv/lib/python3.11/site-packages (from pymilvus) (2.2.2)
Collecting milvus-lite<2.5.0,>=2.4.0 (from pymilvus)
Downloading milvus_lite-2.4.7-py3-none-manylinux2014_aarch64.whl.metadata (5.2 kB)
Requirement already satisfied: marshmallow>=3.0.0 in ./milvusvenv/lib/python3.11/site-packages (from environs<=9.5.0->pymilvus) (3.21.2)
Requirement already satisfied: python-dotenv in ./milvusvenv/lib/python3.11/site-packages (from environs<=9.5.0->pymilvus) (1.0.1)
Requirement already satisfied: numpy>=1.23.2 in ./milvusvenv/lib/python3.11/site-packages (from pandas>=1.2.4->pymilvus) (1.26.4)
Requirement already satisfied: python-dateutil>=2.8.2 in ./milvusvenv/lib/python3.11/site-packages (from pandas>=1.2.4->pymilvus) (2.9.0.post0)
Requirement already satisfied: pytz>=2020.1 in ./milvusvenv/lib/python3.11/site-packages (from pandas>=1.2.4->pymilvus) (2024.1)
Requirement already satisfied: tzdata>=2022.7 in ./milvusvenv/lib/python3.11/site-packages (from pandas>=1.2.4->pymilvus) (2024.1)
Requirement already satisfied: packaging>=17.0 in ./milvusvenv/lib/python3.11/site-packages (from marshmallow>=3.0.0->environs<=9.5.0->pymilvus) (24.0)
Requirement already satisfied: six>=1.5 in ./milvusvenv/lib/python3.11/site-packages (from python-dateutil>=2.8.2->pandas>=1.2.4->pymilvus) (1.16.0)
Downloading pymilvus-2.4.4-py3-none-any.whl (196 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 196.0/196.0 kB 4.5 MB/s eta 0:00:00
Downloading milvus_lite-2.4.7-py3-none-manylinux2014_aarch64.whl (40.9 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 40.9/40.9 MB 5.7 MB/s eta 0:00:00
Installing collected packages: milvus-lite, pymilvus
Attempting uninstall: pymilvus
Found existing installation: pymilvus 2.4.1
Uninstalling pymilvus-2.4.1:
Successfully uninstalled pymilvus-2.4.1
Successfully installed milvus-lite-2.4.7 pymilvus-2.4.4[notice] A new release of pip is available: 24.0 -> 24.1
[notice] To update, run: pip install --upgrade pip
(milvusvenv) root@nvidia-desktop:/home/nvidia/nvme# pip
pip pip3 pip3.11 pip3.6
(milvusvenv) root@nvidia-desktop:/home/nvidia/nvme# pip3 install --upgrade pip
Requirement already satisfied: pip in ./milvusvenv/lib/python3.11/site-packages (24.0)
Collecting pip
Downloading pip-24.1-py3-none-any.whl.metadata (3.6 kB)
Downloading pip-24.1-py3-none-any.whl (1.8 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.8/1.8 MB 10.6 MB/s eta 0:00:00
Installing collected packages: pip
Attempting uninstall: pip
Found existing installation: pip 24.0
Uninstalling pip-24.0:
Successfully uninstalled pip-24.0
Successfully installed pip-24.1
With the latest release of pymilvus, you can now install Milvus Lite on NVIDIA Jetson devices. AARCH64 is the build for NVIDIA.
pymilvus-2.4.4-py3-none-any.whl
milvus_lite-2.4.7-py3-none-manylinux2014_aarch64.whl
As you can see even previous generation NVIDIA Jetson devices pack a lot of RAM, CPU and NVIDIA GPU power.
So I decided to build a small local application on the device utilizing a webcam, the Xavier, Milvus Lite, Python, BLIP and a few libraries. This stored and processed everything locally and saved the images locally. I do send a copy of the images and the generated caption to Slack for remote viewing. We could also send to a remote Kafka server, S3, Milvus server and much more in a distributed fashion. I like being able for the edge to run locally and then connect to the network when needed or available. We may be in a remote desert, forest, tundra, highway, body of water or glacier.
These are the primary libraries used in my demo application.
PYTHON LIBRARIES
- pymilvus
- slack-sdk
- opencv-python
- timm
- scikit-learn
- numpy
- transformers
MODELS
- Salesforce/blip-image-captioning-large
- resnet34
INSERT CAMERA IMAGES and CAPTIONS
DATABASE_NAME = "./XavierEdgeAI.db"
COLLECTION_NAME = "XavierEdgeAI"
DIMENSION = 512
PATH = "/home/nvidia/nvme/images/"
slack_token = os.environ["SLACK_BOT_TOKEN"]
BLIP_MODEL = "Salesforce/blip-image-captioning-large"
milvus_client = MilvusClient(DATABASE_NAME)
fields = [
FieldSchema(name='id', dtype=DataType.INT64, is_primary=True, auto_id=True),
FieldSchema(name='caption', dtype=DataType.VARCHAR, max_length=512),
FieldSchema(name='filename', dtype=DataType.VARCHAR, max_length=512),
FieldSchema(name='currenttime', dtype=DataType.VARCHAR, max_length=512),
FieldSchema(name='vector', dtype=DataType.FLOAT_VECTOR, dim=DIMENSION)
]
schema = CollectionSchema(fields=fields)
milvus_client.create_collection(COLLECTION_NAME, DIMENSION, schema=schema, metric_type="COSINE", auto_id=True)
index_params = milvus_client.prepare_index_params()
index_params.add_index(field_name = "vector", metric_type="COSINE")
milvus_client.create_index(COLLECTION_NAME, index_params)
cam = cv2.VideoCapture(0)
result, image = cam.read()
strfilename = PATH + 'xavier{0}.jpg'.format(uuid.uuid4())
if result:
cv2.imwrite(strfilename, image)
else:
print("No image")
processor = BlipProcessor.from_pretrained(BLIP_MODEL)
model = BlipForConditionalGeneration.from_pretrained(BLIP_MODEL)
inputs = processor(image, return_tensors="pt")
out = model.generate(**inputs)
caption = (processor.decode(out[0], skip_special_tokens=True))
imageembedding = extractor(strfilename)
milvus_client.insert( COLLECTION_NAME, {"vector": imageembedding,
"currenttime":currenttimeofsave,
"filename": strfilename, "caption": str(caption)})The abbreviated code gives you an idea how easy it is. The full source code is linked below.
SOURCE
SEARCH
Image Search is easy!
cam = cv2.VideoCapture(0)
result, image = cam.read()
query_image = PATH + 'xaviernow.jpg'
results = milvus_client.search(
COLLECTION_NAME,
data=[extractor(query_image)],
output_fields=["filename", "caption", "currenttime", "id"],
search_params={"metric_type": "COSINE"},
)NOTEBOOK QUERY — FIND MY DUCKIE
try:
results = milvus_client.query(
collection_name=COLLECTION_NAME,
filter="caption like '%duck%'",
output_fields=["filename", "caption", "currenttime", "id"],
limit = 5,
)
# print(results)
for index in range(len(results)):
for key in results[index]:
img = Image.open(results[index]['filename'])
display(img)
print("Caption: " + results[index]['caption'] + " @ " + results[index]['currenttime'] + "ID: " + str(results[index]['id']))
except Exception as e:
print("An error:", e)
print("Search done")
RUNNING
FUN UNSTRUCTURED FRIDAY PREVIEW
It all works and we can access our local stored Vector Data with easy. We can query by metadata or search by images. This is awesome.
Don’t miss the meetup in New York City on July 25, 2024!
ENHANCEMENT IDEAS
- Search from Slack
- Search from Discord
- Post to Discord
- Jupyter Notebook
- Add filter fields
- Store to local and remote Milvus
- Add more image processing and GPU
- Add OLLAMA
- Run a newer NVIDIA Jetson ORIN (Anyone have one?)
- Integrate S3 / MinIO
RESOURCES
Drop us a star please!
Get Milvused!
Read my Newsletter every week!
For more cool Unstructured Data, AI and Vector Database videos check out the Milvus vector database videos here:
https://www.linkedin.com/company/zilliz/
