Unstructured Data Service

Testing IVF_SQ8 Index in the Milvus Vector Database

How is the performance of Milvus, a database for AI

Test objective

To compare the search time and recall rate as nq and topk varies.

Test metrics

  • Query Elapsed Time: Time cost (in seconds) to run a query. The variable that affects query time is nq - number of queried vectors.
  • Recall: The fraction of the total amount of relevant instances that were actually retrieved . Variables that affects recall rate are: a) nq - number of queried vectors; b) topk - top k result of a query.

Hardware/software conditions

  • Operating System: CentOS Linux release 7.6.1810 (Core)
  • CPU: Intel(R) Xeon(R) CPU E5–2678 v3 @ 2.50 GHz
  • GPU0: GeForce GTX 1080
  • GPU1: GeForce GTX 1080
  • Memory: 503 GB
  • Docker version: 18.09
  • NVIDIA Driver version: 430.34
  • Milvus version: 0.5.3
  • SDK interface: Python 3.6.8
  • pymilvus version: 0.2.5

Parameter setup:

Dataset (SIFT1B)

  • Data base: 1,000,000,000 vectors, 128-dimension
  • Data type: hdf5
  • nlist: 16384
  • metric_type: L2
  • nprobe: 32
  • cpu_cache_capacity: 150
  • gpu_cache_capacity: 6
  • use_blas_threshold: 1100
  • Whether to restart Milvus after each query: No

Performance test

GPU mode (search_resources: gpu0, gpu1)

CPU mode (search_resources: cpu, gpu0)

Conclusion

When nq is small, the search time in CPU Mode is much less than that in GPU Mode. However, as nq becomes larger, GPU Mode is sinificantly faster.

Recall test

GPU mode (search_resources: gpu0, gpu1)

CPU mode (search_resources: cpu, gpu0)

Conclusion

In both GPU and CPU modes, as nq increases, the recall gradually stabilizes to over 93%.

--

--

Scalable similarity search on unstructured data (such as image, video, and natural language) powered by https://milvus.io

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Milvus

Open-source Vector Database Powering AI Applications. #SimilaritySearch #Embeddings #MachineLearning