Elastic Search 8.9: Hybrid search with RRF, faster vector search, and public-facing search endpoints

Elasticsearch 8.9 introduces a range of powerful features to enhance search capabilities and performance. This release focuses on hybrid search with Reciprocal Rank Fusion (RRF), accelerated vector hardware instructions for faster searches, enhanced data ingestion options, and the introduction of public-facing search endpoints through search applications. Here’s what’s new in Elasticsearch 8.9:

Hybrid Search with Reciprocal Rank Fusion (RRF)
Users can now benefit from hybrid search capabilities using Reciprocal Rank Fusion (RRF). This feature enables the combination of various search techniques, such as vector, keyword-based BM25f, and text expansion with the Elastic Learned Sparse Encoder model. RRF seamlessly integrates these methods without the need for complex normalization or weight configuration. With improved relevance, users can achieve more accurate and diverse search results. RRF is available as a technical preview for Platinum+ users.

Accelerated Vector Hardware Instructions
Elasticsearch 8.9 leverages native instructions (SIMD) through the incubating Java API, Panama vector API. This enhancement delivers a remarkable 30% performance improvement for both vector searches and vector indexing. The use of accelerated vector hardware instructions benefits High-Dimensional Navigable Small World (HNSW) and brute force searches. Additionally, encoding optimizations further enhance query latency for brute force vector searches, improving performance by up to 50% in slower searches.

Enhanced Data Ingestion
Data ingestion capabilities are extended with the introduction of new connectors and features. A SharePoint Online connector is now generally available for Platinum+ users, offering improved performance, document-level security, incremental synchronization for data freshness, and local binary content extraction for large files. Furthermore, beta versions of ServiceNow and Dropbox connector clients expand integration options for proprietary data sources.

Public-Facing Search Endpoints with Search Applications
Elasticsearch 8.9 introduces search applications into beta for Platinum+ users. This feature allows the creation of public-facing search endpoints for Elasticsearch indices. The _applications/search-application/<app-name>/_search endpoint enables queries defined in associated search templates, providing search functionality without the need for a passthrough query system. By pairing search applications with a public search endpoint-only API key using the “search_application_query” workflow role restriction, users can easily enable read-only access to the search application endpoint. Additionally, a new Search Application JavaScript client streamlines development.

These enhancements empower Elasticsearch users to:

• Combine various search techniques with Reciprocal Rank Fusion (RRF) for improved relevance and diverse search results.
• Benefit from accelerated vector hardware instructions, achieving up to 30% performance improvement in searches and indexing.
• Ingest data from various sources, including SharePoint Online, ServiceNow, Dropbox, and more, with enhanced performance and security features.
• Create public-facing search endpoints using search applications, enabling public access to Elasticsearch indices while maintaining query control.

Elasticsearch 8.9 is available on Elastic Cloud, offering a comprehensive hosted Elasticsearch solution with all the latest features.

Read more Elastic Search 8.9: Hybrid search with RRF, faster vector search, and public-facing search endpoints

We’re a proud Elastic partner. Read all about our Elasticsearch practice