GCP Retail Search — Operational Best Practices and Explainability

Google Cloud’s Retail Search (now called Vertex AI Search for Retail) is a powerful, fully managed service that simplifies the implementation of high-quality search and recommendation experiences for e-commerce businesses. The advantages of Retail Search hinge on the quality of your underlying catalog and user event data, as well as your serving controls. A strong correlation exists between search performance (measured by KPIs like CTR, CVR, and AOV) and the completeness and accuracy of this data.

Even seemingly clean product catalogs can contain subtle issues that hinder product discoverability and impact search performance. Below are common problem areas and debugging tips to improve your Retail Search results.

Please note:

These debugging strategies offer a good starting point to enhance your Retail Search results. If after addressing these areas you still encounter issues, consider contacting Google Cloud support.

The debugging strategies are classified into two main categories:

1. Relevance and Recall
2. Ranking and User events

Relevance and Recall:

1. Proactively Monitor Data Quality Dashboard

The Data Quality Dashboard, located within the Retail console, reveals your search and browse model’s current performance tier. Strive for Tier 4 (personalization enabled) or Tier 3 (ranking and revenue optimized) when deploying Retail Search in production. This dashboard outlines data requirements, potential obstacles to tier advancement, and how these factors influence search performance.

While proactive alerting features are in development (or may already be implemented), it remains essential to regularly monitor data quality for optimal search experiences.

2. Troubleshooting Irrelevant Products in Search Responses

Retail Search builds its product knowledge base from catalog data i.e. Product info (title, description, brand, URI, and associated product fields). Search relevance depends on the model’s comprehensive and accurate understanding of these products.

Debugging Steps

a. Verify Product Data:

• Employ the getProduct API to fetch complete product information from Retail Search.
• Thoroughly examine the data. Address any missing or incorrect details, which may point to issues within your data ingestion processes.

b. Assess Content Relevance:

• Review the retrieved product information. Evaluate whether all details are directly pertinent to the product itself.
• Check for incorrect or misleading descriptions, text, or assigned categories.

c. Review Custom Attributes:

• If you’ve incorporated custom attributes, confirm that appropriate relevance flags (searchable, indexable etc) are enabled.

Escalation: If the product data offers no explanations, proceed to open a support case.

3. Optimising Query Expansion Usage

Query expansion relaxes search criteria when the original query yields fewer than five results. However, it’s essential to correctly configure a canonical filter within the search query for reliable expansion behavior.

Understanding Query Expansion Behavior

• Query Expansion Trigger: If the combined results of a search query and canonical filter yield fewer than 5 items, query expansion is triggered. This introduces slightly less relevant results to diversify the output.
• Facet Filters and Expansion: Applying facet filters (e.g., color=black) to a canonical filter may prematurely trigger query expansion. This occurs if the narrowed result set is too small (e.g., only 2 black products), negatively impacting the search user experience (UX).
• Canonical Filters and Expansion: If canonical filters are omitted, query expansion might not be triggered at all. For example, using only filters like “in_stock” + “store_id” could result in only 3 products. However, without those filter conditions set in the canonical filter, the backend might “see” 102 total products (including out_of_stock items) and incorrectly assume sufficient results exist, thus not triggering the expansion.

Setting a Canonical Filter

a. Define the Base Filter: This filter constitutes the default filter criteria applied to every search query. Example: availability: ANY(“IN_STOCK”) AND inventory(place_id).

b. Append User Filters: User-selected facets dynamically refine the base filter. For instance, selecting a brand modifies the filter to: availability: ANY(“IN_STOCK”) AND inventory(place_id) AND brands: ANY(“my_brand”).

Ensuring Effective Query Expansion

Query expansion functions as expected only when the canonical filter remains identical to the base filter. Else the expansion trigger may not happen or it can even lead to bad search results.

4. Understanding the ‘Exact Searchable’ Attribute Flag

The ‘exact searchable’ attribute flag is designed for attributes like model numbers or part numbers — alphanumeric strings often lacking linguistic meaning.

Best Practices

• Decompose Strings: For model/part numbers, break the string into n-grams and retain the original string within a text attribute marked ‘exact searchable: true’. This enables matching against any n-gram fragment.
• Avoid Common Attributes: Do NOT apply ‘exact searchable’ to common text attributes like brands or attributes that contain typical English nouns. Doing so can disrupt search relevance when queries match these widely used terms.

The ‘exact searchable’ flag bypasses linguistic analysis, leading to rigid matching. This is beneficial for precise part number or model number lookups but detrimental to the natural language understanding that Retail Search excels at.

5. Retail Search Responses with Relatively Fewer Products (Head Queries)

It’s essential to understand that Retail Search is not intended as a direct replacement for a database. Its strengths lie in ranking and optimizing for relevance, not exhaustive product retrieval.

In practice, top-revenue products typically cluster within the first 200–1000 results. Beyond this range (around pages 6–7 and onward), revenue contribution from individual products tends to decline sharply.

Focus on ensuring that Retail Search effectively ranks and displays the most relevant products within the initial search result pages. This ranking-centric approach, compared to traditional search engines that might prioritize sheer volume, is key to Retail Search’s focus on driving engagement and revenue.

6. Isolate Serving Control Impacts with ‘Vanilla’ Search

Serving controls enable manual adjustments to search responses for specific business objectives. They should not be used to compensate for perceived relevance or ranking problems.

That said, existing serving controls (synonyms, boost/bury, etc.) can occasionally produce unexpected results due to suboptimal configuration. To isolate such issues, execute the search using the “default_serving_config”. This ‘vanilla’ configuration lacks any applied serving controls.

If the unexpected product disappears using the default configuration, the issue likely lies within your serving controls. Create a fresh serving config and add controls incrementally to pinpoint the problematic element.

Debugging Filters

A similar approach helps troubleshoot complex filter conditions (AND/OR combinations). Perform searches with systematically modified filters (adding, removing, adjusting conditions) to isolate any problematic filter that compromises results.

7. Maximize Product Detail During Ingestion (Avoid Assumptions)

Retail Search derives its product knowledge directly from ingested catalog data. Greater detail enhances the model’s understanding. It’s important to note that the model operates strictly on provided data; it will not infer additional attributes.

To optimize searchability, ensure:

• Comprehensive Information: Include as much relevant product data as possible in the catalog.
• Searchable Attributes: If certain text attributes are important for discovery, explicitly mark them as ‘searchable: true’. Otherwise, the model relies primarily on title and description content.

8. Troubleshooting URL Issues

Retail Search extracts supplemental information from the product’s ‘uri’ field. For optimal use, ensure the following:

• Accessibility: The linked webpage must load correctly (including any JavaScript elements) and be publicly accessible (not behind a login).
• Uniqueness: Avoid reusing URIs. Each URI should consistently point to the correct product’s webpage.
• Consistency: Webpage content should accurately reflect the product information within the Retail Search catalog to avoid misclassification.

Ranking and User Events:

1. Validate Event Ratios Against Traffic Patterns

A typical e-commerce user journey progresses from search > product clicks > add-to-cart > purchase. This should generally translate into a higher volume of detail page view events (clicks), followed by search events, then add-to-cart, and finally, purchase events (which may consolidate multiple products).

Ensure the ratio of event types in your data aligns with your site’s traffic patterns. Abnormally high counts for search or detail page views could indicate:

• Bot Traffic: Non-human activity can skew event data.
• Ingestion Issues: Check for event loss during the ingestion process.

2. Analyse ‘visitor id’ and ‘user id’ Distribution and Attribution

The ‘visitor id’ and ‘user id’ fields within events are crucial for Retail Search model training, particularly for personalization features.

• visitorId: A unique identifier for a user on a single device. Avoid including personally identifiable information (PII). Use Adobe Experience ID (concatenated visid_high and visid_low) for Adobe Analytics, or Client ID for Google Analytics. Never hardcode visitorId.
• userId: Represents a signed-in user’s customer ID. PII must be hashed before sending it to Google. Omit this field entirely for anonymous users. userId remains consistent across devices and should never be hardcoded.

Monitoring Recommendations:

• Events per Visitor: Track the number of user events associated with each ‘visitor id’. Limitations exist on the volume of events per visitor per minute (provide a link to relevant documentation) to safeguard against spam or erroneous data.
• Data Export or Pre-Ingestion Analysis: Leverage BigQuery exports or pre-ingestion staging to audit ‘visitor id’ data trends.
• Diagnostic Indicators: Investigate unusually high numbers of user events linked to a few ‘visitor ids’ or a significant mismatch between unique ‘visitor id’ counts and your average daily user metrics.

Important Note: This analysis is best performed on the customer side, as Google and Retail Search have limited visibility into business-specific metrics derived from user events.

3. Monitor Event Errors in the Analytics Dashboard

Regularly review user event errors (and catalog errors) within the Retail Search monitoring dashboard. Focus on the following key error types:

• UNJOINED_EVENTS: Indicates that event impression data references products absent from the catalog. These often occur due to catalog updates (additions/deletions). Since event ingestion and catalog refreshes may not perfectly align, expect some initial errors.
  Remediation: A periodic rejoin process attempts to resolve UNJOINED_EVENTS as the catalog updates. The rejoinUserEvents API allows for manual triggering as needed.
• Data format errors: Missing or incorrectly formatted data will appear as event errors. These require further investigation to pinpoint the root cause.

4. Understanding Ranking for High-Engagement Products

When debugging search result ranking, especially for head queries, remember that relevance is only one factor. Retail Search optimizes for revenue, meaning ranking is heavily influenced by predicted product performance.

Key Metrics:

• Predicted Revenue: Calculated as (Purchasability Prediction) x (Product Price).
• Purchasability Prediction: Derived from historical purchase and click data within your user events. Products demonstrating strong user engagement (clicks, purchases) are more likely to rank higher.

Debugging Approach:

When seeking to understand the ranking of specific products, analyze their associated event data:

• Purchase History: Focus on purchase events to discern buying patterns.
• Click Trends: Review click events to identify products garnering significant user attention.

5. Preventing Attribution Token Mismatches

Retail Search model training relies on attribution tokens to accurately link user behavior to specific search API calls. Ensure that your Search events incorporate the attribution token obtained from the corresponding search response (see documentation link). Inaccurate attribution tokens disrupt model training.

Causes of Mismatches:

• Frontend Caching: Avoid caching tokens between search sessions.
• Coding Errors: Implementations must correctly associate tokens with events.

Troubleshooting:

Unfortunately, no straightforward tool exists to detect invalid attribution tokens. Proactive measures are essential. Inspect Search events, scrutinizing tokens for unexpected patterns or values originating from other calls.

6. Optimizing Dynamic Facet Behavior with Filter Signals

Dynamic faceting allows Retail Search to intelligently select and rank facet attributes based on user behavior. The model infers facet popularity from filters present in search requests. Accurate filter usage is crucial for optimal dynamic facet performance.

Key Points:

• Filter-Driven Predictions: Frequently used filters in search requests signal to the model that the corresponding facets are important to users.
• Intelligent Facet Ranking: The model prioritizes relevant facets in the search response, informed by historical query patterns and associated filters.
• Filter-Driven Learning: When a new attribute is marked ‘dynamic facet-able: true’, include corresponding filters in search events. This initiates the model’s learning process. Ideally, your UI should mirror the dynamically generated facets, capturing and relaying user filter selections in both search requests and events. This feedback loop drives continuous model refinement.