Optimizing search-as-you-type from the front-end perspective

A great search experience is one that’s fast and responsive and that provides users with what they seek. When it comes to WW’s food search on the web, the backend developers did a great job providing a search service that returns relevant results quickly and the design team delivered a beautiful intuitive design for the search interface.

As front-end developers, we needed to marry these two concepts into one cohesive functional system. In our specific scenario, the design team wanted a search-as-you-type feature where the search results react to the user’s query, even if they have not finished typing. Our implementation may start out as follows:

For every keystroke:

Query search service and updated results as they come available.

Let’s say the user is searching our food database for “apple.”

This is working as expected in our development environment where we have fast internet connectivity, a high-processing power device, and proximity to the test service. However, in the real world, there are many factors that affect the responsiveness of external services and ultimately impact the user experience.

Problems

What if:

• Internet connectivity is spotty or has reduced bandwidth?
• Search service is experiencing bandwidth congestion?
• Search service returns too much data?
• User’s device doesn’t have the processing power to handle all the requests?
• User’s typing is faster than the service’s ability to respond?

Imagine the user is traveling in a train and the internet connectivity is spotty, causing latency spikes which in turn cause our search requests to take longer than usual and complete in a different order from how they were requested.

The system indiscriminately updated the results whenever it received them. It put the search interface in a position where the user’s final search term is “apple,” but the search results displayed are for the term “app”

First Iteration

Update the system to ensure these search requests complete in the order they are made, so then we can guarantee that requests and results are synchronized.

In theory, it works as we expect, but we introduced side effects where now the client needs to wait for every previous search request to arrive before they see the desired results.

Let’s say there are two requests: The first one takes a second to complete and the other takes five seconds to complete. Then the user will not see any updates until the slowest request completes, in this case, after 5 seconds.

This detracts from the user experience and the search no longer feels responsive.

Second Iteration

Maybe we can improve the system by immediately updating the search results when the final search term result arrives and ignoring any other request.

Now the system tolerates when requests come out of order by providing users with feedback as they type. But our system is actually doing more work than necessary.

For example, the device is still waiting on and downloading the results of search requests that are no longer relevant. The system is unnecessarily consuming resources like internet bandwidth, CPU load to parse/store the results, and battery power on portable devices.

Maybe we can leverage other patterns to alleviate the unnecessary consumption of resources.

Debouncing the keystrokes

Debouncing dictates that we must wait a duration of time to see if any new keystrokes arrive before the search request goes out.

However, if a new keystroke is detected during the wait time, then the time is reset and waits until no new keystrokes are detected, then proceeds to make the search request.

With this approach our search feels more responsive than waiting for all previous requests to complete. However, it has an artificial delay that can be perceived if one pays close attention.

Throttling the keystrokes

Throttling is like debouncing except that it allows something to be called once within a given time period. So if we had a throttling rule that said only 1 request is allowed every 300ms. We can issue many requests in a 300ms second period, but only one request will go out.

The problem is that it is no longer reacting to each keystroke and the search requests are highly dependent on the user’s keystroke cadence. It causes the search interface to always yield slightly different results.

Problems Encountered

• Requests may complete in random order
• Waiting on results of search terms the user no longer cares about
• Unnecessarily consuming resources
• Unnecessary delays between or after keystrokes

In our original design we made an assumption that once a request is made, it will be completed before another request comes in. We neglected to consider the case where a request is made but doesn’t have time to complete before the next one comes in. We tried to patch the system with each symptom that was encountered. This led us to a bunch of solutions, none of which were ideal.

Request Cancellation

Let’s go back to the original implementation, but with a simple twist:

For every keystroke:

Query search service and update results as they come available.

If a search request is pending when a new keystroke is detected:
- Cancel the previous request
- Make a new request for the updated term

Canceling any outstanding requests provides a few benefits over the previous iterations:

• It removes the need for the system to keep track of multiple requests and their order.
• It reduces bandwidth consumption from the client side because the client cancels superfluous requests.
• The search request is now responding as quickly as the network and service allows it to without any artificial delays.

While we have addressed all problems regarding the unreliable network and potential hiccups with the search service, there are still unresolved inefficiencies with all the implementations.

Caching Responses

As users start pressing backspace to a more broader term:

“apple -> appl -> app”

The system is re-searching for terms it had previously loaded.

We can leverage a Least Recently Used (LRU) caching strategy to remember previously loaded search results. This provides some benefits:

• Presents the results immediately to the user
• Alleviates the search service from unnecessary requests
• Allows for offline mode support on those previously loaded results

The challenging part of using a caching layer is invalidating it when things change. When the client changes, we have direct access to the cache and can flag it appropriately so it gets invalidated. However, knowing about server-side changes can be a bit more difficult.

Stale-While-Revalidate

When dealing with a dynamic system — where the search results can change at any time from anywhere and it’s critical for the results to be up to date — adapting the stale-while-revalidate strategy can mitigate the burden of managing the cache of server-side changes.

• If there are cached results for the search term, restore the result from the cache while simultaneously querying the search service for updates.
• If the search service returns updated results, refresh the search results list with the updated version and update the cache.

Final Implementation

We’ve updated the search system to cancel requests it no longer cares about and added a caching layer to help speed up previously searched results. With these simple techniques, the system is responding as quickly as the circumstances allow it to while being mindful of wasting resources.

The search-as-you-type system is now working as efficiently as possible and provides a fast and responsive user experience.

— Alex Oxrud, Web Engineering Manager, WW (formerly Weight Watchers)

Interested in joining the WW team? Check out the careers page to view technology job listings as well as open positions on other teams.