How to Test the Quality of an eCommerce Search Engine
Without even realizing it, internet search engines have become a necessary and indispensable tool when arriving on a web page. Whether we are looking for information, shopping, comparing travel prices, listening to music or checking the news, search engines are often the starting point for this activity.
Moreover, search engines have evolved as quickly as the network itself in recent years. Say I’m looking for an air conditioner, the process would be more or less the following:
I open my laptop, I type the address of an eCommerce website and then I start to write in its search engine “air …” and I don’t have to do anything else, the search engine does the rest for me. It completes the search term “air conditioner”, shows suggestions (portable, inverter, with or without tube …), includes results in different categories, offers the most searched items today, provides my recent searches, gives me the option to sort results by price, by name, by relevance … it allows me to filter by brands, by characteristics. Even if I spell it wrong, it shows me the correct results.
In summary, thanks to the search engine I find the air conditioner that I want in probably less than ten minutes. This process would have taken me much longer if I had wanted to do it about seven years ago, that’s before company’s like EmpathyBroker existed, using their expertise in search and navigation to develop great online customer experiences, like the one I just described.
Those search and navigation functionalities I’m referring to are characteristics that we can see today in a large number of the eCommerce sites we visit on a regular basis. We’ve come to appreciate them as part of the basic operation of these type of sites and, although we don’t always notice their presence, if we don’t find what we’re looking for or don’t enjoy the experience we soon notice that we miss them.
Many of the top eCommerce sites have been developed by the EmpathyBroker team, of which I am proud to belong. Combining ease of use with a unique and innovative design, together we help turn the search and navigation process into a pleasant, meaningful and often irresistible experience. We like to say we create magic moments. To learn more about how we do this, you can read this article by our founder, Angel Maldonado.
As a quality tester, my responsibility is to make sure that our search engine always works correctly, testing with regular and certain frequency the functionalities I’ve been referring to. Now I’ll briefly describe the main functionalities that a search engine has and what aspects we take into account from a quality department point of view to verify that everything is functioning as it should be.
The main features of modern search engines
- Instant search: With this feature the search engine returns results as we type in a word, without the need to press ENTER. Testing this functionality is quite simple although you have to avoid falling into the ease of always using the same terms.
- Synonymize: This function provides a kind of data driven synonym dictionary used on no results situations, so that the barrier between the word that the user searches for and what is indexed in the catalogue is broken. We provide a basic dictionary of synonyms and then the client adds those that he considers most useful. To test this, we have to analyze user behavior to spot possible synonym problems with specific queries.
- Search by reference: Here we offer the possibility to search for the code or SKU number of an article in order to identify it in the online store. Reviewing this tool can easily lead to errors since we can find that a certain item has gone out of stock and may no longer appear in the catalogue. This, however, is not necessarily an error, it may be that the article is only temporarily unavailable and we need to take this into account.
- Stemming: This functionality consists of taking the root of a word and offering similar results from it, so that, for example, results are shown for singular / plural, feminine / masculine or alternatives of the same word stem … Again, we must avoid testing it always with the same words, trying instead to use the most searched terms every week.
- Sorting: This is the ability to sort results by different criteria such as alphabetical, price, colour, relevance and so on. When testing this functionality, we must not forget that prices may change during sales seasons, so we must pay close attention to these details.
- Suggestions: Here suggestions are offered to the user based on the most searched terms for that day, or taking into account the user’s search history. Within this functionality we need to include several different ways to show suggestions and offer each client the one that best suits their type of business.
- Partial results: Results are displayed for more than one term even if there is no apparent relationship between them. This is easily reviewable on the website of the end user and also in a testing environment, once again we must avoid always testing with the same words.
- Top clicked / No results: In instances where we look for something that does NOT have a relevant results page, a list of the top search results of the website can be displayed instead, offering real alternatives rather than a blank page. We can’t let a user leave a website because he feels frustrated that he couldn’t find anything or wasn’t able to interact with the products!!
- Spellcheck: This tool corrects misspellings through an algorithm that offers results for the correct term. When we test this feature we have to take into account aspects such as the position of the letters on the keyboard and the speed of the end user when writing.
These are some of the key aspects we need to consider when checking the quality of a search engine, and we must ensure that each and every one is checked and works correctly with certain periodicity. We must also give the same importance to the design of the site, to the way we interact with the search engine, the presentation of the results… in short, the user experience, both in the search and in the navigation, are equally as critical.
In addition, we must bear in mind that we want to customize the results for each user. To do it we need to know who the user is and their context (their interests, location, etc.). This is very important to ensure that the results are relevant to each user.
As Q.A. testers we must put ourselves in the place of the end user, distanced from the programming tools that we know are there behind the surface and focus instead on the results offered to the client. That’s the only way we can check how satisfactory the search experience is. And for us at EmpathyBroker this is key. We want users to feel unique, special and understood and to help eCommerce sites deliver a more joyful and meaningful digital experience. Creating an even more personal and special service than you would receive face to face in a traditional shop.
