Search Autocomplete Personalisation

Published in

Twinkl Educational Publishers

4 min readJun 16, 2022

Check out this blog to find out how Twinkl uses data to improve search autocomplete suggestions

Originally published by Anna Sandersfield, Data Scientist at Twinkl, at https://www.twinkl.co.uk/blog/search-autocomplete-personalisation

Helping those who teach is one of Twinkl’s main values and drives everything we do; from the resources we create to the continuous improvements made to our website. One of our aims as a Data Department at Twinkl is to assist customers to more easily find the content they need to teach.

When a customer comes to our website with an idea of the resources they require their first point of call is often to type in the search bar. One aspect of the search bar, and the basis of this blog, is the autocomplete which adjusts the suggested search terms as the user types.

Where we’ve come from

The orginal search autocomplete logic was simple: Suggest the most searched terms to users, filtering by the letters they’re typing in. Whilst this proved successful on the whole there were clear downsides to this one size fits all approach. User groups who didn’t make up the majority of our users, such as those not searching in English, or those from smaller segments (e.g. secondary school teachers), were negatively affected by this approach. Traditionally the bulk of our users have been Key Stage 1 or 2 teachers from England which meant that the popular search autocomplete suggestions were more heavily weighted towards them.

Where we are

In the past year we have been working hard to tailor our suggestions and ensure they are more useful to all of our users. To achieve this we have clustered our users to provide them with suggestions more specific to the country they’re in and the types of children they teach. We’ve also made a conscious effort to include more search terms in languages other than english to accommodate our overseas and non-english speaking users.

Some examples of our tailored suggestions are shown below:

Country: England
Career: School-Based EYFS

Country: Australia
Career: Secondary Teacher (Years 7–10)

Country: Romania
Career: Primary School Teacher

We use a combination of machine learning and elasticsearch to deliver search autocompletions as the user is typing that are more tailored to them.

In addition to this, we have enabled fuzzy matching in elasticsearch to help mitigate against spelling mistakes whilst the user is typing.

Example:

Going one step further with Personalisation

To further personalise our suggestions, we are incorporating contextual information from the users past searches and page views . By boosting terms linked to the user’s recent searches or page views we can suggest terms they’re more likely to search next.

Previous Search Context

We can glean information of what our users have searched for to see what other users who searched for the same term went on to search. Using this information we can therefore come up with a list of terms a user is likely to search for next.

Below is an example of the autocompletions that appear if a user is searching for ‘d’ having just searched for ‘dinosaurs’.

This was recently tested on our users and was found to be more successful than the original method so it has now been rolled out across the site.

Resource and Category Page Context

Using similar logic to the previous searches context we can identify the resource or category page a user is currently on and use data from other user sessions who viewed those pages to predict what terms a user may go onto search given the page they’re on.

Using the following resource (First 100 High Frequency Words Handwriting Worksheets) as an example we can provide suggestions tailored to it:

See below for the autocompletions that would be shown if the user starts typing ‘hi’ whilst on the above resource. The suggestions become far more specific to the resource.