Did you mean functionality with Laravel Scout
In this tutorial, we’ll show you how to create “Did you mean” functionality fast and easy. Data we’ll use will be a large city database consisting of more than 3 million cities. The idea is to show the correct city name in case your users misspell them. The list of cities comes from MaxMind, Inc and you can find the list here. To understand what we are building take a look at the demo page. Setting up Laravel isn’t covered, but you can find plenty of tutorials on how to do this. Our project will depend on Laravel Scout and TNTSearch so lets install those dependencies:
composer require teamtnt/laravel-scout-tntsearch-driver
Add this to your providers array in config\app.php
After that publish the Scout config file like:
php artisan vendor:publish --provider="Laravel\Scout\ScoutServiceProvider"
And in your `.env, set TNTSearch as the default driver:
SCOUT_DRIVER=tntsearch
In config/scout.php set the storage_path
Make sure this directory is writable. Lets start with a basic command that will download and import the list of cities to our database. The database migration and the model look as follows:
php artisan make:model City --migration
Add public $timestamps = false; to your City model since we don't need timestamps. Notice the column n_grams, this will help us later to achieve the did you mean functionality. Now, lets populate the table. Our dataset is a regular file where each line represents a city. A simple command will do the job:
php artisan make:command ImportCities
Don’t forget to register the command in app\Console\Kernel.php
The command will automatically download the file from Maxmind, unzip it and import the cities to our database. We’ll only import cities that have a population greater than 0. Once we have the cities in our database, let’s create the inverted index that scout will consume.
php artisan tntsearch:import App\City
Ok, now we can do a basic search for a city:
App\City::serach('Berlin')->get()
Although this works fine, it’s still not able to fetch a city if you make a typo. Let’s take care of this.
php artisan make:command CreateCityTrigrams
This creates the index of trigrams that we’ll query if we don’t find anything in our cities.index
Our CityController.php looks like:
How does it work
We threw a lot of code at you above. Although this will work perfectly it’s also important that you understand why it’s working and what the logic behind is. Lets, for example, take the city “Berlin”. Now, if we break it into trigrams we become the following:
__b _be ber erl rli lin in_ n__
Let’s assume you make a typo and instead of Berlin you type “Berln”. The trigrams now look like:
__b _be ber erl rln ln_ n__
As you can see, 5 trigrams match the trigrams from the correct form and only 2 don’t match.
Each trigram set of the correct form is stored in cityngrams.index. If we query the index with a mistyped word "berln" well get pretty accurate results since 5 matches were found. It will also return some other results that match the trigrams so we'll have to do one more step to get the best suggestions possible. The levenshtein distance between two words will simply tell us how many characters we need to change to get the same word. So the levenshtein distance between "Berlin" and "Berln" is 1 because we only need to add an "i" for the words to match. So we'll sort our result from our cityngrams.index by levenshtein distance and return the suggestions to the user. Pretty simple but powerful, isn't it?
The front end of the demo page is build with react. In the upcoming months will create an on-line course on how to get started with React and Laravel so make sure to subscribe to our newsletter on our site. Also, follow us on twitter @nticaric and @sasatokic
