Blazingly fast search with laravel scout & TNTSearch

Laravel provides a highly effective first-party package that allows us to perform full-text searches on our models. This is a powerful, driver-based solution that enables us to choose from multiple drivers designed for this purpose. Internally, it supports Algolia, Meilisearch, and MySQL/PostgreSQL drivers.

In this article, I will demonstrate how to integrate full-text search into your project using TNTSearch, an excellent Laravel Scout driver.

TNTSearch is a third-party free and straightforward Laravel Scout driver that utilizes SQLite as a high-speed storage solution for indexing your model data. It is particularly well-suited for small to medium-sized projects seeking fast search capabilities.

I’ve successfully implemented TNTSearch in one of my open-source projects, TV-Alert. I will walk you through the steps I took to do this, and you can explore the source code of that project on GitHub.

Main steps:

• Installing laravel scout and TNTSearch laravel driver
• Configuring our model to index its data
• Full text search on model

Installing laravel scout and TNTSearch laravel driver

First of all we need to install these two packages:

composer require laravel/scout
composer require teamtnt/laravel-scout-tntsearch-driver

After that we should make sure that TNTSearch service provider is loading. Add this line into your config/app.php in provider section:

// config/app.php
'providers' => [
    // ...
    TeamTNT\Scout\TNTSearchScoutServiceProvider::class,
],

Next we’ll publish laravel scout configuration file

php artisan vendor:publish --provider="Laravel\Scout\ScoutServiceProvider"

It will add config/scout.php file into your project files. Open this file then add this section to prepare TNTSearch for laravel scout:

'tntsearch' => [
    'storage'  => storage_path(), //place where the index files will be stored
    'fuzziness' => env('TNTSEARCH_FUZZINESS', false),
    'fuzzy' => [
        'prefix_length' => 2,
        'max_expansions' => 50,
        'distance' => 2,
	'no_limit' => true
    ],
    'asYouType' => false,
    'searchBoolean' => env('TNTSEARCH_BOOLEAN', false),
    'maxDocs' => env('TNTSEARCH_MAX_DOCS', 500),
],

The important part of this config is storage section which tells the TNTSearch that where it should store index database of your models. by default it will store indexes in storage folder of your app. index file names would be something like tv_shows.index which tv_show is name of your model.

To prevent your search indexes being committed to your project repository, add the following line to your .gitignore file.

/storage/*.index

Now we should tell the laravel scout to use TNTSearch as its driver. Open your .env file and put this line on it:

SCOUT_DRIVER=tntsearch

Now we are ready to tell the laravel scout to index our model data.

Configuring our model to index its data

Just by adding Laravel\Scout\Searchable trait to our model we can tell the laravel scout that we want to index that model data:

<?php
 
namespace App\Models;
 
use Illuminate\Database\Eloquent\Model;
use Laravel\Scout\Searchable;
 
class TVShow extends Model
{
    use Searchable;
}

At this moment whenever a record is added or updated on our TVShow model it will be indexed in tv_shows.index file.

An important consideration here is that we usually don’t want to index all fields of our model for full-text search, we can tell the laravel scout to only index necessary fields of our model. To do that we can override toSearchableArray function on our model like this:

public function toSearchableArray()
{
    $array = [
        'id' => $this->id,
        'name' => $this->name,
        'network' => $this->network,
        'genre' => implode(" ", $this->genres ?? []),
    ];

    return $array;
}

Here we are telling laravel scout to index only id, name, network and genre fields. This trick will dramatically reduce size of our index file and will help speed up our search.

It also worth mentioning that we should always index id field so laravel scout could track our records correctly.

You can read more about all configurations and customization available for laravel scout on its documentations.

Full text search on model

Now we can easily do full-text search on our model with search method:

TVShow::search('The big bang');

It will return records that contain your entered term. TNTSearch will sort results based on a scoring method, placing the most relevant records at the top. Additionally we can apply some well-known Eloquent methods to our search, such as where, limit, paginate, and orderBy.

This is live implementation of TV-Alert project on Series-Alert.ir

To highlight our results, TNTSearch provides a useful highlight function:

public function highlightResults(string $term): void {
    /** @var TVShow $tvShow */
    $tnt = new TNTSearch();
    foreach ($this->searchResults as $tvShow) {
        $tvShow->name = $tnt->highlight($tvShow->name, $term, 'hl' ,['wholeWord' => false]);
        $tvShow->network = $tnt->highlight($tvShow->network, $term, 'hl' ,['wholeWord' => false]);
    }
}

Here we used the $tnt->highlightfunction which surrounds related words with a <hl></hl> tag, then we can style those words with a simple CSS code in our HTML output like this:

hl {
    background-color: yellow;
}

Here your can see the highlightResults function in the TV-Alert repository. we handled all search-related tasks in SearchTVShow class. Feel free to explore that class to see how we implemented some tricks to obtain the total number of hits.

Conclusion

By utilizing Laravel Scout with TNTSearch as its driver, we can enhance the search experience for our web applications. TNTSearch indexes our specified models into an SQLite database in our app storage, utilizing this database to deliver fast search results.

You can refine your search results by using the “where” statement and leverage other Eloquent capabilities, such as “order by”

Read more on Laravel scout and TNTSearch documentation.

Live version of our TV-Alert project is accessible at Series-Alert.ir. You can experience its fast search capabilities using the search box. Furthermore, you have the option to subscribe to your favorite TV shows, and the system will send you email notifications whenever a new episode is released.