Laravel Scout (full-text search) P1: Installation, configuration & searching (TntSearch)

A deep dive into the Laravel Scout package, from beginner to advanced. Part 1 will show you how to install, configure and search on models.

This is a three part article series, makes sure to read in order to fully understand this topic :

Part 1 (this article) — Laravel Scout P1: full-text search — P1 (TntSearch)

Part 2 — Laravel Scout P2: limitations, drivers & Builder

Part 3 — Laravel Scout P3: combining search, filter and ordering

Summary

• What is Laravel Scout
• Installation
• Indexing a model
• Deciding on a model
• Installing TntSearch
• Performing a search
• Commands

What is Laravel Scout

Laravel Scout is a first party package (it means that it’s created and maintained by the creators of Laravel) to help you implement a full-text search in your application.

It allows you to implement a powerful search feature like a search bar as you can find in the Laravel documentation.

It’s an API to a driver of your choice like Algolia, Meilisearch, and others. It allows you to easily and automatically index your data. Then you can, with intuitive code, search directly from your model class.

The official documentation — https://laravel.com/docs/master/scout

The package Github repository — https://github.com/laravel/scout

Installation

Let's follow the install documentation guide, you can read the official documentation as we're progressing:

Laravel - The PHP Framework For Web Artisans

First, let's add the package to our project with Composer, simply require it, it will automatically install the latest compatible with your Laravel version:

composer require laravel/scout

This package comes with a new config file, we want to be able to have access to that file and change parameters later. So let's ask the provider of the package to publish its files (in this case only a config file):

php artisan vendor:publish --provider="Laravel\Scout\ScoutServiceProvider"

This will add the file config/scout.php to your project. Take some time to read what this config file offers, for now we won't change anything. At this point, the package is installed and ready to be used, so I would recommend doing a commit which I usually call "composer require laravel/scout" and include the composer.json, composer.lock and config/scout.php files to make it clear when it was installed in your git history.

Indexing a model

At this point, you need to make sure you understand how a full-text search will be performed in your Laravel app (or really, in any applications):

Your database, probably MySQL, is a very powerful tool for storing, updating and filtering data. MySQL does come with a few features for searching text, but it remains basic and not very powerful (talking relatively, lol).

We need to delegate the task of searching to another service, this is where you've probably heard of : ElasticSearch, Algolia or Meiliseach. They will run very powerful algorithm and result pertinent results, but for this to work, it needs access to the data/columns you want to search on. Everytime you create, update or delete a model, your search service needs to know about it that change, this is called "indexing" the data.

Let's start by enabling the searching by going into a model of your choosing and add the trait Laravel\Scout\Searchable, and that's it really, to beauty of using a package !

<?php

namespace App\Models;

use Illuminate\Database\Eloquent\Model;
use Laravel\Scout\Searchable;

class Post extends Model
{
    use Searchable;
}

By default the package will index every column of the model, which is not ideal because you often don't want to search on everything, you will have to many false positive. Imagine this article showing up in your search results contains the word "override", you probably want to index the title, tags, and a small description of 150 characters max. So let's override the function toSearchableArray() and return the data ourselves to be indexed :

<?php

namespace App\Models;

use Illuminate\Database\Eloquent\Model;
use Laravel\Scout\Searchable;

class Post extends Model
{
    use Searchable;

    public function toSearchableArray(): array
    {
        return [
            'id' => $this->id, // <- Always include the primary key
            'title' => $this->title,
            'tag' => $this->tag,
        ];
    }
}

Deciding on a driver

It is time to decide on a driver you will be using to index and search your data. Each driver have pros and cons, I will go more into depth in the part 2 of this article series. I will go over the list here once so you know they exist :

• null — This does nothing, no indexing and no search. Don't use this.
• collection — This is used for local environments and testing, the search is performed by retrieving ALL of your models of then performing a huge foreach loop to find your results.
• database — This will delegate the search to your database. It's performant enough for small to medium project, results are not very relevant.
• algolia — Algolia is a very powerful search platform but is a paid service. There is nothing to install, they host everything for you.
• meilisearch — Meilisearch is also a very powerful search engine, it is open source but you need to install this and manage it yourself.
• tntsearch— Tntsearch is a project from the teamtnt team, it is a PHP lightweight solution and good enough search engine. Everything is indexes locally to your application, so there is nothing to install. This is the only driver from this list that is not natively supported by Laravel.

I currently use themeilisearch driver as I wanted to "step up my game" because it's free and takes into account typos and many more. But I've also used tntsearch for the last three years as it works great for what's it's worth, I would suggest for you to start with this. It is very fast, results are good enough and doesn't require you to install extra services.

Installing TntSearch

Let's require the package (driver) teamtnt/laravel-scout-tntsearch-driver to your composer :

composer require teamtnt/laravel-scout-tntsearch-driver

Then, update the default value of your config config/scout.php file to tntsearch (or update SCOUT_DRIVER in your .env):

'driver' => env('SCOUT_DRIVER', 'tntsearch'),

Finally, because tntsearch is not natively supported by Laravel, the configuration parameters is not present by default, so let's add them manually at the end of the config/scout.php file :

/*
|--------------------------------------------------------------------------
| MeiliSearch Configuration
|--------------------------------------------------------------------------
|
| Here you may configure your TntSearch settings. TntSearch search
| is an open source PHP engine. No extra services are required
| and all indexes are store locally in `.index` files.
|
| See: https://github.com/teamtnt/laravel-scout-tntsearch-driver
|
*/
'tntsearch' => [
    'storage' => storage_path(), //place where the index files will be stored
    'fuzziness' => env('TNTSEARCH_FUZZINESS', true),
    'fuzzy' => [
        'prefix_length' => 2,
        'max_expansions' => 50,
        'distance' => 2,
    ],
    'asYouType' => false,
    'searchBoolean' => env('TNTSEARCH_BOOLEAN', false),
    'maxDocs' => env('TNTSEARCH_MAX_DOCS', 500),
],

As I mentioned earlier, the package TntSearch indexes your data locally, by default in the /storage folder, you can change the path by updating the "storage" value in the config just above (You could do storage_path('indexes')for example). Make sure you ignore the folder from your git repository as you don't want to commit any indexes by adding a new rule in your ./.gitignore file:

/storage/*.index

Performing a search

Let's start by making sure we have data to work with, let's use our Post model we have configured earlier. Let's use tinker for testing purposes :

php artisan tinker

Post::create(['title' => 'Upgrade to Laravel 9']);
Post::create(['title' => 'How to create a CRUD app with Laravel']);
Post::create(['title' => 'Upgrade to Laravel 10']);

Now that we have added data, scout has told tntsearch to index your data, a new post.index file has been created in your storage folder (remember, we're currently working with the tntsearchdriver, other drivers do not store anything locally). Have a look !

Info: You might notice it's actually a database, and with the right software, you can open the file and inspect what has been stored inside.

Okay let's do this ! Luckily Scout makes it very easy to perform a search, a new static method is now at your disposal called search which will return a query builder, so make sure you then call ->get() :

$posts = Post::search('laravel')->get();
$posts->count(); // 3

// TntSearch will understand even if you make a typo !
$posts = Post::search('laravle')->get();
$posts->count(); // 3

It will return a collection of posts, as you've always worked with ! It just works, Scout will take care of performing the search, communicating with your driver and returning your data as a model. You may also paginate your results :

$posts = Post::search('laravel 10')->paginate();

And that's it !

This is where the Laravel Scout train terminus drops you off. The package will perform the search for you and return the collection of models. It's now up to you to decide what to do with it. You may create a dedicated page to display your search results, you may display results in a pop-up or simply filter the original liste.

Here's how the simplest controller would look like :

<?php

namespace App\Http\Controllers;

use App\Models\Post;
use Illuminate\Http\Request;

class PostsController extends Controller
{
    public function search(Request $request)
    {
        $posts = Post::search($request->get('search'))->paginate();            

        return view('posts.search-result', compact($posts));
    }
}

If you've enjoyed this article, you should definitely follow me or add a clap to this article as it would make my day (really, seeing a notification saying someone has enjoyed this article is the best thing I could get 🙏🏼)

Commands

Even though your app is working perfectly, there are times you will need to tweak or debug a few things, so let's go over a few commands you should be aware of.

php artisan scout:flush Post

This will flush the index stored for the given model (here Post), no index = no search. But sometimes you need to reset things because you've changed the list of columns indexed on the model, usually once you flush you then do:

php artisan scout:import Post

As suggested, it will import and index all model for the model Post. If your application already has data in your database when installing this package, you will definitely need to run this on your production server. And lasty:

php artisan scout:status

 🔎 Analysing information from: [App\Models\Post,App\Models\User]

 2/2 [▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓] 100%

 ----------------- ------------- ----------------- --------------- ------------ -------------------- 
  Searchable        Index         Indexed Columns   Index Records   DB Records   Records difference  
 ----------------- ------------- ----------------- --------------- ------------ -------------------- 
  App\Models\Post   posts.index   id,title          3               13           10                  
  App\Models\User   users.index   id,name           19              10           Synchronized        
 ----------------- ------------- ----------------- --------------- ------------ --------------------

The status command will display, for each model that has the Searchable trait, its index status by comparing which model has been indexed and left out.

Conclusion

We've learned the basics of this package, how to install it in your Laravel project, how to enable indexing on your models and how to perform a search. You still have to manage the visual aspect to correctly display results, adapt this to your website style !

We've also discovered there are different drivers, for now you should use a driver that doesn't require much installation. TntSearch is a great compromise because it return relevant results and doesn't require you to have a separate service running.

This article was part 1 of 3, you should continue reading as we will get to understand more about this package: understanding how it works, its limits and constraints.