Open in app

Sign in

Write

Sign in

Laravel Scout (full-text search) P3: combining search, filter and ordering

A deep dive into the Laravel Scout package, from beginner to advanced. Part 3 shows you how to search, filter and order you data at the same time.

Jori STEIN
7 min readFeb 20, 2023

This is a three part article series, makes sure to read in order to fully understand this topic :

Part 1 — Laravel Scout (full-text search) P1: Installation, configuration & searching (TntSearch)

Part 2 — Laravel Scout (full-text search) P2: limitations, drivers & Builder

Part 3 (this article) — Laravel Scout (full-text search) P3: combining search, filter and order

Summary

  • The Scout Builder instance
  • Filtering data with the search engine
  • Filtering data with Eloquent
  • Ordering data
  • Example combining everything

The Scout Builder instance

When you call the ::search function, you might have noticed that it returns a query Builder instance as you would expect, but it's not an (eloquent)\Illunimate\Database\Eloquent\Builder instance. Instead you get an (scout)\Laravel\Scout\Builder instance.

$eloquentBuilder = User::where('first_name', 'LIKE', '%John%')->orWhere('last_name', 'LIKE', '%John%');
get_class($eloquentBuilder);
// "Illuminate\Database\Eloquent\Builder"

$scoutBuilder = Project::search('John');
get_class($scoutBuilder);
// "Laravel\Scout\Builder"


// In fact, the two instances do not share
// a parent class and they do not offer
// the same functions and features
$scoutBuilder instanceOf $eloquentBuilder; // False

Your typical Eloquent builder offers many functions to configure your database query, such as ->where() , ->whereRelation , ->count() , ->with(), etc. which will then be converted into SQL as some point.

On the other hand Scout builder only has a few functions, the list is so small I can list then here:

  1. ->where
  2. ->whereIn
  3. ->orderBy

Those functions will only configure your search engine (Meiliseach, Algolia, etc.), it will tell delegate the filtering to your driver. This means your data can be filtered while searching and/or while retrieving your data from your database.

Filtering data with the search engine

As just explained, when calling ::search you get a Scout builder instance Laravel\Scout\Builder. So following code will tell your driver to filter with the given conditions :

$builder = User::search('John');
$builder->where('is_admin', true);
$builder->where('salary', 50000);

Let’s say you are using Meilisearch, the above code will tell Meilisearch to search all users with keyword “john” but only on those who are admin with a salary above 50 000. But here’s the thing to note : for a driver to filter, it needs to have indexed the data.

class User extends Model
{
    use Searchable;

    public function toSearchableArray(): array
    {
        return [
            'id' => $this->id,
            'name' => $this->name,
            'is_admin' => $this->is_admin,
            'salary' => $this->salary,
        ];
    }
}

There are two mains reasons why your would want to do this :

  1. If you database becomes so large that even MySQL is starting to get slow to filter, this is very unlikely as not all companies actually do reach such dataset.
  2. You can search on your indexed data, which means you can filter on data other than your columns. You can index and filter on data that is hard to query or that needs lots of time/processing power, here are some examples :
// Here we are indexing the "number of days off" and "is admin"
// as they would otherwise require complexe query
// with LEFT JOINs, EXIST and SUB queries.
class Employee extends Model
{
    use Searchable;

    public function toSearchableArray(): array
    {
        return [
            'id' => $this->id,
            'days_off_count' => $this->getDaysOffCount(),
            'is_admin' => $this->premiumOfferActive() && $this->hasPermission('admin_access'),
        ];
    }
}


// Here, we are calculating the total of the invoice, otherwise
// it would require to calculate the total for each invoice
// in the database when a filter is requested
class Invoice extends Model
{
    use Searchable;

    public function toSearchableArray(): array
    {
        return [
            'id' => $this->id,
            'total' => $this->getTotal(withDiscounts: true, withTaxes: true),
        ];
    }
}

⚠️ There is a final step. When filtering with your driver you must tell it which data is used for searching and which data is used for filtering. In Algolia this can be found in the index settings after the indexed has been created and with Meilisearch this can be configured inside of you config/scout.php file (https://laravel.com/docs/9.x/scout#configuring-filterable-data-for-meilisearch).

Filtering data with the database

Depending on how you implement your search engine, you might want to filter the data from your database. As we've just seen, you no longer have access to your eloquent instance, don't worry Laravel Scout isn't letting us down just yet.

You may call ->query(...) from your Scout builder instance (https://laravel.com/docs/9.x/scout#customizing-the-eloquent-results-query) to gain access back to your Eloquent builder:

use Illuminate\Database\Eloquent\Builder;

[...]

$builder = User::search('John')->query(function ($query){
    // get_class($query); "Illuminate\Database\Eloquent\Builder"

    return $query->where('is_admin', true);
});

You may do everything that you are used to do:

use Illuminate\Database\Eloquent\Builder;

$builder = User::search($request->get('search'))->query(function (Builder $query) use ($request){
    return $query
        ->with(['addresses', 'tags', 'role'])
        ->where('is_admin', true)
        ->isActive() // scope
        ->whereRelation('addresses', 'country', 'France')
        ->when($request->has('role', function ($query) use ($request){
            $query->where('role_id', $request->get('role'));
        }));
});

This will become a common approach if you are listing data and want to give the option to your users to search and filter the data at the same time :

Now, it is important for me to tell you one last thing about sorting data. This approach means you are always performing a search even if the user has not requested it, this is because you still need access to your filter which are now written in your ->query(...) function which is available only after ::search(''). This is fine, as most driver will consider a search with an empty value (null or'' ) as "the placeholder" which means it will pretty much ignore your search and return all of your data instead of an empty set. ⚠️ The only exception is the driver tntsearch because it's the only one that works differently on that regard, sending null to this driver returns an empty set, which will mess with your results.

Ordering data

By default, results are given by the drivers in a specific order, by search relevancy (very useful with Algolia and Meilisearch). Which means it will override and ignore any order you have set in your SQL query, this might lead to unexecpted restults, example:


$builder = User::search('John')->query(function ($query){
    return $query->orderBy('salary', 'desc');
});

dump($builder->get());

// { [
//    {
//     'id' => 2,
//     'name' => 'John',
//     'salary' => 30000,
//    },
//    {
//     'id' => 1,
//     'name' => 'jonathan',
//     'salary' => 90000,
//    }
// ]}

Even though we have asked to order by salary in a descending order, you can clearly see it has been ignore because it wants to show "John" before "Johnetta" as it is more relevant for the given search query.

Ordering our data works the same way as filtering it, you must first index the data in your model :

class User extends Model
{
    use Searchable;

    public function toSearchableArray(): array
    {
        return [
            'id' => $this->id,
            'name' => $this->name,
            'salary' => $this->salary,
        ];
    }

Then you can call ->orderBy on your Scout builder:

$builder = User::search('John');

$builder->orderBy('salary', 'desc');

dump($builer->get());

// { [
//    {
//     'id' => 1,
//     'name' => 'Johnetta',
//     'salary' => 90000,
//    },
//    {
//     'id' => 2,
//     'name' => 'John',
//     'salary' => 30000,
//    }
// ]}

Example combining everything

To end this three part series article, let me finish by giving a concrete example of how I have implemented search, filtering and ordering in our application with Meilisearch.

Let's imagine we have a model called "Intervention", first let's start by indexing our data in our model :

class Intervention extends Model
{
    use Searchable;

    public function toSearchableArray(): array
    {
        return [
            'id' => $this->id,
            'name' => $this->name,
            'reference' => $this->reference,
            'duration_seconds' => $this->duration_seconds,
            'created_at' => $this->created_at,
            'date' => $this->date,
        ];
    }

Then we must tell Meilisearch the purpose of each data, so we update our config/scout.php file (Laravel doc, Meilisearch doc):

'meilisearch' => [
    'host' => env('MEILISEARCH_HOST', 'http://localhost:7700'),
    'key' => env('MEILISEARCH_KEY', null),
    'index-settings' => [
        \App\Models\Intervention::class => [
            'searchableAttributes' => ['name', 'reference'],
            'sortableAttributes' => ['duration_seconds', 'created_at', 'date',],
        ],
    ],
],

Let's sync that with our Meilisearch instance:

php artisan scout:sync-index-settings

Finally, let's build our controller. I know I will be using some code many time so let's create a trait called handleScoutRequest.


namespace App\Http\Controllers\Traits;

trait handleScoutRequest
{
    public function getSearchQuery(Request $request, string $query = 'search'): string
    {
        return $request->str($query)->trim()->toString();
    }

    public function customOrder(Request $request): bool
    {
        return $request->has('sort');
    }

    public function getOrderByColumn(Request $request): ?string
    {
        return $request->str('sort')->trim()->toString();
    }

    public function getOrderByDirection(Request $request): string
    {
        // It is expected to get "/api/interventions?sort=salary" to sort by salary in an ascending order
        // and "/api/interventions?sort=-salary" to sort by salary in an descending order
        $column = $this->getOrderByColumn($request);
        
        return str_starts_with($column, '-') ? 'desc' : 'asc';
    }
}

And then the controller :

class InterventionController extend Controller
{
    use handleScoutRequest;
    
    public function index(Request $request)
    {
        $builder = Intervention
            ::search($this->getSearchQuery($request))
            ->query(function (Builder $query) use ($request) {
                $query
                    ->with([
                        'client',
                        'vehicle',
                    ])
                    ->when($request->has('client'), function ($query) use ($request) {
                        return $query->where('client_id', $request->get('client'));
                    })
                    ->when($request->has('vehicle'), function ($query) use ($request) {
                        return $query->where('vehicle_id', $request->get('vehicle'));
                    });
            })
            ->when($request->customOrder(), function ($query) use ($request) {
                return $query->orderBy($this->getOrderByColumn($request), $this->getOrderByDirection($request));
            });
    
        return InterventionResource::collection($builder->paginate());
    }
}

Conclusion

We have seen many things in this article, Laravel scout allows you to very quickly implement a full text search in your application and communicate with very power driver like Algolia and Meiliearch. Things become trickier onec you want to add features together, which forces you to change how you write your code in your controller.

Those three part series articles has only talked about the Laravel side, there still is the visual aspect to handle:

  • The data format
  • highlight search query in your results
  • Creating a search bar with results from different models.
  • Autocomplete
  • Boolean search
  • etc.

I hope my article has allowed you to be on the right track !

Laravel
Laravel Scout
Meilisearch
Algolia
Laravel Packages

--

--

Jori STEIN

Written by Jori STEIN

19 Followers

Software developer, leading a path to code heaven. Let's do this together !

Help

Status

About

Careers

Press

Blog

Privacy

Terms

Text to speech

Teams