How to setup Sphinx search engine with your Laravel App

In this article, I will be talking about the Sphinx search engine and how to use it to install it on the Windows operating system, use it in your Laravel app with Laravel Scout.

Introduction

Sphinx (SQL Phrase Index) is a standalone full-text search engine that provides efficient search functionality to third party applications, especially SQL databases. This search engine was developed in 2001 by a Russian developer named Andrew Aksyonoff to guarantee a (1) good search quality, (2) performed at high speed (3) with a low resource consumption (Disk IO, CPU). It can be integrated with scripting languages such as Python and Java.

The Sphinx search engine has its own data source drivers that are used to interact with different database management systems. We must specify the driver we need in the configuration files.

In a research paper published in 2017 by a group of researchers at Moscow Technological University, a quick comparison is made between four popular search engines (Sphinx, Apache Solr, ElasticSearch, and Xapian). The result (shown in the table below) shows that the Sphinx search engine has the fastest indexing speed (4.5 Mb/sec) and a very fast search speed (7/75 ms).

Setting up Sphinx

First of all, you should download the latest version (for now, the latest version is 3.4.1) of the Sphinx search engine from the following link.

After downloading the binaries package, you should extract its content.

After extracting the package, we should add a folder called “data” within the extracted directory to store indexes. Then we should create three folders called “index”, “log”, and “binlog” within the created “data” directory.

It is good to know that Sphinx has two primary services:

1. Indexer: This service is used to build full-text indexes. By default, Sphinx read the source tables from the configuration file located in “<installation directory>\etc\sphinx.conf”
2. Searchd: This is the daemon used for searching the created indexes. It requires a client to access the Sphinx API.

Next, you need to create sphinx.conf file inside \etc\ folder.

#
# Minimal Sphinx configuration sample (clean, simple, functional)
#

source database
{
    type            = mysql

    sql_host        = localhost
    sql_user        = root
    sql_pass        =
    sql_db          = tenants_4
    sql_port        = 3306  # optional, default is 3306

}

source src_people : database
{
    sql_field_string = name
    sql_query = SELECT id, name from people
}

index people
{
    type            = plain
    source          = src_people
    path            = ../data/index/people

    min_prefix_len = 3
 
    index_exact_words   = 1
    expand_keywords     = 1
 
    # charset_type = utf-8
}


index testrt
{
    type            = rt
    rt_mem_limit    = 128M

    path            = ../data/index/testrt

    rt_field        = title
    rt_field        = content
    rt_attr_uint    = gid
}


indexer
{
    mem_limit       = 128M
}


searchd
{
    listen          = 9312
    listen          = 9306:mysql41
    log             = ../data/log/log.txt
    query_log       = ../data/log/query.log
    read_timeout    = 5
    max_children    = 30
    pid_file        = ../data/searchd.pid
    seamless_rotate = 1
    preopen_indexes = 1
    unlink_old      = 1
    workers         = threads # for RT to work
    binlog_path     = ../data/binlog/
}

And then, through the following two commands, you can index, start searchd daemon.

searchd --install --config E:\sphinx-3.4.1\etc\sphinx.conf --servicename SphinxSearch

indexer --all --config E:\sphinx-3.4.1\etc\sphinx.conf --rotate --print-queries

These commands should be executed in <installation directory>\bin\ folder.

You can create the cron to re-run the indexes once a day at midnight.

Ok, done. You are setup all the configuration files needed in Sphinx server.

Use Sphinx in your Laravel App with Scout

Now, it’s time to use sphinx daemon to connect to Laravel Scout.

Install the plugins we need for Laravel.

composer require laravel/scout:6.0.0
composer require constantable/laravel-scout-sphinx
php artisan vendor:publish --provider="Laravel\Scout\ScoutServiceProvider"

This command will publish the scout.php configuration file to your config directory, which you can than edit and set sphinxsearch as the Scout driver.

'driver' => env('SCOUT_DRIVER', 'sphinxsearch'),

To configure the connection to Sphinx server add the following (i.e. default) connection options.

  'sphinxsearch' => [
  'host' => env('SPHINX_HOST', 'localhost'),
  'port' => env('SPHINX_PORT', '9306'),
  'socket' => env('SPHINX_SOCKET'),
  'charset' => env('SPHINX_CHARSET'),
  ],

Then you need to add these into the providers in config/app.php.

Constantable\SphinxScout\ServiceProvider::class,
Laravel\Scout\ScoutServiceProvider::class,

Alright! That should get Sphinx all set on the server, and ready to go in Laravel. In all models you want to be searchable, add this (also, they all need to have indexes defined in Sphinx settings).

use Laravel\Scout\Searchable;
 
use Searchable;

The following is the example search function.

$people = App\Person::search($keyword, function (SphinxQL $query) {
        return $query->groupBy('description');
    })

Ok, all is done. You are now aware of how to use Sphinx engine in you Laravel App.

Conclusion

In this tutorial, I’ve tried to outline the main aspects of setting up and configuring Sphinx, using it in your Laravel App.

As you can see, by using this search engine, you can easily add a custom search to your Laravel Website.

Thank you!