How to Implement Elastic Search With Laravel and Docker (2)

Photo by Oskar Yildiz on Unsplash

In How to Implement Elastic Search With Laravel and Docker (1), we installed Docker on our local server, setup a Docker container for Elastic Search and created a test database called “northwind”. We also imported sample data to our database.

In this tutorial, we’ll connect our Laravel application to the Elastic Search container we created. We’ll setup a model for customers table and create a controller that converts the SQL data in the table to an Elastic Search index

Connecting Your Laravel App to the Docker Container

Let’s index customers table from our Laravel application. First, we’ll add Elasticquent package to composer.json.

"require": {
   ...
   "elasticquent/elasticquent": "dev-master"
}

Now run composer update in your terminal to download Elasticquent to your Laravel project. After composer updates, add Elasticquent to providers array in your config/app.php file.

'providers' => [
   ...
   Elasticquent\ElasticquentServiceProvider::class,
]

We’ll also add a facade for elasticsearch-php client in our config/app.php file.

'aliases' => [
   ...
   'Es' => Elasticquent\ElasticquentElasticsearchFacade::class,
]

Next, publish Elastic Search configuration file to your config directory with this command:

php artisan vendor:publish --provider="Elasticquent\ElasticquentServiceProvider"

After publishing, proceed to app/config/elasticquent.php to set your default index. I’ll just name it “general”. Now the Elasticquent configuration file should look like this:

What Exactly is Elasticquent?

Elasticquent makes working with Elasticsearch and Eloquent models easier by mapping them to Elasticsearch types. You can use the default settings or define how Elasticsearch should index and search your Eloquent models right in the model. Elasticquent uses the official Elasticsearch PHP API.

You could go for a deep dive by visiting Elasticquent GitHub Repository. I recommend that you do that later. For now, let’s proceed with connecting our Laravel app to the Elastic Search container.

Create a Customer Model in Your Laravel App

We previously setup the Northwind database and populated it with tables and data. One of those tables is customers table. Create a customer model by running php artisan make:model Customer. Open the newly created model, import Elasticquent to the namespace and use Elasticquent trait within the model.

namespace App;
use Elasticquent\ElasticquentTrait;
use Illuminate\Database\Eloquent\Model;
class Customer extends Model
{
   use ElasticquentTrait;
   protected $table = 'customers';
}

Adding Elastiquent trait to customer model enables us to index the model. Every row in the underlying table for that model gets converted to documents which are stored inside the default index “general” that we specified earlier in Elastic Search config. If you want to index multiple tables, you might want to store them in different indices. To do that, just add a getIndexName() method to your models and return the index name you wish to assign to it.

class Customer extends Model
{
   use ElasticquentTrait;
   protected $table = 'customers';
   function getIndexName()
   {
      return "customer_index";
   }
}

So when you are indexing, the data from the model goes to the index specified inside getIndexName(). Do we have to create the index somehow? No. If a customer_index does not exist, Elastic Search creates one automatically.
Now that our customer model is ready, indexing every row in customers table is as easy as:

Customer::all()->addToIndex();

But what if we’re dealing with thousands or even millions of rows? The command above could rip our server apart. Let’s create a controller for indexing and apply chunking for optimal performance.

Create an Indexing Controller

On your terminal, cd into the directory where you installed your Laravel app. Run php artisan make:controller ElasticSearchIndexingController. Open the controller inside your chosen IDE — I use PhpStorm by Jet Brains. Use Elasticquent trait and add some methods to the controller.

The selectCustomerTable() method will be used to select the columns we want to index. You may choose to rename the columns using aliases. When creating the schema for customer_index, Elastic Search will ditch the generic SQL column names and adopt the aliases you specified.

deleteCustomerIndex() method deletes the index attached to Customer model (in our case, it’s customer_index). This method is called inside index() method. That way, the previous index is dropped and then re-created to avoid having duplicate data inside the index.

The index() method uses Symphony’s console output to print the progress of indexing to console. We ran a count on the number of rows in customer table and applied Eloquent’s chunk() method few lines later. chunk() method ensures we don’t exhaust server memory while working with large data sets. In our case, chunking fetches 5,000 rows each and indexes them separately until all rows have been indexed. I once tried indexing 100,000+ rows without chunking and got a crashed app after I ran out of memory. With chunking, 100,000+ rows were indexed in less than 2 minutes without issues. Chunking may be useless for our customers table since we have just a few rows; but in production when you’re likely dealing with very large data sets, you never want to leave out chunk().

Next, we’ll use index() method in a Laravel command which will be created shortly. The command will enable us index customers table from terminal.

Create a Laravel Command For Indexing

Photo by Gustas Brazaitis on Unsplash

With terminal still open in your Laravel project directory, run

php artisan make:command IndexCustomerTable

Laravel creates a command in app/console/commands directory. Open the command class in your IDE and fill in the signature and description properties of the class. The handle() method will be called when your command is executed. We will inject our ElasticSearchIndexingController to the handle() method to gain access to its index() function. Here is our command:

Now our indexing command is created. In a real-world application, customers table will change from time to time. Hence the need to re-index. Whenever we need to do that, we just open terminal, cd to our project directory and run

php artisan search:index

…And we’ll have a fresh customer index.

To confirm that customer_index is now available in our containerized Elastic Search application, open your browser and navigate to

http://localhost:9200/_cat/indices?v

You should see customer_index listed in the response.

In How to Implement Elastic Search with Laravel and Docker (3), we will create a controller that enables us run a search on customer_index. We will have a brief look at Elastic Search Query DSL (Domain Specific Language) and how it is implemented in Elasticquent. We will also discover how to paginate our search results.