Ruby on Rails and DataTables plug-in

Image by Tonya Smyrnova

Overview

Hi there! Are you a Ruby on Rails developer? Ever thought about why you should set up on backend all the pagination, sorting, searching and other stuff for the tables you’re showing in your views? I mean, that’s not a big deal but it means more code in your controllers, code in different actions similar enough to annoy you and still too different and insignificant to make you want to refactor it by let’s say creating a class where you’ll be able to initialize any table and provide additional functionality to it in the same way.

Wouldn’t it be great to have one tool to handle all the tables in your app, to get one ring to rule them all?

Or maybe your tables are not even remotely big and it would be great just to throw the whole dataset to the view, so no one bothers your database with additional requests of retrieving every single page or applying the sorting or performing the search. Why can’t browser do that within the view you just sent to it?

Actually, it can. And actually, it can do that pretty much efficiently with this DataTables plug-in for JQuery which can be the solution for all of your tables. If you go to https://datatables.net/ and check out the docs, you’ll see that the tool is just neat! Also, it offers you features which can cover almost everything you ever wanted to do with your tables.

Let’s start this tutorial with a short plug-in overview so that we can move to some advanced examples later.

So, if you have any valid HTML table in your view

You can get all the DataTables functionality instantly with a simple JQuery code that may look like this

Notice that we initialize our datatable on the particular HTML table with your-table-idid. That’s how you tie DataTables functionality to your tables.

Now, look at your brand new table with column ordering, pagination, instant searching powered by Javascript and basic Bootstrap styling! Isn’t it great?

But you can say that’s just a static HTML! What if I use tables served with the data from my backend?

Worry not! The plug-in supports different sources of data for datatables. You also may use datasets from Javascript arrays or even arrays of objects

Your datatable initialization will now look like the following

Furthermore, you can serve your table with the dataset returned by Ajax

The dataset in this case obviously must be returned in JSON format.

Basically, all the things you’ve seen so far rely on the browser and some Javascript code that offers you a possibility to work with your datatable on the client side. It’s called Client-side processing in DataTables and that’s the default option. But if your table contains hundreds of thousands or millions of records reading data from the DOM will naturally be slow and Client-side processing won’t be good.

That’s where Server-side processing kicks in. When things become slow with a huge amount of data you may do your pagination, ordering and searching on the backend and return the small portions of processed data to the datatable.

You’ll need to set serverSide property to true

Write some code on your backend to process the requests and return JSON data along with additional parameters needed for DataTables such as a number of filtered records etc.

Static HTML tables

With DataTables, your static table instantly gets pagination, sorting by columns and searching. After the table is initialized, all the things including the searching work fast and smooth.

The original docs are very useful but still, every new thing you learn may include something that will confuse you. Also, the docs don’t tell how to use the library in your Ruby on Rails project.

So let’s start with some examples to see how to use the plug-in in your Rails application.

I suppose you’re familiar with Ruby, Ruby on Rails, HTML, SCSS, Javascript and Git to the extent enough for continuing with this tutorial.

If you can’t wait to look at the working setup of Rails application with DataTables plug-in, check it out here. Though I encourage you to go through all the steps below.

Set up a Rails application

I’ll be using Rails 5.2.1 in this example and the details may vary for other versions. Also, I’ll add Twitter Bootstrap so you may want to add it to your project too.

So, create a Rails application via usual flow so that you can see Rails standard homepage or clone the repo mentioned above and continue from this step.

Let’s add the following gems to our Gemfile

gem 'jquery-rails'
gem 'bootstrap-sass'

run

bundle install

and start Rails server

Now we require the files in app/assets/javascripts/application.js so that it looks like this

It’s time for a little disclaimer. I’m not going to expand this tutorial with the minor things it isn’t meant to explain, like avoiding //= require_tree . in application.js file etc. Just wanted to focus on how to use DataTables plug-in without making you distracted by all kinds of best practices that might be used in the Rails application which are not necessary here. That’s all, let’s keep going.

Now let’s change app/assets/stylesheets/application.css file extension to .scss, remove its current content and import Bootstrap files. The file will look like

At this point, you should have Bootstrap working.

Now we’re going to use rails generate controller pages home command to create PagesController with the empty home action which does nothing except rendering home.html.erb view.

Let’s put some HTML in our app/views/pages/home.html.erb view

And some CSS in app/assets/stylesheets/pages.scss file which should be there by now. If not, create it manually

And import it to app/assets/stylesheets/application.scss

Add this to config/routes.rb file

Start your server and go to localhost:3000 in your browser

Got something like this?

Not much, huh? Bet you’d love to make it shine! And that’s actually what this tutorial is about. So finally it’s time to add DataTables plug-in to our application.

Add DataTables

You might hear that in Rails community there’s always a gem for everything you may need. It’s true and if you’ll be struggling with following this tutorial, try to use jquery-datatables-rails.

The gem is really great and may be the best choice for your Rails application, but there’s the reason why I wanted to tell you how to add DataTables to your project manually. The thing is, by the time this tutorial was written, jquery-datatables-rails gem didn’t support the cutting-edge 1.10 version of DataTables. It supported version 1.9 but unfortunately, DataTables 1.10 is not backward compatible with DataTables 1.9 out of the box. It means that if we were using version 1.9 in our app and then wanted to move to version 1.10 according to the docs we would need to take some steps to convert the code we have so that it works with the new DataTables version. So I thought if you already have DataTables 1.9 in your project, why would you need this tutorial? And if you’re reading this and never used DataTables why would you need to use version 1.9 which works great but is a bit outdated already?

Let’s go with the last DataTables version instead and learn how to add the plug-in to your Rails project manually!

Almost all you need to do is to include to your project some Javascript code and CSS. If you look at the release on DataTables CDN you’ll see a bunch of Javascript and CSS files along with images. We won’t need all of them, at least for our little app.

Let’s start with adding a javascript file

//cdn.datatables.net/1.10.19/js/jquery.dataTables.min.js

Minified version of JS file would be ok because we’re not going to (and we shouldn’t) make any changes to it. So download the file and put it in vendor/assets/javascripts/ directory.

The things are a bit more difficult with CSS file

//cdn.datatables.net/1.10.19/css/jquery.dataTables.css

To make it work with Ruby on Rails we’ll need to edit it. So download full (not minified) file and put it in vendor/assets/stylesheets/ directory.

Change the file extension to .scss, find the following part (begins at line 45)

And replace it with

We had to do this so the CSS could properly refer to images through Rails Asset Pipeline.

Download the images

//cdn.datatables.net/1.10.19/images/sort_asc.png
//cdn.datatables.net/1.10.19/images/sort_asc_disabled.png
//cdn.datatables.net/1.10.19/images/sort_both.png
//cdn.datatables.net/1.10.19/images/sort_desc.png
//cdn.datatables.net/1.10.19/images/sort_desc_disabled.png

and put them in vendor/assets/images/

Now we have all the files on their places. Require them in app/assets/javascripts/application.js

And import to app/assets/stylesheets/application.scss

We’ve come a long way and everything should be ready by now. Though if you take a look at our table again you’ll see that it’s as dull as it was before. Yes, we added DataTables plug-in to our application but we didn’t do the main thing. We didn’t tie DataTables functionality to our table in app/views/pages/home.html/erb file.

So open app/assets/javascripts/pages.js file (or create it if you don’t have it or change its extension if you’ve got pages.coffee file instead) and add the following

Boom! You’re now able to use your advanced HTML tables. Just that simple. Restart Rails server and look at your brand new table with all the fancy features provided by DataTables plug-in!

Ajax-sourced tables

Now let’s take a close look at how we can deal with serving tables with datasets provided by Ajax request.

You can continue with the code you have after going through the previous part or just start from this commit. Want to see the code we’re about to write in this part? Here you go.

Set up Ajax-sourced datatable

If we talk about sending some asynchronous requests with Ajax, naturally we’ll need some API endpoint which would return us the dataset that we, in turn, will use for populating our table. But since I don’t feel like creating the whole REST API project for one single endpoint, I’m sure we’ll be fine with just a regular controller action rendering some JSON.

Next problem is a dataset. Apparently, we need some data to put into the table, right? Luckily a co-worker of mine came across a nice public dataset of South Park script lines along with seasons, episodes and characters. And I thought if it won’t help us, then what will?

Before I started to actually write this I created Line model in the project and couple rake tasks for populating lines table with the dataset.

So, if you’re working on your own project, create the same model by executing

rails g model line season:string episode:string character:string line:string

If you cloned and set up locally my project, then the model should be there.

Now, look at app/lib/tasks/data.rake. You’ll find there a task for populating Line model with two lines of every episode of every South Park season. I thought we wouldn’t need all the ~70 000 lines from the dataset in our table and created the task so that it stores only some of the lines. Though feel free to edit the task to make it populate the table with any lines you want.

Run the task

rake data:populate

And check if everything went right by looking what’s in your table. You should have some number (514 if you ran my task without changes) of the records like

#<Line id: 1803, season: "10", episode: "1", character: "Stan", line: "You guys, you guys! Chef is going away.", created_at: "2018-08-09 16:43:53", updated_at: "2018-08-09 16:43:53">

Awesome! We’re now able to proceed and we’re gonna create a new controller action for retrieving the dataset.

So add this to config/routes.rb file

And this to app/controllers/pages_controller.rb file

As you can see get_dataset action which will be executed after sending GET request to /get_dataset path will take all the lines from the database and render them as JSON. You may ask why we render not just the lines but a hash, which have these lines associated with data key. It’s because DataTable by default is looking for a dataset under data property of the object returned by Ajax request. We’ll change this later.

Let’s add a new table to app/views/pages/home.html.erb file. It will be exactly the same as the previous one except it won’t have any rows and cells in it. They will be generated automatically from the dataset. Also, the table should have a unique id (ajax-table) so we could tie DataTables functionality to this exact table.

Now all we need to do is to add this to app/assets/javascripts/pages.js file

The result is actually great! After the page was loaded, Ajax request was sent to our controller action and the table was populated with the dataset. We got searching, ordering and pagination instantly. Try to play with it and make sure everything works.

Let’s go back to our code. See, we set ajax property with the URL to our action and also initialized columns property with title’s (the titles which are now shown in the table header line) and data’s (the keys that DataTables will be expecting to get the value for the current column).

In your browser developer tools, you can find the Ajax request that was sent while initializing the table. Its response should look like

{"data":[
{"id":1803,"season":"10","episode":"1","character":"Stan","line":"You guys, you guys! Chef is going away.","created_at":"2018-08-09T16:43:53.493Z","updated_at":"2018-08-09T16:43:53.493Z"},{...},...,{...}]}

So datatable goes to data property of returned serialized object (by default), gets the array of hashes

{"id":1803,"season":"10","episode":"1","character":"Stan","line":"You guys, you guys! Chef is going away.","created_at":"2018-08-09T16:43:53.493Z","updated_at":"2018-08-09T16:43:53.493Z"}

Then it finds season, episode, character and line keys in each of them, reads their values and puts them into correspondent cells.

Notice that we’re rendering a set of regular records from our database which naturally contain id, created_at and updated_at columns. But since we didn’t mention them in the table initialization DataTables ignores them completely.

Add them to the table initialization and look at the changes if you want.

Additional options for datatable initialization

So we’ve initialized our datatable successfully. Now it’s time to learn what else we can do with it.

DataTables plug-in provides several additional options you may use. For example, if you don’t want to have the records in your table ordered by the first column (which is the default setting), update the constructor in this way

If you reload the page, you’ll see that the table is ordered now descending by Character column.

If the API you’re working with, doesn’t give you dataset under data key, don’t worry — DataTables is smart enough to handle this. You just need to set one more parameter. Let’s update the table initialization code so that it expects the dataset under lines key

We also need to update get_dataset action according to the last changes in the datatable constructor

Still works? Brilliant! This setup will make it work without expecting any key

Actually, there are much more options you may set within DataTable constructor.

If your table has a large number of records and the rendering takes too much time, add deferRender: true and the records will be rendered only when it’s actually needed to show them. For example, after page loading, only the records for the first page will be rendered, and clicking on another page will trigger rendering the next portion of the records.

Another example is if you add stateSave: true your table will store such info as paging position, filtering, ordering etc even after page reloading.

Feel free to check the full list of available options here and to experiment with their combinations.

Styling

DataTables offers numerous default styling options and can be easily integrated with tools like Bootstrap or Foundation. I just wanted to mention the common case when your table cells contain not only text but different HTML elements such as buttons, media etc. Obviously, you need to customize the styles of your columns separately and here’s how you can do that.

Everything’s simple and straightforward. Just hang CSS classes on your columns and style them as you want. Look at your table cells HTML code in browser developer tools

They’re simple <td>’s and only one of them has sorting_1 class. This is the cell of Character column to which we set default ordering.

Initialize all the columns except Line with class property

If you reload the page you’ll see that Season, Episode and Character all got the classes that we initialized them with. Line still has nothing but that’s ok.

Now add the following CSS to app/assets/stylesheets/pages.scss file

And look at the result

There’s nothing really special there. We just changed the appearance of table cells and header and also we updated the styles of our column classes separately. Yeah, surprisingly one can customize their datatable exactly like a regular table because it is a regular table after all.

Server-side processing

Now we’re going to set up another table and get familiar with Server-side processing.

As it was mentioned before, Server-side processing comes to the rescue when your table operates with millions of records and reading data from the DOM becomes unbearably slow. In this case, all you can do to avoid such a nightmare is performing searching, ordering and pagination on the backend, before you stick the data into the DOM.

As usual, you can continue with the code you got after the previous part or start from here. If you want to get a sneak peek at the result of this part look here.

Funny how we can duplicate almost all the code that had been written for the Ajax-sourced table and use it for a brand new table we’re gonna serve with Server-side processing. Really, just add one more route to config/routes.rb

One more action to app/controllers/pages_controller.rb

One more table to app/views/pages/home.html.erb

And one more DataTable constructor to app/assets/javascripts/pages.js. OK, this constructor won’t be exactly the same as the one for ajax-table. We will add serverSide property set to true and of course, will change url so that it matches the new action. But the rest is the same

Restart Rails server and check out the result. Looks good! Oh, wait… Not really good. Not good at all.

It seems all 514 records are going on the first page while there are at least 5 pages and also it says “Showing 0 to 0 of 0 entries (filtered from NaN total entries)” which is just silly. More to come — search is not working at all.

The thing is when Server-side processing is enabled, DataTables sends a bunch of additional parameters within Ajax request. And to handle searching, ordering and pagination we have to return additional parameters along with the dataset back from the controller action too.

Hate to bombard you with huge chunks of code, but I have no choice. Here are the parameters come to our action from DataTables

{
"draw"=>"1",
"columns"=>{
"0"=>{
"data"=>"season",
"name"=>"",
"searchable"=>"true",
"orderable"=>"true",
"search"=>{
"value"=>"",
"regex"=>"false"
}
},
"1"=>{
"data"=>"episode",
"name"=>"",
"searchable"=>"true",
"orderable"=>"true",
"search"=>{
"value"=>"",
"regex"=>"false"
}
},
"2"=>{
"data"=>"character",
"name"=>"",
"searchable"=>"true",
"orderable"=>"true",
"search"=>{
"value"=>"",
"regex"=>"false"
}
},
"3"=>{
"data"=>"line",
"name"=>"",
"searchable"=>"true",
"orderable"=>"true",
"search"=>{
"value"=>"",
"regex"=>"false"
}
}
},
"order"=>{
"0"=>{
"column"=>"2",
"dir"=>"desc"
}
},
"start"=>"0",
"length"=>"10",
"search"=>{
"value"=>"",
"regex"=>"false"
},
"_"=>"1534344124743"
}

What to notice here? The parameters contain draw counter which surprisingly counts the number of draws i.e. actions performed on the table until page reloads. The docs strictly order to send it back to DataTables every time as an integer. And who are we to disobey?

There’s also columns parameter which contains a signature of every column including the flags which show if the column is searchable/orderable or not. That’s the columns for which we’ll have to set up searching and ordering later.

There’s the info about ordering under order key. As you can see now, it tells that the table is ordered by column “2” (which is Character) and the ordering direction is “Desc”. Seems to be correct because if you remember, we initialized our table with default descending ordering by Character column.

The next parameters are start which stands for the number of the first record that must be shown and length which is basically the number of records on a page (10 by default).

And the last parameter we’re interested in is search. It contains the value that we should search for in all the searchable columns and the regex flag which tells whether we should search for it as a regular expression or as just a plain string.

What else should we return along with the dataset and draw counter? There are several parameters in the docs but we’ll be fine with only two of them: recordsTotal as the total number of records in our database and recordsFiltered as the number of records left after a search was performed.

So let’s code a little! Update get_processed_dataset action so that it looks like

What do we do here? We take search value params[search']['value'] and columns signatures params['columns'] from params and use them to perform datatable_filter on our lines. Then we count the number of the lines we got after filtering.

Eventually, we render the filtered lines, draw counter which we just took from params and converted to an integer, the total number of lines we have in our database and the number of filtered lines.

Everything should be great by now except we don’t have datatable_filter method implemented. That’s a shame. Let’s fix this by adding the following code to app/models/line.rb

We defined datatable_filter class method which consumes the search value and the columns signatures. If search_value is blank, it simply returns all the lines we have in the database. Otherwise, it goes through the passed columns signatures, constructs the query by matching search_value against every searchable column and returns the resulting query.

DATATABLE_COLUMNS constant was defined to match the columns we use in the datatable against their indexes as they’re matching by DataTables.

Let’s look at our table now.

It still puts all the records on a single page, the order is random but the search is working now. And the line below the table shows correct info about how many lines we have in total and how many of them were filtered. That’s what I call progress! Let’s not stop!

To fix ordering update get_processed_dataset action

And add datatable_order class method to app/models/line.rb so that the file looks like

We get column index and ordering direction from params and pass them to datatable_order method which takes the column by passed index and then apply an ordering by that column and the direction to our previously queried lines. As you can see column index and ordering direction both come within a nested hash under “0” key. That means that there can be more than one hash like this i.e. more than one ordering condition. Though I won’t cover this use case and make the tutorial even bigger, I’d recommend you to work it out by yourself.

Refresh the page and see how the ordering got back in place and can even be changed now by clicking on column headers as it was before. One major difference is it’s the application backend who’s in charge of this work now.

Ok, there’s one more thing we still need to do and it’s the easiest part — pagination. Instead of implementing pagination by ourselves we’re gonna pass the buck to well-known gem Kaminari. So add gem 'kaminari’ to your Gemfile, run bundle install and restart Rails server. Great, now you can update get_processed_dataset action with some pagination code

We used length and start values from params which stand respectively for records per page number and the number of the record we need to start with. Then we apply them to the lines we queried and ordered before so that the action returns only the rows that were requested for current page by DataTables. Notice that we had to increment start number because Kaminari’s first-page number is 1 rather than 0 as it is in DataTables. Also, remember to get count of lines before you perform pagination and only 10 (or 25 or whatever) records left in lines variable.

For you could comfortably compare the Client-side processed table with the Server-side processed one I changed markup in app/views/pages/home.html.erb to this

The tables are almost identical. The searching and pagination work with the same results.

The ordering is similar too but there’s one minor distinction. When the tables are ordered by some column e.g. Character and there are multiple rows with the same value in this column, the order within those row groups may vary. That’s because we didn’t do any work to handle this, at least on the backend. This case is subtle and, as usual, I encourage you to experiment with it. What else you could experiment with, is using a really large dataset with at least 1 million records and comparing the speed of Client-side processing and Server-side processing implemented on the backend. Notice that ILIKE queries are not the fastest ones, so you may want to use some solution for full-text search, for example, this gem or even something big like ElasticSearch and Sphinx.

Conclusion

We have learned how you can instantly make your tables more powerful. As you could see, adding DataTables to your Rails application went mostly easy and smooth though it has not been without problems we needed to deal with.

We’ve set up the table with Server-side processing. At the beginning, I told you about DataTables plug-in as a solution which could allow you not to hold an extra code for pagination, ordering and searching in your controllers but eventually, we wrote this exactly extra code in the controller. To be honest that’s OK because you’ll never need to do this unless you deal with really huge tables in your app. But even if you do, it’s still nice to have one single tool to manage all 25 tables in your application. I mean even if you have to implement things on the backend for one of your tables which is huge, there are still 24 more tables which DataTables will handle for you.

Thank you for going through this tutorial with me, hope it was helpful and you liked it! And again, there is always much more info in official docs so if you decided to use DataTables in your projects https://datatables.net/ is at your service.