We Help Frameworks Help Us
As technology evolves, the complexity of the problems our software tries to solve get higher. Nowadays, it has become uncommon to see systems that don’t interact with different types of devices or don’t connect with multiple cloud services.
Trying to make all of this parts of the puzzle work together and do it right can become very time consuming and can make us lose focus on the problem we want to solve with our app. We need to be able to dedicate more time to develop the solution to the problem and not so much on other things that aren’t dismissible, but are a chore to work with.
Here at Lateral View we use frameworks like Ruby on Rails that offer us many tools to make our lives easier, cutting down the effort needed to get to solve the actual problem. Many of those repetitive and boring tasks are taken care by Rails.
The issue isn’t that we haven’t done it ourselves before, it’s that we have solved them so many times on almost every app and that it turns into a task that does not add real value to what we are trying to do. It takes up a lot of time that we otherwise could be spendig on making our app awesome.
Frameworks make some of the work for us. That’s great. We can spend less time on boring things and focus on adding value to the app and making it great. But not always things are so ideal. In reality we really can’t blindly trust on what Rails or any framework does for us.
Generally speaking, the way these things are handled are very optimised for most situations, but sometimes we need to make changes to the default behaviour to get things working the way we intended.
Recently, we were working on a project that started to grow its user base. More users equals more data to deal with, and that is where performance issues become noticeable. Some user actions where taking a considerable time and the last thing you want is to make your users wait.
With Rails we used Active Record as an ORM framework that allowed us to forget about dealing with data storage and queries to the database. Even though we have the knowledge to do it manually, it's better to let Active Record deal with it because these kind of tasks usually get repetitive and boring and get in the way of focusing in the real problems to solve. Active Record usually does a great job but sometimes we need to step in to help.
What we did was we dusted off our database knowledge to analyse the queries being done by the app and we found a culprit right away. One query was taking over 10 seconds! It was nothing fancy, it was just aggregating some values but it involved searching over a few million records.
When you are reading a book and need to find something on it, it's better to have good indexes. It makes a huge difference having to read an index a couple pages long to find in what page is what you need rather than reading the whole book. This same concept applies to databases, and the information we needed for that particular query was not found on an index. Good luck searching through about 10 million records, one by one.
To solve this we had to create a new index to be used in this particular situation. A composite index including two table columns did the trick, reducing the execution time from 10 seconds to 20 milliseconds. That's 500 times faster!
These tools are great, but we don't have to forget that sometimes they can need some help from us to work efficiently. Specially if we have the knowledge to get under the hood and tweak things to get them to work the way we need it. Every knowledge bit we have, every previous experience can be useful someday and make a difference. We have to be aware of them and be able to recognize the appropriate time to use them.
