Rails App Performance: In-Memory Calculation Technique

Learn how to implement and get benefits from In-Memory calculations for faster data processing and optimized app performance

Most of the time, we as developers think that let’s just write the code for now, then later we will optimize it, but as the saying goes, “Tomorrow never comes.” So the best practice would be to write code in an optimized way right from the start.

There are many ways to increase the performance of the application; it is a vast topic and subjective to the application you are developing.

There are some ways that allow you to improve app performance:-

1. Use of caching
2. Using indexes for a required table in the database (try not to overdo indexing; it may also lead to lower performance)
3. Fetch only when and what data is required and try to reuse the fetched data from a database as much as possible.
4. For fetching a large amount of data, use batches.
5. Don’t write queries in a loop.

Among many techniques for improving performance, one of the best techniques is an in-memory calculation which is preferable over querying. Let’s understand it.

What is In-Memory Calculation?

In-memory calculation in Rails involves using memory as a temporary storage space for data and performing calculations directly on that data; rather than fetching it from a database each time it is needed. This can be useful for calculations that are performed frequently or for calculations that require real-time data.

For example, imagine having a list of 100 numbers. Now, you want to calculate their sum. Usually, you would go through the list one by one, add each number to a running total, and get the final sum. But, if you store the 100 numbers in the memory of the computer, you can perform the calculation much faster by simply adding all the numbers together in one step.

The first technique is using data structures

Data structures like arrays and hashes: Applications can store data in memory using arrays, hashes, and other data structures to store and process data in memory.

Example using hash:

# Fetch the latest stock prices from the database
stock_prices = StockPrice.all

# Store the data in a hash for easy access
stock_prices_hash = {}
stock_prices.each do |stock_price|
  stock_prices_hash[stock_price.symbol] = stock_price
end

# Perform the necessary calculations on the in-memory data
percentage_change = stock_prices.map do |stock_price|
  (stock_prices_hash[stock_price.symbol].current_price - stock_prices_hash[stock_price.symbol].previous_price) / stock_prices_hash[stock_price.symbol].previous_price
end

Here the stock prices are fetched from the database and stored in a hash. This allows for fast access to the stock prices using the symbol as the key, making it easier to perform the necessary calculations on the in-memory data.

Example using an array:

Suppose we want to find names that are not present in DB

names = ["TestUser", "User1", "User2"]
na_names = []
names.each do |name|
  unless User.where(name: name).exists?
    na_names << name
  end
end
na_names

The above query required N queries to get the result. Instead of this, we can write a single query to find the users and do the other calculation in memory:

irb(main):026:0> existing_name = User.pluck(:name)
      User Pluck (0.7ms)  SELECT "users"."name" FROM "users"
  => ["Test", "User2", "User3"]
irb(main):027:0> na_names = names - existing_name
  => ["TestUser", "User1"]

In this example, the existing_name is fetched from the database each time the calculation is performed. While this approach still benefits from the speed of in-memory calculation, it will be slower than if the data were stored in a cache, as the data will need to be fetched from the database each time it is needed.

Another Technique:-

Storing data in the cache for in-memory calculation

Storing data in a cache is a common implementation of in-memory calculation, where the data is stored in memory and persistently kept there so that it can be retrieved quickly in subsequent requests. This allows for faster access to the data compared to fetching it from a slower source like a database or file system each time it is needed.

# First, fetch the latest stock prices from the database and store them in the cache
Rails.cache.write("stock_prices", StockPrice.all)
# Now, instead of fetching the stock prices from the database each time, we can retrieve them from the cache
stock_prices = Rails.cache.read("stock_prices")
# Perform the necessary calculations on the in-memory data
percentage_change = stock_prices.map do |stock_price|
  (stock_price.current_price - stock_price.previous_price) / stock_price.previous_price
end

percentage_change

Conclusion

It’s important to note that while in-memory calculation can improve performance, it also has some trade-offs. For example, the amount of memory available on the computer is limited, so if you store too much data in memory, the application may start to run out of memory and become unstable.

So we should know that the data in the cache will eventually become outdated, and will need to be updated. You can set an expiration time for the cache data to automatically expire after a certain amount of time. Alternatively, you can write code to explicitly update the cache data as needed. Otherwise, it may lead to incorrect results.

The in-memory calculation is a powerful technique for improving performance in Rails. However, it should be used with caution to ensure that its implementation doesn’t affect the stability and correctness of the application.

Follow Simform Engineering to keep up with all the latest trends in the development ecosystem.