Code without leaking your memory!
First thing come to your mind how to process large data without consuming much memory?! It is easy be a lazy guy.
In Ruby 2.0 they implemented the Enumerator::Lazy
So basically when you read a huge amount of records or big files instead of reading all of the data we only read what we need at a time. it is slower than normal .each or any normal looping but it will reduce the consumption of your application memory so if you want to save memory use lazy.
Example about using lazy:
File.open(“txt”).lazy.each do |line|
puts line
end
File.open(“txt”).each do |line|
puts line
end
The first block using lazy will consume 9400 kb of memory while second block will consume 9676 kb. Total lines in “txt” is 300000.
So we saw how lazy can help you to read large data without consuming big memory. Now we will read records from database efficiently without consuming your application memory.
Did you heard about database cursors ?!!
A database cursor is simply a control structure that enables traversal over the records in a database. Think about it like this, you fetch all users in ActiveRecord:
User.all it will be executed and load all records in your application memory the more records the more memory usage. So a database cursor will simply execute the query and generate a result set (table) in database server and it will return the cursor address related to the result each time you read from the cursor it will return the next record. Using cursor will ensure your application memory is safe.
Resources may help you:
