Threading in Rails
Task: Make an API which makes 10 API calls to other services. Combine their result and return.
Traditional Way: Use some HTTP client (https://github.com/nahi/httpclient for me). A for loop and combine them. It was taking about 20+ seconds for me.
Solution: Threading. Some googling and then I could come up with the code to perform the task.
Line 1: detailed_hashtag is an array of hastags. Eg: [“nature”, “world”, “insta”, ….]
For every one of this hashtag, I have to make an API call. One way is to run each of them in a separate thread but our array size can be big and having those many threads might not be ideal.
Line 2: So we have something called MAX_THREADS.
- I divide the array into batches with each batch size as the size of MAX_THREAD.
- For each of these batches, I will run all all requests in threads and wait till every request returns a response
- What if error comes?
threads.each do |x|
begin
x.join
rescue RuntimeError => y
puts "Failed:: #{y.message}"
end
endImprovements: With 8 threads, the time went down to 4 sec from 30 seconds.
Further More: I didn’t know about this awesome ruby gem to do all these tasks. Parallel — https://github.com/grosser/parallel . Comment below if someone used it.
You may contact the author at abhinav.rai.1996@gmail.com
