Hey Tom Johnell, Thanks for reaching out. I just took a look at the Sidekiq Debounce gem, and I must say I am impressed with the library. I may have to make a change to it myself.
With Regards to your scaling issues, I am afraid I don’t have any answers. We have yet to reach the point in our growth where this is a pain point, so we have been reluctant to address it. As you said, the issue is definitely the linear search over the set, which when large, will take a while.
If I do ever find an answer, I plan on releasing a new blog post about how to deal with it.