Sometimes we accidentally write code that has memory leaks. Sometimes these memory leaks cause Out of Memory Exceptions. Generally, these are unrecoverable scenarios. Generally, whomever wrote the faulty code should chalk it up to being a lesson, learn from it, and move on. Sometimes, you intentionally write code that has memory leaks.
Last week I talked about how my work on the database replication engines RJMetrics Pipeline fell into this very scenario. Streaming client data can sometimes come in to the tune of 4GB per row. Any JVM will quickly run out of space trying to pull this off cleanly.
So, what do you do when you have to perform actions for side effects which will cause your entire system to crash? Ideally you’d re-architect things to be more fault tolerant. If you’re lazy and you’re writing Clojure code, you can use my little library to ease your woes!
AC (https://github.com/AlexanderMann/ac) is a program that keeps your root JVM running cool (shite puns are free). It clones the program you’re running, spins up another JVM, and runs the function you passed it with the args specified.
Examples are in the repo, but it’s pretty straight forward:
“foo” “bar” “baz”)
This is the first step for me on my way to building core.async-process, a library which will allow you to choose another JVM for running your threads on for Clojure core.async. Improvements I’m hoping to make to this code base are: some classpath/jar improvements, the ability to specify various JVM boot properties, and maybe make this a macro which doesn’t require a fully qualified path to the function you want to run. As it stands now, this is all icing to me so I’m not sure when I’ll get around to it.
Here’s to getting a little more sleep.