So I’ve been running a semi-automated process at work for the last few weeks to parse and analyze logs.
It’s basically an ETL process:
- Download the logs
- Extract a zip with one of the logs and do some cleanup so they can be used
- Parse the files into a normalized form
- Load the data into a database
- Generate reports
- Review the reports and email an analysis
- Do other work for the rest of the day
So the real work is in 6 and 7. Right now it’s almost lunch time (when I was originally writing this), I’m pretty tired though… I’ve actually been tied up with other “urgent” work and still haven’t had a chance to do #1.
Actually, in the last few weeks, I’ve gradually automated steps 3–5 though… but that doesn’t matter if I can’t do the first few steps. They have now become the pain point.
I have another tool that partially automates it, but I still need to execute a sequence of steps… which are becoming more and more tedious especially when I get distracted every minute…
Well today I finally got around to integrating steps 1 and 2. I’ve managed to cut down maybe 20 slow steps into just 2 simple ones:
- Enter the password to connect to the servers (this is more about security thing, don’t want to accidentally check-in my password into Git)
- Run a mtools command (it’s tricky to run directly so my app prints the command I need to run, this can also be removed with some research I guess which now I have the time for… hopefully)
So now I have more time and more energy!
I think this is the secret to reaching 10X: how to conserve your energy and use your time for the things that actually matter.
So yea I’m kind of feeling it now but I’m pretty sure I will really feel it tomorrow.
I guess it also goes back to why I became a developer:
- make programs to do my grunt work so I don’t have to (and do it better without mistakes)
- make my life (our lives) easier and time better spent
And also goes a bit into our future when “robots take over the world”
I have a feeling this will take a while but when they do… there’s a high chance we’ll have more time to recognize and focus on more valuable problems that need solving.