Today I became addicted to Bash.
Hey, it’s better than coffee.
I was raised on Windows as a young adult. Everything seemed so natural and friendly: inviting icons, exciting colors, and a GUI that made things like moving files so intuitive — you just click and drag. In truth, for most of my life, the only way I knew how to operate a computer was through a GUI. None of this minimalist command line interface nonsense. That was meant for . . . well, someone. But not me.
As you can imagine, my outlook on user interfaces has changed a little now. I still use a Windows laptop, and likely will until the Singularity hits. But when I’m doing research, I use a Linux interface, usually through a VNC viewer or SSH client. And having used Linux every day for seven weeks straight this summer, I’m rather beginning to like it.
The reason I’m profaning my blog about astronomy with another dull discussion about computing is that today, I might have begun to fully appreciate the beauty of Bash. Bash, for those who don’t know, is a command language commonly used in Linux distributions, and it can do a lot. As you might have guessed, I wrote my first non-trivial Bash script this afternoon. It’s two dozen lines, takes about four hours to run, and edits, analyzes or parses about 100 files. It adds a bit more automation to the last step in the data analysis pipeline I use, by performing that analysis on four separate datasets and grabbing the important information I need. The process would normally require me to do a lot of file editing by hand; messing it up would lose hours of valuable computing time.
To someone who’s been using Windows for a solid decade, Bash comes off as fairly impressive.
I did essentially the same thing last July. In the lab I was working in at Swarthmore, we were doing x-ray spectroscopy using software called Xspec. Xspec fits x-ray spectra to observational data — in our case, a dataset from Chandra. Our models in Xspec used (let me check) 118 separate parameters. Each time we fit a spectrum — usually about ten times a day — we had to specify whether each parameter was forced to be the same as another, whether it would be fit by the algorithm, what value it would take if it was constant, etc.
As you might imagine, this rapidly became a pain in the rear. Also perhaps a pain in whatever areas are affected by carpal tunnel syndrome.
So one night in late June, I decided to just up and automate the whole thing. I would write a script that set up all the parameters just as I wanted and start the analysis. To do so, I picked Tcl (yes, “tickle”), a language from the late 80s which lives on today as part of the Tcl/Tk combination popularly used by Python.
I’m pretty certain I’m the only person in my department to have used Tcl as a scripting language in its own right. There’s just not really a reason for any of us to learn it when “fun” high-level programming languages with scripting capabilities (like Python) are becoming more and more popular. By comparison, Tcl can seem archaic, limited, and not of much use outside Tcl/Tk.
But what tickled me about Tcl (see what I did there?) is that it works very well with Xspec — better, it seems, than Bash or any other scripting language. There are some specialized commands that make it super easy to grab information from the Xspec interface. If I was going to automate my 118-parameter-deep analysis, why not use Tcl? So I did. And I have never had cause to use Tcl since. But for three months, it was very useful.
Given how well Tcl worked for me last summer, today, I figured that Bash would do a darn good job for my data analysis — and so far, it has. I won’t find out if the script’s perfect for another (checks watch) three hours, but everything seems good for now. I tested it modularly before running it — I’m mortally terrified of running any script that seriously modifies files behind the scenes — and I think it’ll work. It’s done fine on the first quarter of the data.
The point is — if there is a point to any of this — Bash is kinda awesome. Again, maybe I come across as a bit . . . overexcited? But it really gives me a lot more free time when I’m running code in the evenings, by reducing the manual labor (ha) required.
Free time to, say, automate the automation?
You can read my introduction to my research this summer here. Yesterday, I wrote about why batch processing is also super awesome, even if it makes me want to fight my computer. Tomorrow’s blog post topic? The extragalactic background light, photons from stars across the universe — and a fascinating topic for gamma ray astronomers.
