Oh no, I found a mistake in my old code — What now?
Research is storytelling with data as characters. You start with a hunch, and then you let the data have a dialogue to prove or disprove it.
Like many applied researchers and data scientists, I spend my days chatting with observational data, the kind that’s already been out in the world, collected by some authority or business and just waiting for someone like me to come along and tell its story.
The beauty of applied research is that it takes messy, complex data and turns it into something meaningful and, hopefully, useful.
Getting the data, however, can be a real mix of experiences. Sometimes it’s a download away, other times it’s a scraping marathon. And then there are those times when I’m literally or virtually knocking on the doors of the keepers of (historical) records.
Eventually, all the effort pays off — one way or the other. It’s extremely satisfying when the data allows us to uncover hidden patterns or reveal some big-picture truths that otherwise we couldn’t see.
Once I’ve got the data in hand or on my computer, it’s like piecing together a puzzle. This is where I roll up my sleeves, clear my desk, and start doing some data cleaning they call it. It’s sort of like prepping for a big dinner; some ingredients are just wash-and-toss, and others need a bit more care and preparation.
Then comes the big dance: the analysis. This is where you see if the data’s got rhythm or if it’s stepping on toes, creating patterns that aren’t really there because, well, maybe a spot in the cleaning was missed. (See for example this article I wrote on measurement error.)
Working with data makes me feel like a detective. A data detective, scrutinizing each and every step, and cross-examining the patterns.
So, there I was, thinking my data was waltzing along just fine until I tripped over an old line of code. It was like finding a wrong note in a song you’ve been listening to for years. It threw everything off.
A merge that was supposed to be the bridge in the second verse found its way into the opening chorus, disrupting the melody I thought I had carefully composed.
The internal alarm bells started to ring loudly.
Step one: I stared at that line. “Why are you here?” I asked it.
Step two: I tried to fix it. I then realized the tune didn’t sound sweet anymore.
Step three: I entered the classic denial tango. “I must have had a reason for this line, right?”
After hours of interrogation, I needed to breathe fresh air, so I took a walk and ended up at the Opera house, where I was met with another surprise. My phone buzzed — it was my cousin Miki, who I had thought was in Albania, writing to say she was visiting Vienna. A pleasant surprise, but unfortunately, my head was still with my data, at home.
Over coffee, I scribbled on the pen and paper I had brought with me just in case an idea hit, but the answer really was playing hard to get.
Day two. I had slept. I was looking at the data with fresh eyes, but that merge was sounding more and more like an unwelcome guest. Did I show it the door, start over, or try to make it part of the act?
The answer didn’t come easy. I mulled it over and over again, added more plots to visualize the error — and tried to see the story from a different angle.
On day three, my body was aching despite having exercised. (Every day actually.)
I decided to take another walk and sit at the Schwartzenberg café. A piece of cake for comfort and to celebrate the mess-ups. Mistakes, after all, are our best teachers.
In this case, I got another reason to celebrate — a fresh angle.
Turns out, that merge was a twist in the plot but not the end of the story. It’s a puzzle, sure, but not one that can’t be solved.
I’m still on edge about other surprises that might be hiding in the code, though.
Science really is like an orchestra where collaboration is key — different instruments come together to create something greater than the sum of its parts. Yet, sometimes, we find ourselves in solo performances, improvising. I really can’t wait to throw my code out there and have other brains give my code an extra pair of (fresh) eyes, poke holes in it, and help me tune it up. Peer review and discussions are what make science truly sing ;-)
