Balancing Work And Learning
Scheduling self study around work, and sometimes into it
At the end of my previous post, I said that my next articles would deal with more coding and data science topics as I work to build up a professional portfolio. Well, those aren’t ready yet, but I had an interesting conversation at work which inspired me to write this article.
One important takeaway from data science podcasts is that there really isn’t a traditional path into data science. Perhaps the state of the profession is changing with more data science undergraduate and graduate programs coming into existence, but that isn’t the impression I get listening to professionals in the industry.
If your situation is similar to mine, using a mixture of self-study, formalized courses/certificates, and previous knowledge; you’re intimately familiar with what I want to talk about today. How do you integrate learning data science specific skills into your schedule with a full-time job and other commitments?
Nights and Weekends
The most obvious answer to this question, of course, is to work in time for study at night and on the weekends. That is what my main approach has been, particularly with regards to the programming certificate I am pursuing.
Luckily, my wife is incredibly supportive, and happy to find a book to read or a video game to play (she really got into Elden Ring) while I’m reading a data science text, working through exercises/assignments, or fiddling with my own projects.
More specifically, here is what I’ve found works:
- Simply reading the material isn’t enough. I’ll often read through a book or chapter once to get a framework in my head, then reread it, and go through the exercises or figure out practical ways to try and apply the material. I prefer actual exercises/questions to simply redoing examples worked out in the text, but the main point is, the skills/information won’t stick if you don’t use it.
- Cramming/coding late into the night isn’t productive (for me). At least, it isn’t as productive as it might have been 10–15 years ago (I have vague recollecionts of a few physics GRE study sessions taking place at 2 am, at a Denny’s in Couer d’Alene). If I don’t set a reasonable limit on my time, my sleep schedule gets thrown off, I risk being useless the next day (both at my job and for more studying), and one might argue what I did produce after 9 pm or so could be improved.
- Try to set aside entire weekend days, if possible. For me, this alleviates the issue of not being able to keep working on self-study material until midnight. I often find myself working ahead in the certificate courses, to the point of being done with all the assignments about half way through. Marking off an entire day can be challenging, but with a good support structure it is very effective.
That last bullet point can be difficult when trying to work around social engagements, family events, and recreational activities. All of these things are important parts of the process. Taking breaks from intense study allows one’s brain to process information, and surprising insights can pop up when you take a break from a problem and do something else.
Tack on things like mowing the lawn, landscaping, house projects, and the like; and you’ve got quite a busy schedule in which to find room for studying and personal projects.
As it happens, a significant part of my self study ramped up just before the pandemic, so I had plenty of reasons to stay home. That also meant I racked up a lot of leave time, and I have occasionally used that excess leave to focus on assignments and study if it was a quiet day at work. That is definitely a leisure not everyone has.
That being said, spending time with my wife is a priority for me, and our malamute is really needy (see photo above…so needy) so I don’t want to use all of my weekends and leave to improve my data science chops.
What’s one good solution to this? Finding ways to incorporate what I have learned and am learning into my job.
When Work and Study Collide
I think it is reasonable to say that most scientific research these days will involve computers in one capacity or another. In physics and astronomy, the amount data coming in from experiments and observatories just keeps growing and the techniques used to analyze them can be (but don’t have to be) quite sophisticated. So, anything which enhances my coding and statistics skill sets is good for work.
That was how I started out, improving fundamental areas which would be helpful in my current job and in the career I aim to transition into. The tricky part came when delving into aspects of data science where prediction, rather than explanation, was the goal or when I was playing around with an algorithm where a direct application to my research wasn’t obvious (at least, not at first).
How do I responsibly merge work and self study:
- Be honest about the ‘use case’. I have to be honest with myself and my ‘bosses’, I can’t just read about some new algorithm and then try to apply it to a work problem just because I want to try it out and get experience with it. When I first learn about an algorithm studying on my personal time, I make an effort to understand how it works, the types of problems it can be applied to, and the types of problems it doesn’t work well for. I don’t want to waste time on the job playing around with something fancy, even though it doesn’t fit the problem, and especially if I know it doesn’t fit. This attitude is good for data science as well, don’t rush to throw a super advanced algorithm and all your GPUs at a problem which might be solved with a simple logistic regression.
- Focus on fundamentals. I’m reiterating this here, because it is important. I was already using (or could easily use) many of the standard data science python packages (e.g., numpy, pandas, matplotlib, etc.) and improving those skills positively impacted my research. In fact, becoming more efficient with numpy led me to realize that I didn’t have to use a binned likelihood approach for a particular problem (my previous attempt had been sloppy and inefficient as hell), that realization led me to look at the data in a different way and see that we had enough data to try some machine learning.
- Frame things as investigating an alternate approach. Machine learning approaches are popping up in physics and astronomy journals, being used for really cool stuff. However, I get the impression that there is still some resistance to these methods in the community. I’ve found this is because there are some researchers who, when they don’t understand an algorithm and/or are very set in their ways, dig in hard. Let’s be honest though, sometimes the researchers presenting the machine learning approach don’t do a good job explaining/demystifying the algorithm. So, I don’t frame things as “Everything we’ve done is bad and this will be a gazillion times better.” Instead, I try to present reasoning along the lines of “I’m seeing a few issues with our previous approach, possibly the parameterization I’ve chosen isn’t ideal, so let’s test a few of machine learning algorithms which could be applicable.” I’ve also been proactive about heading off resistance by organizing a group meeting specifically focused on starting a discussion about machine learning techniques. This included a careful walkthrough on the basics of artificial neural network and showing preliminary results from applying several machine learning algorithms (neural net, random forest, principal component analysis) to a current research problem.
The approach of seeking out reasonable, justifiable ways of integrating the data science skills into my current research has served me well. While I can’t go into a lot of detail or show results from unreleased projects, here are a few general examples.
At the start of a new project several years ago, we were getting data from individual flights detectors had been flown on. We were trying to figure out the best way to store all the ‘meta’ data (flight name, detector, departure airport, etc.) and one colleague mentioned “some sort of relational database”. At the time, I had no idea what that was, so I started with a python dictionary in a pickle file.
Updating the pickled dictionary wasn’t super secure (my code always saved a backup version, but it still made me nervous) and management ultimately felt ‘klunky’. Enter SQL. I read that this was a really good language for data scientists to be familiar with, so I purchased a text and started reading on my own time. I soon saw how this could be implemented for our project, and at that point, learning became work. I tinkered with sqlite3, honed the syntax to create a database (with multiple tables), transferred everything from the dictionary to the database, and wrote code to automatically add new flights to the database as well as other custom python modules to interact with it.
Brushing up on statistics brought linear regression back to my attention, which leads us to another example of using self study to augment my research. I was trying to characterize some properties of the data and see how they depended on the flight path, with an ultimate goal of simulating/predicting this property. In looking for trends, I realized that, for some parameters, a linear regression worked well. The R² values wouldn’t be stellar, but the residuals were approximately uniformly distributed, so starting from this one parameter of the flight path, I wrote code to use a series of regression equations, adding in the Gaussian ‘noise’, to realistically simulate this aspect of the data using the flight path.
As a final example, we’ll get back to the conversation which spakred the idea for this post. For this same project, we’ve worked up a set of python modules, including a GUI application to process and analyze the data and query our database. With the nominal project end in sight, I’ve been fine tuning this package a lot, lately.
One thing which keeps coming up in articles and podcasts, however, is making a dashboard, something I don’t know how to do (yet). I have been taking notes on modules to look into, but this task hasn’t moved up to the top of the list.
Then, earlier this week, I received a call from a colleague to talk about the software package I’ve been fine tuning. My colleague says “I’ve been thinking, maybe the thing to do is some sort of dashboard”. Just like that, I have an opportunity to move something from my self study list to my work list (never mind the time I have recently put in to rework the code to be object oriented and much easier to read) and an idea for a new post.
Of course, things are rarely that simple. We have a lot of particulars to work out before I can begin diving into dashboard devlopment in earnest (Exactly what will it do? Does the sponsor, who will have to host the dashboard, want to do this?, etc.), but it’s a start.
A Endnote
Everyone’s path to data science will be different, and I know some people will face, or are facing, more significant challenges than what I’ve described. I have a good secure job, one in which I can justify cross over with my learning goals, and I know that isn’t the case for everyone. Each day seems to bring more news about layoffs in tech, and I wish nothing but good results in the job search for those affected.
This article/blog post simply tells my story and how I am approaching the situation. Your story will be different, maybe some of what I’ve written will resonate with you, maybe it won’t.
I do sometimes wonder if I should take Chandler’s advice from an early episode of Friends, quit my job and get The Fear to really motivate the job search. Truth be told, I’m not ready for that yet (and I’m not a big fan of fear), but I haven’t ruled it out.
Feel free to share your tips for optimizing self study, scheduling, and finding a work-life-learning balance by connecting with me on LinkedIn or commenting here.
Looking Ahead
Well, I’ve learned the danger of promising too much, but I think I can give people a reasonable idea of what I have planned for future posts. I’m close to turning a little bit of code I wrote for myself and my family into my first github project. It’s a bit more software engineering than what I ultimately want to do, but it showcases some data science skills and one thing which is very important to me, family. I will share that with everyone soon and comment on how I see it helping to prepare me for a transition to data science.
