Making a career as a research scientist can be the most fulfilling and life-affirming experience. Yet I have seen many students tempted by the prospect, only to retreat in short order to the relative comfort of engineering. They often interpret the pullback as a personal failure and a sign they’re not good enough. It’s never a matter of personal worth or talent, however. You need a different kind of temperament to thrive in a research setting, one that is often paradoxically orthogonal to what makes an engineer thrive.
Here are some of the dominant tensions I have seen researchers face at some point in their careers:
1. Research is about ill-posed questions with multiple (or no) answers
Your university training has largely taught you how to solve well-posed problems with unique answers. But treating research as an exam problem is a sure way to fail. Much of what you do in research does not get you closer to the answer, but rather enables you to understand the question better.
Measuring progress in units of learning, as opposed to units of solving, is one of the key paradigm shifts one has to undergo to be effective in a research setting.
2. You will spend your entire career working on things that don’t work
Almost by definition, once something works it is no longer research. There is a deep level of angst stemming from the realization that, in the best of scenarios, most of your career will be defined by progress that falls short of actually solving anything. These are steps in the larger journey from concept to working technology.
Failing to understand and embrace this simple reality almost led me to give up on my research career. It was 2004, and my area of research—speech recognition—was in a strange state: It arguably didn’t work, but was being shoved down users’ throats for cost-reduction purposes. I felt the heavy, resentful gaze of anyone who had ever used an automated 800 number. I also noticed a disturbing trend at conferences: More and more papers were being published about emotion recognition, which was a nice way of framing the problem of figuring out exactly when a customer was annoyed enough that it was time to send the call to an operator. I got out of the field for a few years, which was tremendously helpful in gaining more perspective on being resilient as a researcher.
3. Your work will probably be obsolete the minute you publish it
The treadmill of progress is relentless. Nothing I have done in my career is state-of-the-art today. Very little was still state-of-the-art by the time it had gone through the lengthy publication process. We measure impact in terms of the number of citations, often ignoring that many of these mentions use one’s work as a baseline to show how it is no longer competitive.
FOMO—in this case, the fear of someone else beating you to publication—is a huge stressor for many of my colleagues. My constant advice to those afflicted is that if you’re worried about being scooped, you’re probably working on the wrong problem in the first place.
If it is a problem you expect to see solved soon by the community, it is probably not a line of research worth your time in the first place.
4. With infinite freedom comes infinite responsibility
Good news: You’re in control. Bad news: You are in control. There is no spec, no blueprint. You may be exploring a completely wrong path, and that’s okay. Because you’re okay with it, right?
As a research manager, much of my role is to act as a therapist for researchers contemplating the abyss of possibilities. I often set boundaries to the research agenda—not because it matters where the boundaries lie, but because their mere existence helps lower the stress of the unknown.
More often than not, merely reinforcing the path already taken and giving people permission to say no is sufficient. Decision fatigue is a real thing. Charles Sutton has a nice post about the stress and necessity of liberally saying no as a researcher (read the whole series here).
5. Paradoxically, much of research is about risk management
Research is inherently risky. Pile risk onto risk and you’ll get certain disaster. This is why you have to be ruthless about eliminating every other risk from the equation. First and foremost, make sure you trust — and have earned the trust of — your collaborators before engaging in joint research. Most failures aren’t technical; they’re human. Avoid introducing political and institutional risk. Make sure your funding is secure. However, never compromise on the research risk itself (such as, for example, lowering the ambition just to make the project more palatable to your institution) — that’s how tepid research is born.
Wanting to work on risky research is the greatest lie researchers tell themselves: We are all a lot more risk-averse than we think, and every bit of safety you can add to a project directly goes to your research-risk credit line.
6. You will need to retool often
At the time scale of a career, paradigm shifts occur regularly. It is likely that someone with a better mousetrap will sweep away the expertise you’ve painstakingly acquired over a decade. Your ability (or, more important, your willingness) to follow the problem wherever it takes you—even if your true love is the tool you happen to be currently wielding to solve it—can make or break your career. My own Ph.D. thesis is using a toolkit that probably no one should care about today. Good riddance.
Also, it’s often the case that big breakthroughs come from bringing together two presumably disparate lines of research. This often means being willing to learn and absorb the perspective and tooling of a completely new field before you can begin to evaluate the possible connections.
7. You’ll have to subject yourself to intense scrutiny
There is nothing more suspicious than a single-author paper. I would certainly never trust my own results if they hadn’t passed the scrutiny of a peer invested in the outcome. The social dynamics of research collaboration are actually part of what makes the scientific product valuable, because the very act of conducting research is prone to tunnel vision and self-reinforcing feedback. In another story, I explore a defining moment of my career that a healthy dose of peer skepticism likely would have prevented.
Willingness to be vulnerable is a defining characteristic of the best researchers.
8. Your entire career will largely be measured by one number
And that number will be public. Yes, h-index fetishism is a thing, and one you can’t easily opt out of, because not making it public naturally raises questions. It’s easy to forget that h-index is a relatively new phenomenon, given that the metric was only conceived in 2005.
For all its limitations, I find it remarkably robust (read: hard to game) and well correlated to my own personal evaluations of researchers whose work I know well. It also has a lot lower variance than information you get from an academic’s web profile, some of which are terrific works of propaganda art. One day, our enlightened selves may all be judged based on our Bacon-Erdös-Sabbath number. Until then…
9. You won’t work a day in your life
I’m sometimes asked what it’s like to work alongside so-called “geniuses.” People are curious about what makes them different, and where the essence of their success lies. The only thing these people have in common is that they work their asses off. That’s the truth, though it’s not the whole story. Every single one absolutely works harder than you (and certainly me), but another common feature is sharp focus and dedication. Few would call what they do “work.” At their roots, geniuses absolutely love what they do, and they are willing to devote their whole selves to it. Everything else derives from that.
In my experience as a manager of both researchers and engineers, success as a researcher stems more from how one manages to navigate the pressures of conducting research than from inherent smarts or hard work.
Working on things that don’t work, without a compass, in a forever changing environment, under the public scrutiny of your peers, takes a certain courage — or folly.
At the same time, sitting in the seat right next to them, engineers are actually building things that will endure. They’re solving well-defined problems and exercising the same level of creativity and mastery over their subject matter. Building things that have to work — and are expected to work — takes another kind of bravery and dedication to getting to the finish line. It also takes a healthy dose of self-criticism—which is equally difficult to subject oneself to, especially when it can’t be waved away with a “never mind, it’s just research…”
Whether it’s research or engineering, defining the challenge that best suits your personality can take a long time (years, in my case). It can change depending on where you are in your career and personal life. For most of us in industrial research, the great news is that we don’t always have to make this a permanent career choice.