How to maximize the impact of your research

Research impact is the product of a good idea, supporting evidence, and its communication. As with any product, if one component is zero, so is the impact. Many researchers are most comfortable with their hands on the keyboard, so they spend a week or two finding a research idea, several months in experiments, and a few days writing a paper about it. Successful researchers balance their time and efforts.

The Idea

The beginning of knowledge is the discovery of something we do not understand. — Frank Herbert

The idea stage involves finding a problem, proposing a solution, refining, and vetting.

My favorite method to find a problem is to think about a big challenge, like making a self-driving car or a vision system with child-like intelligence. Try to formulate a solution using existing techniques. Chances are, you can’t because there are major unsolved subproblems, and you can’t even begin to solve those. So recursively dive deeper until you reach a sub-sub-subproblem that is not yet solved, but is approachable. You can do this for multiple big challenges and find the approachable subproblems in common.

I can’t describe how to solve a problem in the abstract, so on to the refinement step. My first refinement method is to write a mini-paper with a problem definition, motivation, and experimental method section. This helps ensure that the idea and solution can be formalized well enough to validate and communicate, and the process of writing helps clarify ideas and find overlooked potential problems. My second refinement method is “to become friends with the pixels” as Jitendra Malik says and do small prototype experiments to test understanding of the crux of the problem and potential of the solution.

Killing bad ideas as important as nourishing good ones. A good idea that you didn’t think of is little loss, but a bad idea can waste years of your life. Will solving this problem make a really big difference to somebody? If not, there’s a good chance nobody will want to invest time to read the paper. Do you have a quick way to validate whether your solution is likely to work? If not, tread carefully. Is the idea obvious enough that many others will think of it (e.g. applying the latest advance in deep networks to an existing problem)? If so, let them. I like to air my ideas by describing them to others, to individuals and even in workshop settings. Sometimes others like the idea and sometimes they don’t, but more importantly, I learn whether I really believe in the idea.

The Evidence

Usually, evidence comes through experiments. The goal of experiments is not to demonstrate superior performance, but to validate and demonstrate that the proposed solution solves the target problem for the hypothesized reasons in the hypothesized cases.

In engineering, experiments typically require an idea of the form: “If I do ______ instead of ________, the result will be improved in ______ cases, as measured by ______. For example, if I include attention maps (instead of not) in my recognition architecture, object detection will improve for occluded objects, as measured by average AP on MS COCO dataset.

Good experiments clearly show

1. Existence and size of expected effect (e.g. improved results)
2. If the solution has multiple parts, the impact of each part of the solution
3. Characterization of when the solution applies or helps
4. Limitations of the solution

Writing

The single most important thing is to give writing time. I am writing the paper the entire time I am working on a project, maybe not on paper, but at least in my head. It’s almost always better to spend an extra week writing than to squeeze in an extra ablation or tune parameters a little better. Some tips:

1. Write to teach, not to convince.
2. In writing, follow the advice of Coco Chanel: “Before you leave the house, look in the mirror and take one thing off.” If you can remove a word without changing the meaning, do it. People can’t store many words in their heads.
3. Writing is a process. When you finish writing a paper, you have a rough draft. Read it and scrap and rewrite most of it. Then, you’ll have a good draft. Somehow, as a graduate student, most of my papers ended up being version 14.
4. The paper should read well if someone reads it from start to finish. It should also read well backward, or inside out. Some read the intro first, some the conclusion, and some browse the figures.
5. In later drafts, think defensively. Will your writing satisfy the reader looking for big ideas, for practical impact, for technical detail? Will it engage the novice and the expert?

If English is not your native language, you are at an unfair disadvantage, but you need to invest the time to master effective communication in English to succeed as an independent researcher. To become a better writer, learn to read critically, not only of the ideas but also the exposition, and then read and write often, and read what you write.

The Cycle

Idea discovery and refinement, experimentation, and writing work best in parallel and in a cycle. After coming up with the initial idea, quick experiments can help validate or refine solutions and problem formulations. A brief write-up helps vet and clarify the idea. Often, after doing a full round of experiments on my first solution, I realize that I need to go back to the drawing board and overhaul the approach. Likewise, writing a complete paper draft often reveals missing experiments and inspires better solutions. For these reasons, be open to revising the key ideas, and write your first draft a month before the paper is due.