On being rejected (by conferences)…

11 years ago I was involved in academic research. In my last two years in college I was working in computer graphics, preparing my master thesis. The sort of work I did was not the most common one. Usually when people think about computer graphics they imagine videogames or CGI, but I was involved in something else: non-photorealistic rendering. To better understand what I’m talking about, let me just show you some of the stuff I was rendering in real time.

This was a niche area of CG, and very quickly my boss started to motivate me to submit papers to some heavy-duty conferences in this domain. There was one that was very reputable and he recommended, but I wasn’t very convinced at first: Pacific Graphics.

Then, I saw where it would take place… Hawaii! That got me interested! If I was accepted, I would have the opportunity to present my work in Honolulu, on the other side of the world.

The way these conferences work in academia is that you don’t just submit an abstract and wait to get accepted to present your work. You have to submit a full paper, describing everything you did and what you accomplished. The paper will be included in the conference’s published notes and your presentation will only reflect what is written on paper. So there is a lot of work being put in each paper and competition is fierce. Nevertheless, I worked hard and submitted my proposal.

Then, I waited. The dreadful wait all conference submitters have been through. About a month went by and at last, there was a verdict about the paper.

It got rejected.

I went into the details, trying to find out why I had been rejected, and it was clear there had been a thorough “process” behind my rejection. Usually in this type of conferences, you get evaluated by 2 or 3 anonymous reviewers, maybe in some cases you get a fourth reviewer, and each one gives you feedback and states if it is a “go” or “no go”. I got 7 reviewers!

You could see by the dates of review how this “story” happened. First there were 2 reviewers but they weren’t sure about my work. They liked it, they found the paper well written and interesting, but there were doubts regarding it’s novelty. “Is his work introducing real advancements to this field? I’m not sure…”. So one of them gave me a “go” and the other one gave me a “no go”. I was stuck right between the line that determined my paper being accepted or rejected.

So the conference organizers called up a second pair of reviewers and guess what? The feedback was the same. One “go” and one “no go”. So they called a third pair of reviewers. Can you guess what happened then? Yep. One “go” and one “no go”. I had 6 reviewers, half of them accepting the paper and the other half rejecting it.

So a final seventh reviewer was called in to make the final decision, alone. And let me tell you about this review: it was brutal!

The “no go” was “unquestionable”. The reviewer said my ideas and decisions were horrendous, that my work was going into a direction that made no sense, and that, furthermore, the quality of my paper was so bad that “he even made mistakes writing the names of some of the authors he is referring to in the state-of-the-art”. I didn’t even know who he was talking about in this last comment. I just used Bibtex and the referenced authors’ data had been pulled from what was posted online. I looked into the dozens of references in the paper and could not figure out what was wrong to be honest.

This was a real let down. I started putting into question everything I had been working in and even had a discussion with my boss regarding the quality of our ideas. He, of course, tried to reassure me about our work and how being rejected in conferences was just part of our professional life: we had to deal with it and move on. But I wasn’t entirely convinced, I was really under the impression that what I achieved just… sucked!

A few months went by and, fortunately, I got accepted for a very different conference, with a paper I had written together with a colleague of mine, Vera. It was an arts-meets-technology themed conference and our sort of work, getting computers to render 3D objects as pencil drawings, was a great fit for it. This one happened in Banff, Canada — an absolutely stunning setting since it was an arts institute right in the middle of the national reserve of the canadian Rocky Mountains. I believe the following picture will work better than any description.

We prepared our joint talk, we rehearsed it and there we went. After nervously waiting for our turn to speak on the first day, we presented our work! At the end, we had a lot of questions coming from the audience and most of them were regarding the technical stuff behind our achievements. When we were done, we were really happy and allowed ourselves to take a triumphant deep breath. So I went outside.

I distinctly remember being out there leaning next to a wooden rail, surrounded by pine trees, closing my eyes as I enjoyed the sunshine and the fresh air. Our talk was done and we had delivered it successfully.

But then, this person came up to me.

“Hi” he said, one of the people in the audience that had asked some questions, “I just saw your talk and would really like to say that your approach is wrong because bla bla bla, bla bla bla, bla bla…”

I stood there listening to him, completely taken off guard. When he was finished, I just said:

“Errr… ok, thanks for the feedback. I’m João by the way. Nice to errr…. Meet you?”

“Oh yes, I’m Hans. Hans Priesner!”

I’m making this name up, I don’t actually remember his name, since this happened a long time ago and it is not relevant at this point.

Hans continued his reasoning on how my work sucked and I stayed passive, not really wanting to get into an argument with this person who “needed” to let me know of all of that. As soon as I had my chance, I got the hell out of there.

But then, as the day progressed, I started thinking there was something familiar about what Hans told me. I had heard that “bla bla bla” before. So as soon as I could, I rushed back into my room and fired up my laptop. I looked in my archive and opened the email with the reviews from Pacific Graphics. Suddenly, I found those familiar words. What Hans had just told me was exactly the same as what that seventh reviewer had said regarding my work in the paper that got rejected.

I then went online and tried to figure out who exactly Hans was, finding out that he too did research in non-photorealistic rendering of 3D objects — he rendered pencil drawings from polygon meshes while I used a mathematical model to achieve the same thing.

Finally, I re-opened the paper I had submitted to Pacific Graphics and went into the reference section. There it was: “Hans Prizner”. The Bibtex reference that I pulled from an online database had Hans’s name wrong. I’m portuguese, so it wasn’t obvious to me that the german name “Priesner” wasn’t spelled like that.

So I concluded what was obvious at this point: the seventh reviewer was in fact Hans. He did research in the same niche field as I did, he had a different approach towards the same goal and he saw his name wrongly spelled in my paper and took it personally.

There were a few things I learned with this small episode:

1. Don’t make a rejection absolute: Initially, I focused too much on that seventh review and took it as scientific fact: my work sucked. I disregarded the good things others had said about my work and only looked at the really bad review from just one person. A review in a lot of ways is always going to be an opinion, so you should see it for what it is. My work wasn’t stellar and widely accepted by group of peers in a highly competitive research area, but it didn’t “suck” to the point where it was rejected by everyone.
2. Understand that reviewers are human too… and humans have their own biases and context. Hans wasn’t perfect in any way. He was nearly offended to see someone else’s work that attempted to reach the same goals as his but through a different path. Trying out different stuff is what research is all about! Furthermore, he got even more offended by seeing his name wrongly spelled in a paper. Don’t get me wrong, but I have a “very portuguese” name and I’ve lost count of the times someone misspelled it in the global community. There’s no need to get offended by it. It’s an easy-to-fix and honest mistake.
3. Sometimes it’s just bad luck. I mean, think about it, what were the odds that the decision to accept my paper or not would come down to this person? One of the few in the world capable of seeing the differences between his own work and mine. A person who doesn’t like seeing things done differently and even gets offended at seeing his name wrongly spelled because of a stupid Bibtex reference I would have gladly corrected myself immediately.

If you think about all of these lessons, you’ll see that reviews hide a lot of factors we never think about. Maybe the reviewer had really strong opinions regarding what you’re saying. Maybe he was grumpy that day. Maybe you were unlucky and he had just accepted 2 other papers that were really similar to yours, so he can’t accept another! Or maybe, and this happens frequently, the reviewer actually has a point. You need to further improve what you’re doing and that’s OK.

Be humble, accept the feedback and try your best to learn from it. But always see it for what it is: some other person’s opinion that you may or may not agree with.

After this episode, I decided to tweak the Hawaii paper a bit from what I learned and submit it to another equally awesome conference. Can you guess what happened?

It was accepted.