… something on submitting and reviewing for conferences

I have now reviewed sessions for at least half a dozen conferences over 3 or 4 years and for at least two different organisations.

These are my observations from that time, and I offer them as a single perspective; feel free to disagree. There is an element of shedding some light on what goes on during the reviewing process — which can be opaque to the first-time conference submitter.

How is reviewing done?

First of all, reviews are done ‘blind’ or anonymously for the conferences I have dealt with. The reviewers do not know and are not supposed to know who has submitted each talk. And I suspect if they did try to guess would get it wrong.

Reviews are also done in confidence and independently; I often don’t know who else is on the panel, and if I do happen to speak with someone we don’t discuss the specific proposals themselves.

This is because sessions should be judged on the content of what is proposed, not by who is submitting it. This should ensure someone doesn’t just get in on their reputation along, but has to come up with interesting ideas for talks.

So don’t drop your names in to the any of the session descriptions; this is breaking the anonymity; some review systems can actually filter this out, but not all.

Secondly, feel free to quote them but don’t name-drop industry names into the proposal — “I was pair programming with…”, it just sounds like you’re trying to get in by association.

What happens after the reviews

Once the review process has happened, the scores are used by a Programme Panel to put together a coherent, balanced schedule.

This won’t be anonymous — they need to make sure one person isn’t given multiple talks at the same time, or indeed more than one or two sessions at all.

They need to make practical decisions like ensuring two very similar talks aren’t scheduled, and that the rooms available will suit the session format and number of possible attendees.

Whilst I have a bit of an aspiration to be involved one day, I’ve not been involved in this yet; it sounds a bit of a nightmare really, but an interesting one, and they do it very well.

The Key-notes are selected by a different process, and mainly invited directly by the organisers. I guess that doesn’t preclude them coming from the review process, but that’s not its purpose.

Keep it self contained

Some conferences can get hundreds of session proposals. A panel of reviewers are used for a few reasons:

• So no one reviewer has to read hundreds of proposals
• To reduce individual biases and gather a range of opinions on each submission
• To include different areas of expertise in the review process
• To balance out strong opinions and pet topics of the reviewers
• To sanity check the suggestions made by submitters

Realistically reviewers are asked to go through between ten and thirty (the enthusiastic may manage more), this also allows each review to ‘calibrate’ their reviews — I tend to do a few and then go back to the first one to even out scores. Fewer than this and the reviewer won’t get a big enough sample size to give them a feel for or overview of the submissions.

Either way, there is only a finite amount of time for each submission, and also remembering that they should be anonymous, inserting a link to a video where you’ve giving the talk before both breaks anonymity and puts an expectation on the reviewer of spending 10, 20, 30, 45, 60 mins watching a video to be able to judge the submission — ask yourself what if everyone did that, and what disadvantage a new speaker would have as they wouldn’t have a video.

Level Playing Field

Submission forms are designed to give a level playing-field to everyone, and by any word/character limits (alongside the details in the Call for Proposal) give guidance on how much to submit. This, again, is so that they are fair.

I’ve also seen people just post a link to the details of a workshop they’ve given previously, but this still breaks anonymity and is pretty lazy; even just copying and pasting those details in to the submission form would not rely on the reviewer going hunting.

To be blunt, I don’t follow links to videos or previous-workshops, it should all be in the form.

I have in the past, as a submitter provided some citation references to back up what I’m saying in my submission, but I’ll come back to that later.

Level of detail

The thing that most repeatedly astonishes me when reviewing is the lack of detail in a substantial proportion of the submissions.

Reviewers tend to prefer more detailed conference submissions

Mainly so they have confidence in what they speaker(s) is/are planning to talk about and how they will meet the objectives of the session — Mark Dalgarno, organiser of many conferences.

There’s a few reasons I want to see a decent amount of detail:

• To show that you’ve thought about the issues you’ll talk about
• To show that you’ve thought about the organisation of the session
• To show you have something interesting to say
• To show that you have enough material to fill the time asked for

This is an important point — if you’re asking for a 90minute workshop slot, I want to see much more detail than if you’re asking for a 45 min one, and even more if you’re expecting a half or full day of the conference. Remember even a 90 minute slot is a significant proportion of the conference, and has the Opportunity Cost of 2 slots of half the length — are you going to be more impactful than them?

Don’t just say “we will share some of the things that we learnt”; spell them out to the reviewer so that they see they are logical arguments.

I always include a short breakdown of the timings involved, eg for my “Should we Just Rewrite This?” talk:

5 mins — Introduction
5 mins — Interactive section on ‘Why do a rewrite’
10 mins — Rewriting: the risks involved
5 mins — When a rewrite might be the best option
10 mins — What are the alternatives?
5 mins — Tie up & Conclusions
5 mins — Questions

Although it may not bear any relation to what actually happens at the end of the day/talk, it gives an indication of the emphasis of the talk, how much interaction you intend but mostly it shows I’ve thought about it!

Plans Are Worthless, But Planning Is Everything — Eisenhower*

Short and Long Synopsis

The conferences I’ve been involved in usually ask for a title, a short bit of text and a longer one.

The short one is often 70 or 80 words that will appear as a summary in the conference programme. The longer one is often for the reviewers only, and can include additional notes that you wouldn’t want in to go public (do check this though).

The short one will be used by a conference attendee to decide which of 4 or 5 sessions they want to go to, and often on the spur of the moment, so it should be snappy and tell them what they will get out of the session.

The longer needs to sell the content of the session to the reviewers, and you could think of it as a short blogpost developing and describing your arguments; a breakdown of timings as I’ve described above; what you intend to explore in the session and what an attendee will learn or takeaway.

You’re competing with lots of other proposed sessions, so it’s in you interest to sell to the reviewers you can fill 45 mins or whatever of interesting material.

All too regularly, I’ve had to use the following comment on sessions:

“There’s not enough detail here to give this anything but a neutral score”

“This looks interesting, and with more detail I may have given it a 5 out of 5, but can only give a 4 on what’s here.”

I’ve seen a single line try to justify a 150/300 minute slot. That’s a half to full day of a conference. Don’t be that person.

Don’t get me wrong, doing a PhD, there seemed to be an attitude that you judged a thesis by measuring how thick it was; I kept mine to 120–130 pages (though perhaps my external examiner disagreed, as he did with everything else ;) )

I’ve rarely slated a submission because I didn’t like the content, (though I do have a dislike of proposals to put the Architect back in to Architecture, or doing Big Things up-front), I’m reasonably keen to hear the dissenting voice, it’s been more because they’ve give a woeful amount of detail for the amount of time they’ve asked for.

Don’t be too defensive, nor boastful

I saw one proposal that specifically tried to address points they’d had as feedback from previous reviews. They both seemed utterly irrelevant to the conference in question; one was about what industry sector they worked in, this may have been important to that particular conference but was irrelevant to the one being reviewed.

More generally, work any feedback in to the proposal as a whole and address it that way.

Conversely, I found one proposal bragging about the feedback scores they’d received from a previous conference a little aggravating (perhaps I shouldn’t) and a little disrespectful to an independent, anonymous review process.

Conclusion

Whilst I’ve given a few strong opinions here, and they are only one perspective, but any conference doing an open CfP (Call for Proposal) and worth its salt will want a fair process; it might well be you disqualify yourself if you break anonymity.

Remember you’re trying to persuade a group of people you know what you’re talking about and are able to speak (or facilitate) for however long on an interesting topic.

If you are at the point that you want to share your ideas in semi-public, you can’t go in to the process trying to protect them by not giving details; give some examples of what you learnt or don’t bother submitting.

Finally, the conference’s I’m involved in are really keen to get a blend of old-hands and new faces and ideas; it took me a few attempts to get something accepted, but I made sure I learnt from the process and asked for feedback.

I hope you’re not put off by this post — it can be a bit disheartening being rejected, but don’t make it because you only spent 10 minutes writing a few lines; get your ideas down and sell your session idea to the reviewer.