Selecting Speakers for Android Makers 2019
In order to be more transparent, the selection committee for Android Makers 2019 will explain the process behind how we built the agenda for our conference.
This post has been written by Marion Hayoun, Djavan Bertrand, Stan Kocken and Eyal Lezmy. We also want to give a special thanks to Mike Wolfson for reviewing this article!
The committee
The committee is the group that decided which talks will be on the agenda. It is composed by 6 developers:
- Djavan Bertrand, iOS developer at Tag Heuer
- Marion Hayoun, Android developer at Canal+
- Stan Kocken, Android lead developer at Dashlane
- Eyal Lezmy, Android developer at Tag Heuer and Google Developer Expert
- Gautier Mechling, Android developer at Innovorder and Google Developer Expert
- Adrien Vitti, Android developer at Smart&Soft
Most of the committee are organizers of the Paris Android User Group. The event is co-organized with BeMyApp.
The role of the committee
The mission of this group was to:
- ensure interesting topics
- keep quality high
- ensure a variety of topics
- keep the level of the talks properly adapted to the audience;
- encourage inclusion and diversity
We tried to forecast the trending topics so that our audience would be well prepared for them. We sometimes have hints that a given technology/product will be soon deprecated. In this case, we avoided selecting talks about that subject. If we think that a technology will be growing, we tried to get more content about that.
Part 1: Call For Papers
How did we get so many great speakers for a 2 days conference?
First, we opened a “call for paper” (CFP) website. This is a place where speakers from all over the world, from any background, could propose a talk. This website was open for many weeks and received 253 talks proposals from over 200 different speakers.
While the CFP was open, the organization team promoted it on social networks. The committee was also in charge of recruiting key speakers, which is a long-running task for some speakers. Organizers regularly solicit throughout the year getting speakers interested in supporting the event.
In the CFP, each speaker filled their profile with information about their company, speaking experience, and social media. This helped us evaluate their experience.
Each proposal also included: a title, a public description, a private message for the committee, the speech format (length of the talk, or if it is a workshop) and language (French or English).
The following chart shows the received proposals over time. You can see, this year we received a lot of proposals right before the CFP closed.
Part 2: Ranking the entries
During the CFP, each member of the committee reviewed each of the talks independently. It can take up to 30 minutes per talk, and sometimes we had to ask the speaker for more information to evaluate a proposal correctly, which took even more time. The output of this review was a score for each talk. We rated each talk (from 1 to 10), which was used later during the selection.
During this process, we considered different aspects of every talk. The criteria were:
Subject
We evaluated the relevance of the topic. We asked ourselves: Is this topic new or is it something already covered? Is it going to interest enough people?
For example “Getting started with Kotlin” is a topic that has been covered a lot recently. There is a limited value of this topic unless it’s discussed from a different angle.
Description
The abstract describing the talk will be published later in the schedule. We asked ourselves: Does this description have enough info to explain the content provided during the talk? What will be the take away for the attendees? How will the speaker cover this topic?
To let the speaker give more details, there a “Message for the committee” section where they could give an outline, provide a link to the talk’s slides, or include any other information proving the value of the proposal.
Speaker
To evaluate the speaker we asked: What is their experience? What is their expertise on this topic? Do they have any previous talks we can review?
Having experience as a speaker is a plus, but we did refuse some very qualified speakers based on the topic. We did accept some in-experienced speakers that had other examples of their expertise on the subject (like a tech blog post, or professional experience).
Conflict of interest
A committee member can decide if they are not able to rate a proposal impartially. This can happen if the speaker is working for the same company, if the topic conflicts with a talk that they have proposed, or any other reason they feel. In this case, the member won’t vote for the proposal or be part of the debate about it.
Part 3: The selection
Picking up the winners
Once the CFP had been closed, each talk is reviewed by each member of the committee.
Then we start organizing the selection nights. The members will gather and discuss the line-up for the conference. For the AM19, we met twice, and each session was over 5 hours.
To prepare these meetings we compute the available slots, to know how many talks we could fit within our agenda. This year there were 130 slots. A slot is 25 minutes long, so a regular 40 minutes talk would take 2 slots, and a 3 hours workshop would take 7.
This year we started by reviewing the talks with the largest standard deviation between ratings of the members. We debated about them and made sure we all understood the proposals the same way. The members could change their rating based on this discussion.
Now we were ready to pick our first talk!
We sorted all the talks by average rating. For the highest rating, we debated the talk, looked for similar proposals, and other talks by the same author. If we decided to go with this one, we added it to a list of “pending acceptance” and declined all the others from this speaker and the others with conflicting content. We kept the second best talk on a topic as a backup, in case the first speaker needed to cancel.
Then we checked the next highest ranked proposal, and continue the process.
We tried to have only one talk on a specific topic and one talk per speaker. There are very few exceptions to this basic rule.
This process took a long time (two full evenings for a total of 10 hours). When we had enough talks for the conference, we started sending approvals to the first wave of speakers. We used this to adjust our overall consistency based on the first responses we receive.
We are an international conference and we aim to have most of our talks in English. But, because we are in France we keep some slots in French. We had 4 rooms, so decided to keep at least one talk in French for each time slot. (approximately 25% French, 75% English).
We used this format to display the CFPs during the selection process:
When a talk was refused or accepted, the email was not directly sent to the speaker, because we sent the emails by waves.
Sponsoring the speakers
Some speakers required help with travel and accommodations. Our conference is not expensive compared to many others on the same scale, which doesn’t allow us to pay for every speaker, but we did have a limited budget some based on need. When we selected talks, we took this into consideration but we rarely refused a talk only because of this.
Some speakers asked for help after they’ve been selected, which was hard for us as we already allocated our budget to the speakers who requested it as part of their proposal.
None of our speakers were paid by the conference organizers, which is why we are extremely grateful for the donation of their time and effort. We couldn’t do this conference without them.
Some of our talks were “sponsored”. That means a few companies paid to have an employee selected to speak. We had 4 sponsored talks on our agenda (out of 69 talks). In this case, we worked with the company and the speakers to find the best talk quality to ensure these were as good as the other talks on our schedule.
The Heartbreaker
Selecting the speakers is hard!
We had 253 proposals for the 69 spots. This means a talk had a less than 30% chance of being selected.
With such a “small” amount of slots to fill, the committee had to eliminate a significant number of very good proposals. It was very frustrating for the whole team to refuse some valuable content. If your proposal was refused, don’t feel bad about your content or skills.
Every year, a few refused speakers ask why their proposals weren’t selected. We provided constructive and transparent feedback to all of them.
Part 4: The Agenda
Once we received acceptance from a speaker, we started to build the agenda. For each talk, we evaluated the potential audience, room size and chronology of similar topics (an introduction talk must happen before an expert version for example).
A couple of weeks after the end of the CFP, we shared the first version of the agenda. It could still change, but the attendees could start to get an idea about what the conference will look like.
Conclusion
To conclude, we wanted to share that we are incredibly proud of the quality of all the proposals we received. We could have accepted more than 90% of them, but we only had room for 27% of them. It doesn’t mean your talk was not good if it wasn’t selected, just that some other proposals were a better fit with our requirements and our constraints.
This year we are very proud of the result. This is not only due to our work but is thanks to the huge community of speakers who submitted amazing content.
For that, we want to thank you from the bottom of our hearts.
See you at Android Makers 2019!
Update: After publishing this article we received feedback and wanted to bring some clarification on the “encourage inclusion and diversity”.
We have never selected a talk based on the ethnicity or gender of a speaker.
We encourage inclusion and diversity by reaching out to valuable speaker to submit a talk before the CFP closing date. Our role is to increase the number of proposals from groups that are underrepresented in the dev industry to increase the chance of having diversity in the lineup.
Indeed, from our experience, once the CFP is closed and the selection is starting, the die is cast. At this point the chance to be selected for any speaker will be the same. If we had ever decided to lower our requirement to select more people from an underrepresented group, then it would have shown and harmed our community in the long run.
