Great questions, Julien! Here are some answers:
1. How did we analyze what was wrong with our previous format?
It was a team self-analysis, triggered by external comments. Basically, we got the feeling that what we were delivering wasn’t at the right quality level. Then a discussion clarified that our design critique were too mellow, so we tried to reignite them into a better forcing mechanism towards design quality.
2. How do you assess “good design” ? Is it a completely subjective feedback, or on design principles for instance?
It’s a mix of both. The usefulness of sole subjective feedback of a panel of informed peers is not to be underestimated, though. But yes, we also have a kind of framework, closer to design checklist than design principles. (I’ll share more on that in a coming article.)
3. “No preparation” is to waste some precious time and attention. Don’t you think that some kind of preparation is still useful?
Maybe “no artifact needed” would be more accurate. The idea is to show what you’re working on. Don’t create a presentation, to make it pretty — show us your screen. Now, yes, taking a few minutes before the session to clarify the pitch of your approach is necessary. And that’s a good thing, because the simple exercise of explaining something always makes it clearer in your head. But that’s it. We’re all designers, we can bear with your messy file.
4. “If you need more time, it’s probably because you haven’t decided what is important and what is not.”: I just wanted to say that this is amazing
Thank you. The truth is we’re struggling with that specific point. It’s hard to be concise, because most of what is being shared is work in progress, which by definition is not as clear as it should be. We’ve decided to increase that constraint, by reducing the pitch time to 2 minutes.
5. Have you considered inviting people that are not in the product team in your design critiques?
I think that’s a false good idea, because the success of those sessions is coming from the level of trust, the shared knowledge and the feeling of being among peers. That allows us to go fast, without having to spend time reexplaining obvious notions or sugar-coating a point of view. (Of course, we have other reviews with other folks, following a very different format.) That being, we invited a guest once, with one rule: if you come, you have to show one thing you’re working on. Our guest blew us away with an amazing project, very nice moment. Oh, and the other day, two 13-year old kids joined our session. They struggled to ask questions, but they were 100% focused during the whole time, and participated in the final vote with enthusiasm!