Design methods for meetings: resources & lessons learned
CMU Graduate Design Thesis 2018–2019
This post is part of an ongoing documentation series for my design thesis research on Maintaining Policy Intent for City Planners. Please find the full collection here.
For most of Fall 2018 I spent a lot of time talking to planners and working alongside policymakers to understand how they maintained policy intent. I also took a deep dive into studying design for policy, an emerging discipline that clarifies the value propositions for incorporating design thinking, design research, and design tools in public policy-making and implementation. While the research revealed a handful of directions in which I can take my thesis, I decided to focus on designing for a specific context of policymaking: planning meetings.
This article includes:
- Why planning meetings?
- Why design methods?
- Existing Design Resources for Public Sector Employees 💪
- Lesson-learned: why don’t planners use these design-inspired toolkits?
1. Why planning meetings?
Meetings, according to my previous research, are (1) the single most common method that covers all aspects of maintaining policy intent (clarify intent, follow through, inform all, and make it real), (2) a collaborative environment whose outcome is highly dependent on human behaviors, and (3) depending on the organizational culture and capacity, a convention that has seen little to no changes in how it is done.
Planning meetings — core meetings, task force meetings, and public meetings — are all supposed to carry through the initial policy intent (a.k.a. accomplish the community goals) but can often fail to do so for a range of reasons. Low attendance, low engagement, grandstanding, conflated discussion, no tangible outcome … you name it. The bottom line is that poor meetings, by not maintaining the intent efficiently, impacts the quality of policy outcomes.
2. Why design methods?
Applying design methods and mindset to a process generally means that there will be a higher emphasis on creating or addressing the tangible human experiences from the initial inquiry to the creation of proposed solutions. In the case of a comprehensive planning process, the following four aspects are the most impactful.
To zoom-in more into the policymaking stage, you’d see that both comprehensive planning and design have the divergent-convergent thinking, or the “double-diamond”, embedded in the process. This framework makes design methods much more organized and approachable to policymakers. It is also noteworthy that since many design methods are designed to guide discussion and encourage collaboration, naturally they would be best applied in contexts where people work together— you guessed it right — planning meetings!
3. Existing Design Resources for Public Sector Employees
In recent years, more and more public agencies and governments have sought to innovate their practices and have invited designers to discrete projects or stand-alone “innovation labs” to push for organizational change. A common product of these activities are design toolkits — collections of design methods created specifically for public servants to help them tackle specific challenges in their daily work.
- Design for Public Service by Nesta and IDEO
Slightly long read and has a full chapter on theory of design, toolkit is heavy on instructions - Civic Service Design Tools + Tactics by NYC Opportunity Service Design Lab (Mayor’s office)
Easy to read website/medium post/downloadable materials, has case studies on public service redesign in NYC — less about policymaking - Development, Impact, and You (DIY) Toolkit by Nesta
Website and PDF toolkit. Very succinct layout and worksheet-heavy tools. Designed for public and social innovators. - Open Policy Making Toolkit by Policy Lab (UK)
Web-based toolkit, featuring categories like low-cost tools and what to do if you only have 24 hrs. The toolkit includes both design methods and use of data/technology (they have a separate digital service manual). - Policy Method Toolbox by Department of the Prime Minister and Cabinet (Australia)
Also a web-based toolkit with skim-able bullet points. The collection is not as comprehensive as Open Policy Making Toolkit. - SILK Method Deck by Social Innovation Lab Kent (UK)
A deck of cards. very light on explanation/how to but good for an overview or if you know the tools already and just want to pick and choose. - Legal Design Toolbox by Legal Design Lab (Stanford University)
It focuses more on law/legal affair than policy and has a list of communication design techniques and digital tools. - Innovation Tools by Australian Government Public Sector Innovation (Australia)
It has a table of what tools to use/not use by stages of a process and an interesting innovation diagnostic tool. Much of the instruction/content is still work in progress.
4. Lesson-learned: why don’t planners use these design-inspired toolkits?
“Here are all the great toolkits … why haven’t planners used them?” My assumption is that even if planners learn about the tools (which most of them don’t), very few will actually apply the tools because the tools weren’t designed to encourage adoption.
Hypothesis: Existing design toolkits aren’t designed to encourage adoption, which makes planners less likely to implement design tools in their meetings.
To verify my hypothesis, I worked with planners from different municipalities for ten consecutive days to understand their existing meeting challenges as well as both their motivations and hesitations behind adopting new design tools in meetings.
Did I verify my hypothesis? Yes — in a way. I found that the form and instructions of design tools can be improved to encourage adoption, but simply doing so won’t help planners make the actual leap. The lessons I am about to share do not all provide solutions to increasing adoption. They are nonetheless important context of which designers should be aware.
Without further due, here they are:
#1 Planners concern quality of human interaction over tangible outcome.
When I asked planners to describe successful and failing meetings, I found that they often first referenced the quality of interactions before the tangible outcome of the meetings. Disclaimer: planners do really care about the outcome when you ask specifically. This is to say that planners may be (more) attracted to tools that promise to improve human interactions.
#2 Existing tools support information delivery but do not shape the consequential discussions.
The most common tools used in planning meetings are maps, powerpoints, staff reports, visuals — they help clarify information presented but they don’t shape how people discuss upon the information to move forward with the intent. Except for certain public meetings, almost no activity, other than “talking”, are employed to shape the discussion in most meetings. This finding is encouraging because most design methods and tools are actually designed to shape discussions — the latter portion that’s lacking.
#3 Resource and benefits are two big concerns for trying new tools, and mapping them out only reinforces their willingness to try (or not try).
When I asked planners to list and rank their criteria for applying new tools, resource (time, cost, etc.) and benefits (outcome) are two universal factors for adopting any new tools. All planners are very sensitive to time not only because that they are underpaid and overworked but also that they can’t afford to waste other people’s time. Cost is another critical item for that, especially in the public sector, every purchase needs to be justified. The promise of producing good outcome is also critical because it gives legitimacy to not only persuade themselves but also the management.
I originally created a mapping exercise to see if there would be a hierarchy of factors across participants. While there was no pattern, as seen in the diagram above, I found that this exercise reinforced people’s willingness to try — doubtful one became more discouraged and hopeful ones more excited about the outlook after the opportunity mapping exercise. It reminded me that designing around people’s baseline tendency will be key to success.
#4 Criteria for new toolkits and tools reflects a sense of urgency.
For planners, the faster they can get to the “how” of the toolkit the better. It means that they’ll skip through all the design manifestos and the “whys” and only focus on highly visual materials to learn how to use in a minute or two. Aside from the constraints mentioned in #3, evidence of success and innovation (“really out of the box”) are important — because that’s why it’s even worth the effort to change.
#5 Top features for the ideal toolkit should help planners introduce the tools to others.
I asked planners to imagine their ideal toolkit and list out the top-five features, and the result neatly captured all the findings above. In addition, planners also pointed out an important factor: ease of introduction. “What is it” and “worksheets” are ready-made material for them to introduce the tool to others and get approval, so the design of these elements need to be more inclusive to other stakeholders.
#6 The ideal product of a meeting, with new tools, should be tangible, accountable, and human.
I again asked planners to imagine their ideal outcome of using the tools. It was then progress-oriented items like action items, responsibilities, roadmaps got universal support. Echoing #1, strengthened morale and relationship is not only important as a public service, especially in public meetings, but also important for ensuring that meeting participants are actually willing to accomplish items assigned to them after the meetings.
#7 The initial low when trying new tools is universal, but the experience of trying is also universally positive.
Observing the journey of planners actually trying the tools (of their choice, which they already gravitated towards), it came with no surprise that almost everyone experienced some low point in confidence in the beginning— feeling scared, confused, unsure, and doubtful. Nevertheless, once planners tried the tools they all reported to be significantly more positive and willing to try more.
The finding gave me some initial validation of the tools’ usability, while pointing out the larger challenge of confidence. Confidence can still be an issue even when the planners, after all the struggles, are willing to sit down and try the tool. This realization led to this research question:
How might we design the tool on-boarding experience to reduce the initial confidence sink?
Most importantly, the confidence building has to work for not just the planner but everyone involved. To bring a new tool to a meeting, the tool has to be introduced at least three times: to the planner, to the manager, and to the meeting participants.
Final Thoughts & Next Steps
As my research scope narrows, it has gradually moved away from designing brand new activities but to designing new on-boarding experiences that increase adoption of tried-and-true methods. A part of me feel that I am avoiding designing the meat of the “true innovation” (a.k.a. the design methods), yet another part of me cannot unsee the problem of effective design toolkits sitting on the shelves and never get used.
Ultimately I need to identify the (best) tools for maintain policy intent in meetings and come up with a matrix to measure the tools’ impact. The design of on-boarding experience will address the “human-interaction” piece of of the whole process, despite small in comparison to the larger issue. I’d appreciate any thoughts, critiques, and recommendations on moving forward with all the research findings.
It sounds so fun! Can I join? 🙋♀️🙋♂️
YES! From now to end of April 2019 I’ll be looking for small groups planners to work with — to explore potential solutions and to test out the final design. These are paid opportunities that can be done remote or in-person. Please email yinjenw@andrew.cmu.edu if you are interested!
I’ve also been doing Lunch & Learn talks in planning departments where I introduce design tools and run fun activities with planners to try out these tools. If your department is interested in having me, please do let me know as well :) I’ll bring donuts and coffee! 🍩☕🎉
