Design Workshop: How We Got Creative with a Technical Audience at the AI Summit

By Ashley Golen Johnston & Russell Huffman

As software designers, it’s our job to help our non-designer counterparts understand the importance and potential of user-centered experiences. Effectively executing this idea, while not new or foreign in the software industry, is not always easy.

In July of 2018, our design team was invited to facilitate the annual IBM Watson AI Summit. The Summit is a three-day gathering of IBMers from sites around the world, who convene for the purpose of generating new ideas and methods for the development of artificially intelligent software features. The attendees of this annual Summit are comprised of some of the most highly respected experts in the field of artificial intelligence from various disciplines, including executives, engineers, offering managers, designers, and researchers.

The Summit historically had generated features and ideas with limited concrete end-user value, even if those ideas were technically interesting and conceptually viable. Our goal, as this year’s design facilitators, was to empower our highly technical attendees to flip their mental model and think in the context of solving user problems, rather than developing sets of new features with unclear user impact.

We had a daunting task ahead of us. We needed to formulate a plan to structure the Summit in a way that would enable the participants to focus on problems our users face today and think through plausible solutions to those problems.


What Did We Do?

In order to effectively facilitate the Summit, the design team had to guide the group with appropriate and meaningful activities, keep the discussions on track, empower the participants to think about the users and shift the focus away from technical minutia. With all of these factors in mind, we began planning with the knowledge that there were many unknown variables that could change how we facilitated. We chose, edited, and appropriated various Design Thinking and Gamestorming activities with the knowledge that the agenda could easily change, or those particular activities may not be effective.

For each day of the Summit, we meticulously planned a schedule of activities, presentations, and open discussions. We knew the group was brimming with ideas and that ultimately, the conversations that would occur during the Summit were equally as valuable as the activities we were preparing, so we padded all of the activities to include discussion time.


The Activities

We outlined a series of activities inspired by Design Thinking and Gamestorming, to accomplish various goals for the Summit. We even made up a few activities of our own. Each activity had a clearly defined purpose that was explained to the participants before we began.

The Introduction Activity

The importance of a good introduction is often overlooked. The Summit had a diverse mix of participants, and while all participants know of one another, few knew each other personally. We took the introduction activity as an opportunity to let the participants have some fun. We created a game where each participant had to introduce themselves, announce where they were from, and then select the real acronym out of the display of both real and fake IBM acronyms. This activity quickly turned into an opportunity to laugh and share anecdotes about careers, personal stories, and thoughts on software in general.

Hopes & Fears

Once the introductions had concluded, we decided to dive into another activity before jumping into presentations and ideation activities. We asked the group to participate in a Hope & Fears activity, where each participant wrote down their hopes and fears about the Summit. The goal of this activity was to gain a shared understanding of the participants expected outcomes and an understanding of the failings of previous Summits. The participants were encouraged to speak candidly, and in doing so the design team gained an understanding of how to guide the participants to avoid the pitfalls of years past. This alignment on expectations confirmed some of our assumptions and allowed us as facilitators to confidently move forward with the rest of the Summits activities.

Team Formations

To kick off the creative thinking activities, we decided to break the participants into teams. The use case at the center of the Summit was a rather large end-to-end use case. To help focus the participants on the right parts of this large use case, we divided them into four teams. Each team was given a persona, needs statement, and an as-is workflow map for a small section of the end-to-end use case to discuss. The goal of this activity was to allow each team to build empathy and begin to identify pain points for their users. By the end of the activity, participants had begun to mark their workflow maps with lightning bolts to indicate poor experiences, and map their users’ feelings to steps in the process.

Big Idea Vignettes

Once our teams had gained some empathy for their users, it was time to begin brainstorming. Each team member was asked to generate 3–5 big ideas that could solve their user’s pain points. However, the caveat for this activity was that the ideas could not be technical in nature. This presented the teams with a challenge — they had to think of solutions, without thinking about the technical feasibility or implementation of those solutions. The teams faced this challenge head-on and began engaging in animated conversation and generating a gamut of ideas from the practical to the absurd.

Post-Up Activity

Following on the heels of the Big Idea Vignettes activity was the Post-Up Activity. The goal of the Post-Up activity was to allow the participants to switch from creative thinking to technical thinking, and begin brainstorming methods and techniques to actualize their Big Ideas in the product. The teams were also given supplementary technical prompts that were meant to narrow their focus and ease this switch in thinking. The supplementary prompts were universally abandoned by all the teams, but the team members were able to successfully switch to focused technical thinking on their own. Given the background of the participants, the conversation around technical methods was particularly spirited and actually pushed the activity over its allotted time.

The $100 Test

After brainstorming both big and technical ideas, the teams moved on to the $100 Test activity. For this activity, the goal was for the teams to identify the most valuable ideas from each group by assigning each idea a relative dollar value when weighed against all of a given teams ideas. The teams quickly moved from room to room evaluating ideas- with a pitch team member left behind to “sell” ideas to the visiting teams. By the end of the $100 Test, each team had a come to a general consensus over which ideas were of the highest value.

The Innovation Generator

Once each team had “spent” their allotted $100 dollars, all of the teams converged together again to evaluate the high-value ideas and plot them on the Innovation Generator matrix. The goal of the Innovation Generator matrix was two-fold: 1.) to ensure that all ideas could be mapped back to a real user need and 2.) to encourage discussion as to what innovations could make a given idea more novel. As a whole, this particular activity failed. While all of the ideas could be mapped back to a real user need, no innovations were generated. However, this activity did spur productive group conversation that allowed us to combine, eliminate, and modify ideas.

Speedboat

After spending the majority of the Summit generating ideas, it was time to shift gears. For the Speedboat activity, all participants were asked to write down the things they feel hold the product back from moving forward. These thoughts were the “anchors” to our “speedboat” (the product). The goal of this discussion was to empower the participants to think about the things that hold us back, and actions we can take as individuals to change them. This activity was a long, but incredibly productive discussion that resulted in the generation of a concrete list of follow up tasks to be completed.

Prioritization Grid & Questions & Assumptions

We chose to close the Summit with a hybrid activity. We asked the participants as a group to help us plot the ideas from the Innovation Generator into a grid based on value to the user, and level of effort to develop. While the participants were debating how to plot each idea, the facilitators recorded outstanding questions and assumptions to be used for estimating risk when aligning the new ideas with the product roadmap after the Summit. The group was able to effectively plot each idea, and even delineate between ideas that required experimentation and those that did not.

A majority of our activities were actually drivers for conversations. Engagement in the activities themselves wasn’t always optimal, but all of the activities led to fruitful discussion among the participants. As facilitators, it was important for us to remember that the goal of the workshop wasn’t to simply perform activities, but rather it was to enable different ways of thinking and give focus and direction to these important conversations. When these discussions would veer off topic, or away from a real user need, we were there to redirect the conversation.


What We Learned

Our facilitation of the Summit was challenging and full of successes and failures. From those successes and failures, we learned a few key lessons in planning and facilitating a design workshop.

Lesson 1: Conversation mediation is essential

The group at the Summit was very willing to talk and many individuals had strong opinions, which was positive and negative. While it was tempting to shut conversations down to keep on schedule, much of the conversation was productive and necessary, and we had to be conscious of when to let the conversations run and when to intervene. This required us to listen carefully to conversations as they were happening, and anticipate the desired end goal of a given dialog. When the parties would shift away from what we thought was the end goal of the conversation we would have to interject, usually with a question about the original topic, to gently nudge the participants back on track. When this approach wasn’t effective, and the participants continued further off track, we would pause the conversation and add it as a parking lot item to discuss at a later time.

Lesson 2: User test your activities

In retrospect, it would have been extremely helpful to practice a few of our activities with some developers ahead of the Summit. Had we done so, maybe we would have known that the Innovation Generator and the technical prompts for the Post-Up activity wouldn’t work for our needs. However, because of these failures, we were forced to learn how to effectively pivot and improvise with the group when activities didn’t go according to plan. The need to be flexible and to recognize when it is appropriate to pivot the group were valuable lessons for the team.

Lesson 3: Have a plan for note taking

Some activities, such as the Prioritization Grid, were heavy on group discussion. We as facilitators found ourselves trying to capture what was being said, as it was said. This was extremely difficult to do while simultaneously driving activities. Many of these conversations were also unstructured, and it was difficult to take cohesive notes, get actionable takeaways, or summarize the conversations. In the future, we will formulate a plan of action for note taking ahead of the Summit (like assigning a dedicated note taker).

Lesson 4: Be organized, but also flexible

Organization was another key factor in our success. The pre-workshop planning effort that the team undertook allowed us to prepare a well-paced but flexible schedule, and set expectations with the participants before they arrived for the Summit. But, the best-laid plans can change for any number of reasons. Our plan had to change several times during the course of the Summit, but our willingness to adapt and improvise helped us keep the workshop moving forward.

Lesson 5: You need the respect of your audience to facilitate effectively

Workshop facilitation can be intimidating, especially when the participants are executives or distinguished members of the industry. We learned that successfully leading a room has to do less with authority and more to do with respect. Even though we were asking senior level participants to perform tasks they would not typically do, the workshop worked because there was an established respect. A key factor in gaining this respect was our ability to outline tangible goals for the participants every day. This allowed us to explain each activity, and its purpose, in the context of the day’s end goal. Once the participants could see the justification and intent behind the items on the agenda, they were able to buy into performing the activities we laid out.


Final Thoughts

The process of organizing and facilitating the IBM Watson AI Summit was an incredibly valuable exercise. While there were some bumps along the way, we did achieve all of the goals that we outlined before the workshop. One of the key factors to our success was our ability to successfully tow the line between the creativity of a Design Thinking workshop and the tactical execution of a technical workshop. Our blend of team building, technical, and Design Thinking activities helped us motivate a group of 20 highly technical participants to think differently and generate user-centric ideas that were valuable to our product.

Ashley Golen Johnston is a UX Designer at IBM based in Pittsburgh, PA. Russell Huffman is a UX Designer at IBM based in Austin, TX. The above article is personal and does not necessarily represent IBM’s positions, strategies or opinions.