Partnering for Creative Design

A look back at the NSF-funded C3DaR project on supporting creative design.

Understanding designer intent from (a) sketching behavior as well as non-sketching behavior such as (b) pointing and (c) gestures. From Chandrasegaran et al. (2018).

I recently found myself writing the final report and the outcomes for the NSF-funded project “C3DaR — Collection, Creation, and Collaboration for Engineering Design and Reflection,” which just wound down this past summer. The project was funded for three years with two no-cost extensions. Project outcomes are supposed to be written for the general public, should summarize the outcomes of the award in 200–800 words, and are posted verbatim on the NSF website. As I was doing this, it struck me that this is a perfect topic for a retrospective blog post on the C3DaR project as well. Below follows the outcomes report I posted on Research.gov.

Design drives innovation, which today drives a large part of the U.S. economy. Take the Apple iPhone, which is widely recognized as a revolutionary product that forever transformed the landscape of mobile computing overnight. To maintain this competitive edge in design, it is clearly in our best interest to invest in science and technology that will not just sustain the current status quo, but that will propel our designer population to entirely new levels of creativity and innovation. Understanding how innovation happens, how designers work together, and how they can be aided, is critical to support and foster such progress.

The C3DaR project focused on developing such an understanding, building tools that aid innovation by helping designers collaborate, and helping researchers understand how designers work together. A joint venture between University of Maryland professor Niklas Elmqvist (human-computer interaction and computer science), and Purdue University professor Karthik Ramani (creative design and mechanical engineering), the C3DaR project adopted a radical approach to early-stage engineering design through the effective use of digital media. The C3DaR approach (pronounced “cedar”), short for Collection, Creation, and Collaboration for Engineering Design and Reflection, supported collecting existing ideas or sources of inspiration, creating new concepts, and collaborating with their teams to develop these and other ideas.

More specifically, each goal included the following components:

1. Collection helps designers gather materials in the early stages of the creative process, such as web browser integration for collecting inspirational material, design scrapbooking for composing content into multimedia montages, and diagramming for organizing and recording textual and hierarchical ideas.
2. Creation helps designers to make and modify design artifacts, such as textual and visual prompting to guide the designer’s sketching process, integration with a naive physics engine to enable an intuitive sketch-to-life design approach, and auto-completion of 2D sketches to enable rapid content creation.
3. Collaboration enable multiple designers to coordinate their combined efforts while avoiding interference and maximizing their productivity, such as social-media-like discussion, tagging, and voting for design artifacts, motion gestures and annotation to facilitate expressing dynamic behavior, and integrated visual conflict resolution techniques to coordinate the team.

Throughout the project’s duration, from Summer 2015 to Summer 2019, our team consisting of the two faculty members and a succession of graduate students and postdoctoral scholars made significant progress towards this vision. Below we review the most significant results from all this work.

Sketcholution summarizes the evolution of a sketch both as a comic (left) and as a numbered summary (right).

In the Sketcholution project, we focused on creation (the second C) by observing that sketches are more than groups of strokes; the order and timing of the strokes can also be useful for reflecting on a sketch, communicating it to others, and using an earlier version of a sketch as a starting point for a new one. We take advantage of this fact, we proposed a new method for grouping together the strokes in how a sketch was drawn based on their distance in time and space. This allowed us to generate “comics” that represent the highlights of the drawing process, as well as summarize the evolution of the sketch using numbered boxes in the same frame.

• Zhenpeng Zhao, William Benjamin, Niklas Elmqvist, Karthik Ramani. “Sketcholution: Interaction Histories for Sketching,” International Journal of Human-Computer Studies, 2015. [PDF]

In the Co-3Deator project, we turned to collaboration (the third C) for creating 3D models by proposing the idea of “team-first” ideation tools. Similar to “mobile-first” web design, team-first ideation tools are designed from the ground up with the designer team in focus. Specifically, in Co-3Deator, this is manifested in the way that all 3D parts that one designer creates is automatically and continuously segmented and shared with the rest of the team while the user is still drawing it. This enables other team members to reuse and remix the work of the original designer with no explicit sharing operations. For example, if a designer is creating a helicopter design, another designer could easily reuse the landing gear for an airplane design even while the first designer is working on other parts of their chopper. Collaboration becomes effortless and natural.

• C. Piya, Vinayak, S. Chandrasegaran, N. Elmqvist, K. Ramani. “Co-3Deator: A Team-First Collaborative 3D Design Ideation Tool,” Proceedings of the ACM Conference on Human Factors in Computing Systems, 2017. [PDF]

Ten minutes of coded video where the researchers have analyzed the behavior of a participant designer to understand their actions (top) and their intent (bottom).

Finally, in the “designer intent” project, we wondered whether we could go beyond traditional sketching (like in the Sketcholution project) by trying to infer the designer’s intent from both their sketching as well as their non-sketching (pointing, gesturing, representing) behavior. Based on painstaking video observations of designers solving practical design problems, we were able to identify several categories of actions and their purported intent. The paper was awarded with an honorable mention for best paper at ACM DIS 2018 in Hong Kong.

• Senthil Chandrasegaran, Devarajan Ramanujan, Niklas Elmqvist. “How Do Sketching and Non-Sketching Actions Convey Design Intent?,” Proceedings of the ACM Conference on Designing Interactive Systems, 2018. (Honorable Mention) [PDF]

Beyond the above papers, the C3DaR project yielded at least half a dozen additional publications in well-regarded academic journals and conferences. It also fully or partially funded a total of six Ph.D. students and one postdoctoral scholar. We ran several PK-12 summer camps on creative design using these ideas. In other words, C3DaR was a very successful NSF-funded research project with many and varied forms of impact.