PAIR @ CHI 2023
This week, members of the People + AI Research (PAIR) team at Google presented their work in Hamburg, Germany, at the 2023 edition of CHI, one of the world’s premier annual conferences on human factors in computing systems. Here, we highlight three timely PAIR papers presented at CHI, and give you a brief overview of the research. We hope you’re intrigued to dive deeper into the full papers, which look at designing tools for generative AI prompt programming, using large language models to prototype functional user interfaces quickly and efficiently, and how tech industry practitioners have used the PAIR Guidebook not only to address the challenges of designing for AI, but also for other purposes such as for cross-functional communication.
Programming without a Programming Language: Challenges and Opportunities for Designing Developer Tools for Prompt Programming
Alex Fiannaca, Carrie Cai, Chinmay Kulkarni, Michael Terry
“Prompt programming,” or writing prompts for large language models is clearly in the news these days with increasing interest in the emerging field. In this paper, PAIR’s researchers observe that developers often write prompts to solve complex problems without the structured and strict grammar of traditional programming languages like Python or Java. The researchers share new methods for extracting semantically meaningful structure of natural language prompts in the absence of a structured grammar.
Another challenge prompt programmers face is that in existing tools for developers, there’s little guidance or helpful support for prompting. To address this, PAIR’s researchers demonstrate a range of potential prompt editor features for developer tools, like semantic highlighting, autosuggest, and structured data views.
PromptInfuser: Bringing User Interface Mock-ups to Life with Large Language Models
Savvas Petridis, Michael Terry, Carrie J. Cai
It’s not only professional developers who need guidance with prompting LLMs. This exploratory study looks at how people with no AI programming experience or training can also quickly prototype functional user interface mock-ups, too — with prompt programming.
The researchers, working with five designers, found that with prompt programming, design prototypes are faster to create. This speed can then help inform designers on how to integrate AI and conduct user research sooner in the product design process. Based on their study, PAIR’s researchers built PromptInfuser, a Figma plugin for authoring LLM-infused mock-ups.
The plug-in introduces two novel LLM-interactions: input-output, which makes content interactive and dynamic, and frame-change, which directs users to different frames depending on their natural language input. The result is more tightly integrated UI and AI prototyping, all within a single interface.
Investigating How Practitioners Use Human-AI Guidelines: A Case Study on the People + AI Guidebook
Nur Yildirim, Mahima Pushkarna, Nitesh Goyal, Martin Wattenberg, Fernanda Viégas. (This paper received an Honorable Mention at CHI 2023.)
Back in 2019, PAIR published the first edition of the free, online PAIR Guidebook to share best practices, exercises, and examples for designing for and with AI. To date, the Guidebook has more than 550,000 unique users around the world — and this paper looks at how industry practitioners use it.
PAIR’s researchers conducted interviews with 31 practitioners and found that practitioners use the guidebook for support in early phase ideation and problem formulation to avoid AI product failures. They also found that AI guidance is useful not only for addressing AI’s design challenges, but also for establishing an internal culture around human-centered AI, and that practitioners used the PAIR Guidebook not only for addressing product design concerns, but also for internal training resources and clearer cross-functional communication on AI challenges.
