Surfing the HCAI World at CHI 2024
CHI (pronounced “kai”) conference on Human Factors in Computing Systems is the premier international conference of Human-Computer Interaction. This year’s conference was held on 11–16 May 2024 in Honolulu, Hawaii, USA, with 3955 attendees.
Aloha! In this post, I highlight some of Social and Responsible AI team’s work at Nokia Bell Labs, Cambridge, UK and cover selected papers on HCI+AI (HCAI) topics including AI risks, societal impacts, taxonomies, morality and ethics, design practices and tools, and (Gen)AI-assisted work and creativity. This is a curated list of papers; you can access the full program of more than a 1000 papers. Also check this nicely curated list of HCI+AI CHI2024 pre-prints, or explore papers via this amazing visualization.
The Social and Responsible AI team at Nokia Bell Labs presented 2 papers in the conference’s main proceedings, and organized the 2nd edition of the Human-Centered and Responsible AI Special Interest Group (SIG). The SIG is an initiative to bring together world-class experts both from academia and industry working on Responsible AI.
Special Interest Group (SIG) on Implications of Regulations on the Use of AI and Generative AI for Human-Centered Responsible Artificial Intelligence (HCR-AI)
The SIG focused on addressing the ethical, legal, and societal challenges associated with the rapid advancements in generative AI and AI regulations such as the EU AI Act. This year’s SIG placed an emphasis on how human-computer interaction (HCI) research can evolve in the face of these challenges, aiming to develop new theories, evaluation frameworks, and methodologies tailored to navigating the complexities of AI ethics and regulation.
By bringing together a diverse group of stakeholders from various disciplines, the SIG aimed to foster discussion on developing AI technologies that are not only technologically advanced but also ethically grounded, accountable, and sustainable, steering AI towards outcomes that benefit all humanity. The discussions were centered around four themes: AI regulations and HCI research, impact assessment, governance, and evaluation frameworks.
- AI regulations and HCI research: Current AI regulations appear fuzzy, often sidelining HCI research despite its potential to contribute significantly to Responsible AI. Proposals included integrating HCI’s socio-technical skills into compliance processes, offering training for researchers to join AI advisory boards, and identifying mechanisms to influence AI policy effectively.
- Impact assessment: Concerns were raised about deskilling and the lack of reskilling opportunities in the AI-driven workforce, coupled with difficulties in anticipating AI’s societal impacts. Participants emphasized the need for thorough impact assessments before deploying new technologies and suggested focusing on creating meaningful human interactions with AI, especially for the younger generation.
- Governance: There’s a perceived gap in global compliance and governance. Discussions were centered around developing clearer governance models and practical tools to assess AI systems, urging a move towards more concrete and actionable regulations that include stakeholder involvement.
- Evaluation frameworks: Participants stressed the challenge of evaluating responsible AI use, particularly due to the lack of universally agreed-upon criteria. They called for a multi-stakeholder approach to define context-specific responsible AI criteria, focusing on the entire lifecycle of AI systems to ensure transparency and accountability.
Paper (pre-print): https://arxiv.org/abs/2403.00148
Group discussions: https://bit.ly/4bzu7GA
Paper: User Characteristics in Explainable AI: The Rabbit Hole of Personalization?
With the increasing integration of AI in daily life, ensuring that AI systems are transparent and understandable to users is critical. In this paper (in collaboration with researchers from the University of Glasgow), we cast doubt on the effectiveness of tailoring Explainable AI (XAI) systems based on user characteristics such as age, gender, and personality traits. We conducted a study, involving 149 participants, with the aim to identify whether personal traits (e.g., age, gender, personality) influence how individuals engage with, comprehend, and trust AI explanations. To study that, we developed and deployed a prototype AI system to flag inappropriate comments, and examined the relationship between user traits and their interaction with the system.
Surprisingly, only age and the trait of openness (a personality trait) revealed a significant correlation with users’ understanding of AI, questioning the current push towards highly personalized XAI solutions. This may suggest that users are more alike than different in their interactions with XAI systems. In other words, this challenges the notion that personalization based on user characteristics is key to improving XAI effectiveness.
These results call for a re-evaluation of user-focused research in XAI and suggest that designers might instead focus on general usability improvements applicable to broad user groups.
Paper (pre-print): https://arxiv.org/abs/2403.00137
Paper: Guidelines for Integrating Value Sensitive Design in Responsible AI Toolkits
In this paper (in collaboration with researchers from the Imperial College London), we proposed an approach that blends Value Sensitive Design (VSD) — a methodology that considers human values throughout the technology design process — with the development of Responsible AI toolkits. These toolkits serve as practical tools that guide developers in creating AI systems that align with ethical values, such as fairness and transparency.
Through a series of workshops involving 17 early-career AI researchers, we examined how two existing RAI toolkits (the Nokia AI Design toolkit, and the MIT Blindspots) incorporate VSD principles in their design. The study revealed that collaborative and educational design features within these toolkits, including illustrative examples and open-ended cues, significantly enhance the understanding and integration of human and ethical values in AI systems.
Building on these insights, we formulated six practical design guidelines for enhancing value sensitivity in the development of Responsible AI toolkits. These guidelines aim to empower other researchers and developers to incorporate human values into AI systems more effectively.
Paper (pre-print): https://arxiv.org/abs/2403.00145
Next, I cover selected papers that I found interesting on HCI+AI (HCAI) topics including AI risks, societal impacts, taxonomies, morality and ethics, design practices and tools, and (Gen)AI-assisted work and creativity.
AI Risks, Societal Impacts, Taxonomies, Morality and Ethics
The study “Deepfakes, Phrenology, Surveillance, and More! A Taxonomy of AI Privacy Risks” (by @hankhplee et al.) constructed a taxonomy of twelve AI-specific privacy risks based on an analysis of 321 AI incidents, highlighting the challenges in protecting privacy as AI technologies evolve. By involving citizens directly affected by e-government services in the research process, another paper (“Hostile Systems: A Taxonomy of Harms Articulated by Citizens Living with Socio-Economic Deprivation” by Colin Watson et al.) identified a range of harms that these systems can inflict (e.g., accessibility barriers and systemic biases). The authors developed a taxonomy based on the study that offers a structured way to assess and mitigate harms, suggesting that future research should focus on creating more inclusive digital systems that consider the broad spectrum of user experiences. This could involve developing methodologies that are capable of capturing a wider array of impacts, both positive and negative, and translating these findings into actionable design principles that prioritize marginalized communities. Another way is the Ability-Diverse Collaboration Framework by Lan Xiao et al., which seeks to reframe how technology is designed to support collaborative interactions among users with varying abilities. This framework emphasizes the importance of designing for interdependence rather than independence, reflecting a shift in how technologies should facilitate interactions.
The paper “Exploring the Association between Moral Foundations and Judgements of AI Behaviour” by Joe Brailsford et al. used the Moral Foundations Theory to predict responses to AI behaviors. The authors found that an individual’s technical understanding of AI systems primarily shaped their moral assessments rather than fundamental moral values (e.g., care, loyalty, or authority). Specifically, individuals who perceived AI behaviors as incorrect were more likely to ascribe significant agency and attribute human-like qualities to these systems, relying on their moral instincts to guide their judgments. Additionally, “Which Artificial Intelligences Do People Care About Most? A Conjoint Experiment on Moral Consideration” by Ali Ladak et al. tested whether features of AI systems such as autonomy, emotion expression, prosociality, among others affect moral considerations. They found that the presence of such features increased moral consideration, with the strongest effects from having a human-like physical body and the capacity for prosociality. As AI is perceived a threat to humanity (e.g., taking over our jobs), the greatest degree of moral consideration might only be extended when AI demonstrates positive and good intentions.
Design Practices and Tools
In “Designing a Card-Based Design Tool to Bridge Academic Research & Design Practice For Societal Resilience,” researchers (@NoviaNurain et al.) have developed a set of design cards aimed at translating academic insights into practical applications for enhancing societal resilience. These cards are designed to be generative tools that integrate complex research findings into the design process, facilitating more informed and robust design outcomes. “The Situate AI Guidebook: Co-Designing a Toolkit to Support Multi-Stakeholder, Early-stage Deliberations Around Public Sector AI Proposals” by @anna_kawakami et al. provides a structured process for deliberations in the public sector. The guidebook, developed through an iterative co-design process, aims to ensure that decisions about AI projects are made responsibly with consideration for societal, legal, and data constraints. The Farsight tool (“Farsight: Fostering Responsible AI Awareness During AI Application Prototyping”) by Zijie J. Wang et al. integrates AI ethics early in the prototyping phase. It does so by providing AI prototypers with relevant news articles and editable use cases based on their prompts. Farsight encourages consideration of potential harms and promotes user-centric design thinking. However, how one can evaluate such tools? The paper titled “A Scoping Study of Evaluation Practices for Responsible AI Tools: Steps Towards Effectiveness Evaluations” by @glenberman et al. critiques the current evaluation practices for responsible AI tools, noting a focus on usability over effectiveness. This study called for a broader approach to evaluating these tools, one that includes assessing their impact on AI development practices and outcomes.
“Design Principles for Generative AI Applications” by Justin D. Weisz et al. presented six foundational principles aiming at guiding the design of generative AI applications. These principles, developed through an iterative process involving feedback from practitioners, are vital for creating user experiences that are both effective and safe. However, the challenge remains to continuously update and refine these principles as generative AI technologies evolve and to validate them across broader application scenarios. Another paper, titled “DirectGPT: A Direct Manipulation Interface to Interact with Large Language Models” by @damienhci et al., explored the idea of enhancing user interaction with large language models (LLMs) through direct manipulation techniques. This approach, exemplified in the DirectGPT interface, allows users to manipulate outputs directly, making interactions more intuitive and efficient.
(Gen)AI-assisted Work and Creativity
“LabelAId: Just-in-time AI Interventions for Improving Human Labeling Quality and Domain Knowledge in Crowdsourcing Systems” by Chu Li et al. explores AI’s role in enhancing the quality of crowd-sourced labeling while also improving the domain knowledge of crowdworkers. In a similar way, “Human-LLM Collaborative Annotation Through Effective Verification of LLM Labels” by Xinru Wang et al. introduced a collaborative approach between humans and LLMs for improving data annotation accuracy — the method used LLMs to generate initial labels, which are then verified and refined by human annotators.
“How Knowledge Workers Think Generative AI Will (Not) Transform Their Industries” by Allison Woodruff et al. presented a study on the perceptions of knowledge workers regarding the impact of GenAI on their fields. The paper highlighted concerns about potential negative impacts such as deskilling and disinformation. Another paper, titled “How Do Data Analysts Respond to AI Assistance? A Wizard-of-Oz Study” by Ken Gu et al., examines how AI can enhance data analysts’ workflows by assisting in planning and execution. The authors found varying perspectives on what planning assistance means. On the positive side, assistance was seen as helpful for guidance on the overall workflow. On the negative side, it was seen as distracting when there was a tension between the intended goals of the assistant and the goals of the analyst. In a similar way, the paper titled “User Experience Design Professionals’ Perceptions of Generative Artificial Intelligence” by Jie Li et al. reflected on how UX designers perceive the impact of GenAI on their practice. They found that experienced designers are confident in their originality, creativity, and empathic skills, and found GenAI’s role overall as assistive. The factors of “enjoyment” and “agency” were emphasized as uniquely humans where humans will always remain in the driver seat over AI output. However, there were serious concerns over setbacks for junior designers who may be impacted by skill degradation, job replacement, and creativity exhaustion.
“Reading Between the Lines: Modeling User Behavior and Costs in AI-Assisted Programming” by @HsseinMzannar et al. studied the interactions between programmers and AI code-recommendation systems (e.g., those generated by Copilot). Interestingly, programmers may spend a large fraction of total session time (34.3%) on just double-checking and editing Copilot suggestions, and spend more than half of the task time on Copilot related activities. Another paper, titled “Shaping Human-AI Collaboration: Varied Scaffolding Levels in Co-writing with Language Models” by @dhillon_p et al., investigated how different levels of AI assistance impact the co-writing process. It identified an intriguing U-shaped relationship between AI scaffolding and writing quality. Low scaffolding (i.e., next sentence suggestions) disrupted writing flow, while high scaffolding (i.e., paragraph-level suggestions) significantly boosted quality and productivity. That is because sentence-level AI suggestions may hinder the writing process by forcing writers to switch between creating and assessing fragmented inputs. On the contrary, paragraph-level AI suggestions provide a structured framework that writers can adapt, leading to better quality and higher productivity.
Overall, it was an amazing experience (shout-out to the organizers; and much mahalo Hawaii for hosting CHI 2024).
See you next year in Yokohama, Japan for CHI 2025.
