UbiComp/ISWC 2023 — back to a face-to-face conference in Cancun, Mexico

Marios Constantinides
SocialDynamics
Published in
6 min readOct 18, 2023
Bienvenidos (Welcome).

In 2019, I attended UbiComp/ISWC, the premier interdisciplinary conference on ubiquitous, pervasive, and wearable computing, in London, UK. Back then, I wrote that, “I was looking forward to next year’s conference in Cancun, Mexico…viva!” However, UbiComp 2020 took place virtually due to the COVID-19 pandemic.

Four years later (Oct 8–12), UbiComp/ISWC took place in Cancun, Mexico (kudos to the organizing committee for hosting its first edition in #LatAm).

FairComp: Fairness and Robustness in Machine Learning for Ubiquitous Computing

I co-organized the 1st workshop on fairness and robustness for ubiquitous computing, along with Sofia Yfantidou and Dimitris Spathis (in-person), and Tong Xia and Niels van Berkel (remotely).

We had three fantastic keynote speakers. First, Dr. Akhil Mathur (Meta AI, UK) addressed challenges related to responsible AI in ubiquitous computing systems, such as data challenges stemming from high variability. He also shared some of his recent work on understanding bias propagation in on-device machine learning workflows, emphasizing that fairness is a process, not just an optimization task. Second, Prof. Ricardo Baeza-Yates (Northeastern University, US) discussed cases of irresponsible AI and proposed solutions through principles, governance, and regulation. Third, Prof. Flora Salim (University of New South Wales, Australia) delved into the trade-offs between group fairness and individual fairness, presenting her group’s research on machine learning robustness (i.e., the ability of generalization to different data qualities) for ubiquitous computing applications.

Left: Akhil Mathur; Right (top): Flora Salim; Right (bottom): Ricardo Baeza-Yates (remote).

The workshop featured five paper presentations, covering frameworks and techniques for achieving fairness, privacy, security, and robustness in machine learning models, with a particular focus on applications in ubiquitous and mobile computing. Specific topics included designing fair ubiquitous computing systems, enabling private and secure collaborative learning for human activity recognition, using smart contracts for ethical data collection and usage, evaluating the fairness of privacy-utility mobility models, and assessing fairness in clustered federated learning. These papers are published in the Adjunct proceedings of the conference.

In the afternoon, we conducted a collaborative activity among attendees, followed by a panel discussion on “Ethical & Responsible UbiComp: A Case for Fairness and Robustness.” During the collaborative activity, we divided into two groups to discuss challenges and solutions related to (a) data collection, annotation, exploration, and manipulation and (b) model development, evaluation, deployment, and use. We reached a consensus that there is a lack of emphasis on fairness in data collection and model reporting for UbiComp studies/applications. One proposed solution was to create (or adapt) a specific “data-card” and/or “model-card” tailored to UbiComp applications, providing standardized guidelines for reporting datasets and models. During the panel discussion, we invited Dr. Akhil Mathur, Prof. Flora Salim, and Prof. Anind Dey. We covered a variety of topics, including fairness in specific UbiComp application domains (e.g., health, mobility, human-activity recognition), discussed various trade-offs such as accuracy vs. fairness and fairness vs. privacy, and touched upon upcoming AI regulations.

Left: shots during the collaborative activity; Right: Anind Dey, Akhil Mathur, and Flora Salim (from left to right) on discussing “Ethical & Responsible UbiComp: A Case for Fairness and Robustness.”

WellComp, EarComp, Omnibuds, and GenAI4PC

Shout-out to the Pervasive Systems group at Nokia Bell Labs Cambridge (UK), who also organized the 6th edition of the WellComp workshop covering topics related to computing for well-being and the 4th edition of EarComp focusing on earable computing. The Cambridge team also introduced the new Omnibuds device, which is a sensory platform for human augmentation. Unfortunately, the GenAI4PC symposium on Generative AI for Pervasive Computing coincided with our FairComp workshop, and I couldn’t attend. Special thanks to Shyam Tailor and his colleagues from Google and University of Cambridge (Robert Harle and Yojan Patel) for promoting such an important and timely topic in the UbiComp community. In the symposium, Dimitris Spathis presented some of his recent ideas on working with LLMs and numerical/temporal data, such as those derived from wearables or electronic health records.

Main Program

The main program featured over 180 papers (IMWUT, ISWC long/brief/notes) that covered a wide range of topics, including human sensing and health monitoring using wearable devices, sensing emotions, stress, and mood, as well as privacy, security, and robustness of ubiquitous systems, and mixed and immersive reality systems and interactions. Here’s a summary of some papers that captured my attention, spanning topics from responsible AI to mobile sensing applications to interactions in physical/virtual spaces. The full program is accessible online.

Responsible AI and Model Performance Evaluation

Jatinder (Jat) Singh, in his paper titled “Auditable XR: The Practicalities of Mobile System Transparency,” underscored the necessity for transparency in extended reality (XR) technologies and stressed the significance of auditability to ensure accountability. He introduced Droiditor, an open-source Android toolkit that captures audit-relevant data from mobile applications. Sofia Yfantidou presented her paper on “Uncovering Bias in Personal Informatics,” discussing biases in data generation and machine learning, particularly affecting users with specific health issues and female users, emphasizing the need for fairness in personal informatics. Another paper, titled “Designing Reflective Derived Metrics for Fitness Trackers,” explored explored how users of fitness trackers perceive derived metrics (e.g., stress scores), and how such metrics can support users’ wellbeing, while Lakmal Meegahapola discussed the generalizability and robustness of mood detection models across different countries in his paper titled “Generalization and Personalization of Mobile Sensing-Based Mood Inference Models: An Analysis of College Students in Eight Countries”. I also appreciated the effort by Xuhai Xu et al. in releasing the GLOBEM dataset, and evaluating the generalizability of depression detection models across universities. Another paper, titled “Mood Measurement on Smartphones: Which Measure, Which Design?,” investigated the impact of design alterations on user compliance, user experience, and the validity of mood measures, providing insights for designing effective mood tracking tools. Another paper, titled “What and When to Explain? On-road Evaluation of Explanations in Highly Automated Vehicles,” explored the role of explanations in highly automated vehicles, focusing on enhancing passenger trust and situational awareness. It discussed the impact of visual explanation types and timing mechanisms on passenger experiences, emphasizing the importance of understanding users’ needs in real-world driving conditions.

Mobile/Wearable Sensing Applications

The paper titled “Detecting Social Contexts from Mobile Sensing Indicators in Virtual Interactions with Socially Anxious Individuals” focused on mobile sensing to understand social contexts, particularly in the context of socially anxious individuals. The authors explored the feasibility of passively detecting social contexts in virtual interactions and its potential for identifying changes in social anxiety status. Another paper introduced “Mites,” a system designed for general-purpose sensing in buildings, emphasizing the challenges in deploying high-fidelity sensing systems in real-world building environments — check out also our work on sensing physical comfort during meetings using miniaturized devices. I also found the idea of using metaphors to communicate health information intriguing. In their paper titled “Sounds of Health: Using Personalized Sonification Models to Communicate Health Information,” the authors explored the use of personalized sonification models to convey health information through sound, discussing the feasibility of using sound to communicate health and wellness status and providing an alternative to visual displays for health information.

Natural Interactions in Physical/Virtual Spaces

The authors of “HyWay: Enabling Mingling in the Hybrid World” introduced a system designed to facilitate mingling between physical and virtual users in various settings, such as office spaces and conferences. They stressed the importance of bridging the awareness gap between physical and virtual interactions, enabling seamless communication and collaboration. A paper, titled “Hello, I am here’: Proximal Nonverbal Cues’ Role in Initiating Social Interactions in VR,” used nonverbal cues in VR to ease the initiation of social interactions. Another paper, titled “Affective Touch as Immediate and Passive Wearable Intervention,” explored the use of affective touch as a passive wearable intervention to alleviate anxiety in real-time, highlighting its potential in providing immediate help during high-stress situations. Finally, the paper titled “Using Wearable Sensors to Measure Interpersonal Synchrony in Actors and Audience Members During a Live Theatre Performance” investigated interpersonal synchrony in live theatre performances. This research used wearable sensors to understand social interactions among actors and the audience, showcasing the potential of wearable technology to enhance our understanding of in-person social dynamics in a live performance setting.

Overall, it was an amazing experience. Once more, a big shout-out to the organizers. Now, I’m looking forward to UbiComp/ISWC 2024 (and possibly MobileHCI 2024) in Melbourne, Australia.

--

--

Marios Constantinides
SocialDynamics

senior research scientist @ Nokia Bell Labs — hci, ubiquitous computing, ML, data science, responsible AI