HAI Reading Response

Critical response #2: Stanford CS 470 / MUS 356, Music and AI
Prompt: Challenge assumptions and conventions and discuss new possibilities presented in these two articles:

• https://ccrma.stanford.edu/courses/356/readings/2014-power-to-the-people.pdf
• https://medium.com/artful-design/humans-in-the-loop-b83e3bffa65e

Response:

HAI talks about reforming the machine learning design process to be human-centric. These two articles describe some tactics to modify the process and boons of adapting new methodologies. The promise is that we will end up with products and models that are more impactful and fit more seamlessly into the processes that we currently have. The articles provide frameworks for a more human-centric process including…

• The concept of involving non-ML scientists more frequently in the design process
• Questions to ask ourselves like “In what ways can the system I am designing incorporate human curation into the loop?,” “What might an interaction model or “user interface” look like?,” and “How does the AI model support such an interaction? What do I need to tweak to make it work?”
• 3 principles: value human agency, granularity is a virtue, interfaces should extend us

They move the conversation forward by getting us to ask questions and reconsider our values but fall short in creating concrete frameworks that can be integrated into ML 101 curricula and strategies to get ML-adjacent technologists more familiar with the models we build for them, both of which seem critical for realizing the goal of a more seamless workflow between two fields. I think that in order to accomplish this we need to challenge the assumption that non-ML scientists’ core value add is in output critique, build tools to elucidate how specific weights and biases (WAB) models work in an intuitive way, and understand the implications of taking the HAI design approach.

These articles describe traditional design methodologies where end-users offer feedback that sends ML engineers back to the drawing board. It’s based on this assumption that end-users should only interact with outputs. What does challenging this look like? I think it’s about incorporating programming and ML concepts into general education.

WAB models are inherently inscrutable because we are not built to distill meaning from numbers — we can’t perceive timbre from looking at a sound wave. Buildings tools that provide insight on what models are doing can allow end-users to give more practical improvement recommendations. One example of this is a 3Blue1Brown youtube video where Grant Sanderson displays how different layers and neurons represent pieces of information that are combined into outputs, a bit like a logic game.

Finally, we need to challenge the assumption that the current way we accomplish tasks is the right way. The models proposed by HAI create technology that fits seamlessly into existing procedures. When we do this, we allow for rapid progress and development of tech that is helpful but it seems like we put ourselves at risk for missing large leaps forward where entire processes are redesigned. We might miss the opportunity to reshape the way we learn concepts or make music.

I said that these articles don’t create concrete frameworks and frankly, neither did I. The process of integrating design concepts into tangible principles and methodologies seems challenging. At the same time, we don’t want to slow the rapid advancement that new models and data processing techniques bring. For now, it seems like the questions and values presented in “Humans in the Loop,” are invaluable to keep in mind.

10 Activities: “A tool that…”

1. Literature reviews: finds studies, confirms or denies your hypotheses, and guides you to find what comes next from existing research.
2. Music education: listens to you play an instrument and suggests exercises to improve deficiencies.
3. Commenting code: takes short hand comments you make and formalizes them to make code more intelligible.
4. Scheduling: takes all those “we should get lunch” plans and actually makes plans for both of you.
5. Recommendation letters: writes recommendation letters for students based on student and professor input
6. Diagnosis: takes as inputs information about when a person sleeps, what they eat, etc and connects it to symptoms.
7. Software customer service: helps people understand how to use software and problem solve issues.
8. Spam filter: prioritizes emails for you and automatically deletes/reports spam mail.
9. Driving: automatically drives and allows for the user to set priorities like speed and fuel efficiency.
10. Connections: connects people who haven’t met but should.

It seems like most of the ideas out there (automatic meal planning, movie recommendations, deciding what to wear, etc) take away all those decisions that make up real life. “What should i eat for dinner?” “What do I want out of a TV show?” I tried to make these 10 *not* replace everyday life experiences.