Machine-Teaching at Bloomberg
Week 3 (Jan 28 — Feb 1, 2019)
Previous week recap: Team Muffin/ChiHuaHua dived deep into our project brief and started to tackle the challenge of designing better data annotation tasks, tools, and guidelines for Bloomberg’s teams to assist them in creating better machine learning models.
As Team Muffin/ChiHuaHua entered into our third week of our capstone project, we’ve ironed out our team processes and organizational structure, and moved into background research. This week was all about immersing ourselves deep into the knowledge surrounding our problem area: machine-teaching at Bloomberg. Our team jumped into literature reviews, secondary research, interviews with experts and professors in this domain.
Machine learning: A human-centered approach
Machine-training is a fascinating problem space for human-computer interaction. Though it is a technical problem, it is also a human problem as well. HCI is important to training machine learning models as it can promote both efficiency and quality in creating annotations. Studying these tools with an HCI perspective can create tools and processes that people are motivated to use and use well. On the other hand, without HCI, annotation tasks can have unclear instructions, lead to failure to annotate, and produce inconsistent labels. Essentially, HCI and UX can affect the whole pipeline of machine teaching, impacting the quality of machine learning models. We see in examples like Google Doodle and Captchas, how well-designed machine-training tasks can be extremely successful in getting ground truth on a data set from a large group of people.
Understanding the stakeholders
In our kick-off meeting with Bloomberg, we planned to cover the basics about the project, but even more, we have planned to conduct a co-design activity with them. We are hoping that through this exercise, we will have fun, gain insights about our sponsor’s mindsets, promote empathy by putting them in the shoes of the end-user, and encourage them to think more creatively about the project brief.
In the design activity, we will all play the role of data annotators completing fun data annotation tasks (such as, differentiating muffins from ChiHuaHuas). After putting ourselves in the mindset of an annotator, we will think of out-of-the-box ways to reframe the problem by having “How Might We” questions. For one of these questions, we will rapidly generate design solutions, and finally, think about how a user might feel using one of these solutions by producing an empathy map. We’ll keep you posted about how this design activity goes!
The problem of machine-teaching is a human-machine problem, especially in the context of helping Bloomberg. Our challenge is as much a technical challenge, as it is a very human challenge.
Next steps — researching four domains of the project
- Understanding the end-user
- Understanding the organizational context
- Understanding the technological domain
- Understanding the annotation platform
As I mentioned before, the problem of machine-teaching is a human-machine problem, especially in the context of helping Bloomberg. Our challenge is as much a technical challenge, as it is a very human challenge. In our research plan, we plan to uncover the messy contexts by considering Bloomberg’s cultural make-up and organizational make-up. Armed with human-centered design activities and research methodologies, we are pressing forth.
We are just itching to begin deeply understanding the needs of the stakeholders involved in our challenge.
Next up: the real kick-off meeting, co-design activities, directed storytelling sessions, and setting up contextual inquiries!
About this series
We are writing a series of medium posts to track our progress throughout our capstone project for Carnegie Mellon University’s Masters of Human-Computer Interaction program.
Team Muffin/ChiHuaHua is working with Bloomberg to augment their data annotation tasks, tools, and guidelines for their teams, assisting them in creating better machine learning models.
Feel free to reach out, comment, and share.
