Interaction Design Studio II

Microsoft Design Expo

Project in collaboration with Lisa Otto and Calvin Keetae Ryu

This project was sponsored by Microsoft

5/4/16

Final Presentation Day!

Here’s the link to our presentation PDF our final video and the testimonial of the ICU fellow that we worked with.

5/3/16

Final team dinner before the presentation!

4/30/16

We recorded our Narration pieces today. After finalizing the script for this as well yesterday. Tracy’s roommate Abby, a music student at CMU, was able to help us by recording all the narration lines for the video.

4/29/16

Today we took some time to finalize the CUI Link script and had one of the second year MDes students, Lorraine, record the lines to be the voice of link in our video.

4/25/16

Today we went over to the hospital to film our final pitch video. We were lucky because they had an entire unoccupied wing of the ICU that they let us film in.

We were also able to go over to the UPitt school store and get some lab coats and scrubs to wear during filming.

4/22/16

We’ve been working on prototyping with the 3d printer a lot this week… we’ve had a few failed attempts with the models, so we’re still working on fixing those to the fidelity that we would like.

4/20/16

Today we also went through and did a rough cut video with narration and clips from our written storyboards to work through transitions and timing for the video:

Sketch Video

4/18/16

We have been developing our final storyboard for the movie as well as a more detailed layout of our presentation.

We’ve also got our working script setup and have been coordinating the scenes for our movie. We have secured a room at UPMC to film in as well as recruited our actors, and gotten our props ready and worked on printing the form of the device that will transmit the technology.

4/13/16

Phase 4 Final Presentation

4/7/16

We conducted another mini workshop and interview today with a participant who has previously spent time in an ICU as a patient. We had him participate in our scenario study that had the patient, doctor, and CUI participate in dialogue and then he was able to shed light on some of his experiences and how the technology fit into those situations as well and as a patient what comfort levels might be with this type of device.

Interview Notes

4/6/16

Today we worked on the flow of the project presentation and synthesis of some of the information that we learned from our workshops yesterday. We’ve worked through the presentation structure and need to develop the content and presentation “story” flow to convey the work that we’ve done for phase 4.

Notes From Workshops

We also took a break to have a team bonding outing and get free gelato at Mercurios for grad student appreciation week!

4/5/16

We conducted two workshops today the first with one of the nurses who we have been working with and the second with the ICU fellow who we have been working with. We did two activities with each of them. The first activity was a test of the CUI where we had 3 pre-scripted scenarios and we had them read through them speaking the doctor portion while one of us called in on speaker phone and acted as the CUI answering the doctors questions and providing additional information.

The second activity we did had to do with the EMR. We made a simple mock EMR and input some of the information incorrectly. We then gave the doctor and nurse each a set of rules that they could follow to interact with the CUI to change or add to the information that it had input into the medical records. We used a paper cut out version of the EMR and we enacted the role of the CUI by answering or moving around different information as was necessary.

One final small activity that we did in both of our workshops was a form activity. We made some basic foam shapes to represent the different types of technology that might be how the audio and sound information is taken in and transferred and then had our participants try them out and think about how they would be useful in each situation. We also asked them about any insights into why certain forms wouldn’t work well also.

Nurse Workshop:

Scenario exercise with CUI impersonation

EMR reorganization exercise:

Testing out prototype forms:

Doctor Workshop:

EMR reorganization exercise:

Testing out prototype forms:

4/02/16

We met to work through more of our workshop activities today. We looked at how we wanted to refine the form prototypes that we were going talk to our participants about:

We also worked on the scenarios that we talked through with them: Scenarios

and we further developed the EMR exercise: EMR Exercise — Digital Version

Explanatory systems diagram:

3/30/16

Today we spent a bit more time working through how we will want to devise our workshop. We decided that we’ll need to test both the physical forms of what type of device that doctors would be comfortable having the CUI embedded in as well as the form that the interaction will take when the information is taken in and input into the EMR.

Steps before our next meeting:

Tracy: create prototype forms and write EMR information

Calvin: design EMR representation (digital and physical workshop pieces in google sheets and cut paper)

Lisa: Write script for conversations and rules for information organization interaction.

3/29/16

We spent a lot of time today working through what the story line that we will use to convey all of the different interactions and possibilities for the technology that we want to display in our system. We have 4 major pieces the first being the connection to the EMR system what we are calling “the brain”, the second through fourth being the different patient situations (non responsive, facilitating common interactions, and emergency)

We also started to rework a few aspects of our journey map to help see where all of our pieces were fitting in.

Questions that we need to answer and pieces we need to prototype:

3/28/16

Today we took a look at our different “rough sketch” journey maps and had a very lengthy discussion about how we want to integrate then with each other and what will be the most important journey aspects to highlight the different interactions that we want to be a part of our system,

Beginning to work out rough journey map:

3/26/16

We met today to review the findings from our second workshop and synthesize the positive and negative feedback about our different scenarios. We also took some time to compare the situations to the ones from the previous session to know which ones were continuously the most well received situations and elements that we should include in our final design.

We also took a bit of time to synthesize what we learned from our CUI role playing exercise. We found that, to our surprise, people responded the best to our more arrogant and quantitative personality, they really disliked the emotional/personable one, and were a bit ambivalent about the friendly helpful one.

3/25/16

We did our second workshop today with a few nurses from UPMC, one of them worked on an orthopedics floor and the other one worked in the ICU. We went through our storyboards that we revised to fit their work situations a bit better as well.

We also repeated our CUI prototyping exercise where we acted out different personas and had them react to the personas to see what they responded the best to when thinking about a personal assistant and how they would want to interact with it.

The Friendly Persona:

They overlooked this persona, it wasn’t bad, but wasn’t anything out of the ordinary either.

The Emotional Persona:

They really disliked this persona and thought it gave too many opinions and too much information.

The Arrogant Persona:

Surprisingly this persona was well received, which was contrary to our expectations. They liked the additional information that it provided and the fact that it was very analytical and didn’t try to do things that were human as the previous two did somewhat.

Debrief:

3/23/16

We met during studio today and synthesized a lot of the information that we learned from our workshop with the medical student. We made notes during the workshop with red post its for things that she liked, yellow for things she was ambivalent about, and green for the ones that she really liked. We want to be able to compare these to our results from our next workshop to see if there are any inconsistencies or confirmations of the findings. We’re also using what we learned to restructure a few of the scenarios or add a few more that will expand our range of results a bit further.

3/22/16

Earlier today during research methods we began to group some of our persona areas that we’ve been working with and define their similarities and differences by behaviors, goals, and demographics.

We also defined the behaviors, goals, and demographics of a few different CUI personas that we were working with as well:

We ran our first workshop of phase 4 with one of the medical residents who we’ve been working with for the past few workshops. We had two different activities that we went through with her. One to test out various scenarios and places where the CUI might fit within her work and a second role playing activity where we enacted the personality of the CUI to try and get her response to and preferences for different levels of interaction and knowledge.

3/21/16

During studio today we had a workshop using Little Bits to give us some other options for how to prototype our designs or work an interaction into various workshops that we are running if need be. We did a very quick charrette and created a scenario, storyboard and prototype to display the concept of a doctor being able to get audio feedback when they touch the door of a patients’ room to tell them patient vital signs before the enter.

Storyboard:

Prototype:

When someone touches the spoons they complete the circuit and the speaker emits the sound of the patients heartbeat.

After the Little Bits prototyping exercise we also started to work through how we might begin to prototype our CUI. We each took on a different persona and ran a quick trial run with some MA students to test out our delivery and the types of results we would get if doctors asked us for information and we answered in the different personalities, and how they would react to that.

Guidelines of scenario:

3/20/16

Today we met to look at the potential scenarios that we each had sketched out for our workshop “speed dating” exercise. We also discussed the different CUI personas that we wanted to test out with our participants and recruited people from studio to help us with a test run of the workshop tomorrow, so that we can work out any issues before trying it with our real participants.

We’ve also scheduled a workshop with a few hospital nurses for this coming Friday.

Scenarios (to be re -sketched in same style and format):

Group 1:

Questions to Consider:

• Where would you like to receive information? (before, after, or during)
• How would you like to receive the information? (device type)
• What level of privacy should the information have? (only you can hear or something that is heard aloud)
• Do you want to be able to talk to the technology our loud or have it only give you feedback as is deemed necessary for the situation?

Group 2:

Questions to Consider:

• How much control of the situation do you want, should the CUI be able to record everything?
• Do you want specific notes to be taken and then have the ability to edit them later, and would this save you time or give you more face to face patient time?
• Is it enough to just have information sent back to you in an audio form or do you need to also be able to see it to properly synthesize everything?
• Do you want the technology to anticipate what you are expecting it to ask you or only to follow your commands and requests in the situation?
• What form are you comfortable with the technology taking?

Group 3:

Questions to Consider:

• What form are you comfortable with the technology taking?
• Are you comfortable with the CUI having access to both personal EMR information, your own interactions and conversations with patients and colleagues, and the outside internet resources?
• Would you trust the outside information?
• Do you want this technology to appear in your environment outside the workplace? If so what form would it take and is this discreet enough?

Group 4:

Questions to Consider:

• How much control of the situation do you want, should the CUI be able to record everything, do you need to have “off the record” conversations?
• What form are you comfortable with the technology taking?
• Would it be helpful to know what incapacitated patients are thinking?
• Do you need to send information to doctors whom you are working with or are covering your patients while you are not on call? Would you trust CUI to do this for you or respond to command to send specific information to doctors?

Personas to experiment with:

3/15/16

Today we met to take a deeper look at the specific scenario that we want to focus our prototype in, as well as forms the prototype could take, and how we might begin to plan out how our prototyping workshops will flow. We realized that this phase of workshops will need to be very carefully planned and scripted as well as being specifically tailored to each audience that we’re working with, so that we are able to get the best feedback possible to inform our design.

Brainstorm of some initial physical forms that a CUI might take:

We realized that one of the most difficult aspects of the problem that we will need to navigate is not the CUI’s collection of information, but how and when the doctor will access that information and in what form and level of control?

The scenario that we have decided to focus in on is the situation in which doctors are taking in information from the patient and machines that monitor them, and the recording and synthesis of that information, as well as different factors that could potentially be added layers of complexity in that situation at any time.

We also took 5 minutes each to write up a very rough scenario example and we tried out the bodystorming method of design with one person being the doctor, one being the patient, and one being the CUI. This simple (and very rough trial) of the method made us realize just how carefully we will need to present and script our workshops with potential users. It also helped us to realize how certain forms or ways of getting feedback (mostly the obvious ones that we just tested for this first run) were actually very awkward, or that there was a specific point in time that was going to make a lot more sense for getting the feedback.

This was just a brief exploration of the method and we plan to do a lot more refinement of our process with it , but it was definitely very informative even in a short and fast trial such as this.

3/14/16

Phase 4

Today we began phase 4, the prototyping phase of the project. We developed a timeline and plan for the next 4 weeks so that we are able to schedule multiple workshops with participants. We are hoping to do another workshop with one of the medical students that we’ve been working with, as well as the doctor, and a group of nurses.

We spent some time during class today thinking about our opportunity space from a different angle by trying to envision the worst case scenario, so that we can take this into consideration when designing prototypes. We don’t want to accidentally go down a route that would lead to this end.

We also took some time to think about the different actors and situations or factors that impact the actors and map them out in a web of connections.

3/2/16

Phase 3 Final Presentation to Microsoft

3/1/16

We received some of our postcards back from our participants about their day to day experiences and stresses at doctors that we were able to include in our synthesis of findings for our phase 3 presentation.

2/28/16

Today we went through a bit more synthesis of all of the different information from our workshop. We outlined everything for the presentation and created a rough storyboard to display our ideas in a very conceptual framework:

We also transcribed the workshops so that we could analyze the conversations more thoroughly Workshop #2

Transcript analysis of workshops:

2/27/16

We started to put together and organize all of the different levels of our presentation today. Some of the models that we are working with are important parts of the explanation of the model of patient illness and how that is impacted by both doctor patient interaction and diagnostic frameworks.

2/23/16

From now until the phase 3 presentation we’ll be working on the synthesis of our generative research goals through several different methods such as a weighted matrix evaluation, affinity diagramming, transcript analysis, the Baty method, and later on in the process a bit of journey mapping.

Weighted Matrix:

Affinity Diagramming

2/22/16

Participatory design workshop #2 with Dr. Yoon (we previously interviewed him for the interview phase of our research)

We asked Dr. Yoon to arrange the objects from most to least used and explain why he arranged them in that way. Next, we discussed which tools were most valued for his job and why?

Aside from his phone Dr. Yoon placed a lot of value on the ultrasound machine because he is a pulmonary specialist and deals with people who have lung issues.

Then, we asked if he could add reduce or combine any of the objects, which ones would he choose and why?

Dr. Yoon chose to combine the phone ultrasound machine, pen, paper and stethoscope.

For the next activity we gave Dr. Yoon a bag of 100 beans. The beans represent all of the patient information from the day. We had him sort them by volume based on where he was getting the information from.

Next, we had him pick the most frustrating or stressful point of the information transfer and build the environment where it happens and then explain it to us.

After this we asked him to show how the environment and information transfer could be improved to decrease his stress level or work better for him.

He showed a computer system that would prioritize ICU patient needs according to his own pre-set preferences as the provider, to help him make better critical decisions.

Some of the additional tools that Dr. Yoon talked about wanting to include or modify in the different exercises are shown below:

2/19/16

We spent some time today reviewing our notes and findings from the workshop. We discussed major themes and potential directions for next steps. We also decided on the modifications for the next workshop on Monday.

For Monday’s workshop we have slightly altered the activities:

Personal Inventory

Participants are asked to empty their pockets and show the different tools that they carry with them throughout their day. We also have paper cut out images in case anyone doesn’t want to actually empty their pockets. Then we ask each participant to arrange the objects from most to least used and explain why? Next, we will discuss which tools are most valued for their job and why? Then, we will ask them if they could add reduce or combine any of the objects, which ones would they choose and why? Finally, we will ask them what are you thinking about when you use this object (in a hospital/work setting) and do you find it reliable?

Visual Representations

(each participant will be given a bag of 100 beans) The beans represent all of their patient information from the day. We will have them sort them by volume based on where they were getting the information from. Then we will have them subdivide them within the categories and explain what’s happening in each

Next, we are going to have the participant pick the most frustrating of the info gathering points/exchanges and build the environment where it happens and then explain it to us. Also, if they could make it more ideal, how would they do this?

2/17/16

Today we ran our first participatory design workshop with 4 third year medical students at the University of Pittsburgh Medical School. The workshop was a fabulous success and we were able to gain a lot of new insight into the territory and even start to envision a bit more clearly what type of concepts we might be testing in our next phases of the project.

Visual Representations

(each participant was given a bag of 100 beans) The beans represented all of their patient information from the day. We had them sort them by volume based on where they were getting the information from.

Conceptual User Journey Map/Mind Map
First we had the participants draw draw a typical path throughout the day. This could be as abstract or concrete as they wanted it to be.

Then we had the participants think about different points in time throughout the day when they communicate, thinking about where they both give and receive information, as well as the type of information and mark those points along the path.

We then had each participant explain to us what the different points along their path were.

Then we had them think about all of these points and which one is the most frustrating or stressful part to them. Using the blocks we had them build the environment that this stressful situation was happening in and act it out with the toy people.

All of the participants made their models and used the blocks in very symbolic ways to show things in the environment that weren’t necessarily tangible, but were important emotions or types of information that drive the situation and how they have to react in it. They were much more symbolic in their representations than we were expecting, which was great!

After listening to the situations of stress from the participants, we had them reconstruct their models to a more ideal situation.

At the end of the activities we had a lengthy discussion on the question of “if you could have a personal assistant to help you with any aspect of your job, what would they do?” The participants had excellent ideas and many suggestions that revolved around someone or something (they thought about technology on their own and actually preferred that for an assistant) that could help them with dense patient information gathering, sorting, and analysis, as well as many other things, which we wrote about in our summary of the workshop session.

Overall, the workshop was a great success! The participants ended up staying for an hour and a half when we had only asked them for about 45 minutes, but they were engaged and enthusiastic for the whole session and it was very informative for us.

Some images of the maps created by our participants during the user journey/mind map exercise:

2/16/16

We took some time this morning during our research methods class to test out our activities for conducting our first participatory design research workshop, which we have planned for tomorrow. We reviewed the plan with Kaki and she helped us to frame the questions in ways that would be easy to understand and that we would get the best results. Min and Kaylee were also nice enough to be our test subjects and help us do a trial run of the workshop to work out any glitches before the real one.

Conceptual User Journey Map/Mind Map
1. Draw your typical path throughout the day. This can be as abstract or concrete as you would like it to be.

2. Now think about different points in time throughout the day when you communicate, think about where you give and get information and mark those points with an X along your path.

3. Now think about the type of communication at those points, was it through the computer or talking to someone, or texting on your phone etc.? Use the colored stickers to categorize which is which and if we don’t have the specific channel listed then feel free to write down additional types of communication means.

4. Think about all of these points and which one is the most frustrating part to you. Using the blocks build the environment that this stressful situation is happening in.
– Using the toys in the model, explain what the environment is briefly and “act out” the
scenaraio that causes you stress.

5. Is there a way that you could remodel this environment or scenario that would be more ideal for your needs, could you show us this?

Visual Representations

(each participant was given a bag of 100 beans) The beans represent all of your patient information from the day, sort them by volume based on where you’re getting the information from. You can add other categories if there is something we haven’t included of if you need something more specific.

Diary Study Postcards

Later in the Day we spent some time creating a mini diary study for some of our interviewees both in Pittsburgh and contacts that we had who were out of state.

We know that doctors and medical students have a very hectic schedule so to entice them to participate in our study we created theses easy to fill out pre-addressed pre -stamped postcards where we are going to have them write about one unprepared/frustrating/stressful situation per day on and then mail them back to us to gather information in the moment rather than just from recollection.

2/14/16

Today we met as a team to plan out the different generative research workshops that we want to do with our focus group and the possible forms that this would take.

We decided on two different methods to start with. We are first going to do a mini diary study of sorts with some of our interviewees and contacts that are not located in Pittsburgh. We are planning to send them 7 postcards and to have them fill out the cards answering questions about their preparation in the hospital environment and access to information that they need each day as well as illustrating it and then having them mail the pre-stamped postcards back to us. We are designing this activity to only take 5 or 10 minutes/day so that we can gather as much information as possible while respecting the limited time that people in this line of work would have to give us.

For our second activity we are getting together a group of medical students and residents who (thanks to some contacts from our prior interviews) and are planning to do a workshop about their activities in the hospital and stress points, triggers, and important touchpoints in their daily routines. We are also hoping to include some personal inventory studies and cognitive mapping in this workshop.

2/12/16

Phase 3

Today we met to recap all of our feedback from the presentation and make a plan for moving forward in phase 3. We decided that we would meet this weekend after we had each had some time to review the generative methods that Bruce has been teaching us in research methods class. We will then devise the different workshops that we might want to do and decide how we will structure them and what we want to learn from them. We will also be working to recruit doctors that would be able to participate in the workshops for us.

2/8/16–2/10/16

Phase 2 — Final Presentation

Some feedback that we received:

• overall positive, Kevin liked the direction and seemed to identify with the need very much
• This is a scenario that could be later mapped to lots of different areas beyond just the hospital setting, so many people could relate to/use this concept
• The argument for human computer symbiosis here is very strong because of the high stakes nature of the situation and environment in the hospital
• Be careful about the number of things that we listed for barriers and concerns (Irina commented that she wasn’t sure how we were going to cover all of these concerns, until once we had stated our example scenario which made our intent more clear)
• Think about the different technologies that this could start to be applied with and what it might look like a bit more
• A helpful book for us to look at would be “Seeing what others don’t — the remarkable ways we get information” by Gary Klein
• Make sure to consider where the interaction with the technology is taking place. Is it ok for it to be in front of patients or not?
• Will this technology affect patient anxiety at all in either good or bad ways?
• Consider HIPPA and patient privacy
• ** It is very important to make sure that you are demonstrating the connection between the research and the problem space **
• make sure the design fits the brief and that your situation makes a good case for the need of the CUI in this space.

To celebrate being 2/5ths complete with our Microsoft project we all went out for pizza at The Porch up in Schenley Park

2/7/16

We spent a lot of time this weekend working through our presentation and synthesizing all of our research information into a cohesive narrative. It took several different iterations to find a way to diagrammatically show where we have found insights and directional shifts in where our interventions points will be.

2/4/16

This morning we went over to the Language Technologies Institute at CMU and talked to a few professors who specialize in Conversational User Interface research to learn a bit more about how the technology works and where there are going to be limitations or areas of potential growth. They were also kind enough to give us a Windows phone that they had so that we could try out the Cortana voice interface and learn a bit more about it.

Interview with Professors Maxine Eskenazi and Alan Black of the CMU Language Technologies Institute

Later in the day we went to UPMC Presbyterian Hospital to interview a fellow in internal medicine there to understand how he sees wellness in the workplace and learn more about his work environment and the daily life of a doctor in the ICU of the hospital. He also took us on a tour of the hospital, so that we could get a better sense of the space that we’re working within.

Interview with Dr. Yoon

After our hospital tour we went out for another team dinner in Oakland to recap all of the day’s interviews and information.

2/3/16

We took some more time during studio today to continue our exercise in mapping out connections and intervention points:

2/2/16

We spent some time after our research class reviewing different research methods as well as mapping out all of our ideas from different angles to see where we could find connections and intervention points:

2/1/16

We are continuing our informational interviews to learn more about problem spaces in the hospital environment relating to the wellness of caregivers. today we interviewed Tracey who is the Vice President of Pediatric Services at Hasbro Children’s Hospital in Providence, RI.

Interview with Tracey

We were also able to interview Chrissy, an Administrative Coordinator in the division of emergency medicine at Boston Children’s Hospital in Boston, MA.

Interview with Chrissy

and finally we took some time to talk Katie who is a Clinical Pharmacist at Brigham and Women’s Hospital in Boston, MA.

Interview with Katie

We also took some time today to go through what we’ve learned from our interviews so far and begin to try and synthesize a bit more. We used a Rose, Bud, Thorn method of categorizing our findings from interviews as well as beginning to map them to see how they are all interconnected and in what ways they impact each other.

I also took some time today to do a 1 hour fly on the wall observation of the CMU University Health Services. Below you can see a diagram of the space and how people were using it as well as some observations that I had while I was observing the environment as well as how I felt being a part of the space.

• Quiet space except for a high pitched hum coming from the lights
• Everyone waiting is looking at their phones or has their laptop out with headphones in
• There was one person at the desk and several nurses that were coming back and forth from the back
• When the nurse came out to call a new patient she only looked into the sick waiting room and not the healthy one which was a separate room and made it difficult for people waiting there to hear her or know she was ready to take them
• The space has a slightly sterile hospital environment smell
• the waiting rooms were fairly open and had wide corridors and passageways leading to them
• No one acknowledges you when you walk in
• Afternoon TV shows were playing in the sick waiting room and you could hear the sounds of the characters and traffic on the TV show
• The nurse behind the desk could be heard moving papers and file drawers around
• Students come in and use self check in carols instead of checking in a the main desk
• The tack board on the wall has posters about quitting smoking, getting healthy, eating well, and safe alcohol consumption
• There was a comments and suggestions box in the healthy waiting room
• People can be seen outside standing and waiting for the bus stop by the window
• The environment is warm and comfortable, the creamy neutral tones of the wall and laminate wood floor give a more relaxed and comfortable feel to the space
• Fluorescent lighting in the space is somewhat offputting

1/31/16

Today we decided to go out for a team lunch in Squirrel Hill before going to our next interview. We went to Curry on Murray and took some time to test out the set of cards for team building exercises.

Our interview today was with another medical student in her 3rd year named Claire who is studying at the University of Pittsburgh Medical Center.

Interview with Claire

Since Claire was located in Pittsburgh we were able to spend a bit more time talking to her at her home.

1/30/16

In an effort to learn a bit more about CUI and gesture and these concepts that we are supposed to be designing for we had a studio game night to try out the Xbox that Peter and Bruce got for us with a few gesture and voice command based games such as DDR and charades.

1/29/16

After Reviewing our progress with Bruce and Peter on Wednesday we decided that we need to spend some more time looking over a few questions to decide how we want to define the project as we are moving forward:

What is a CUI?

A CUI is a back and forth between the human and the computer. In this instance the computer needs to not only be aware but also tounderstand the context of the situation. It should be able to both process and comprehend what is going on. This requires an understanding of language. We looked at what language is really? We think it can be more than just spoken and the computer needs to understand and process these things. Language can be in many forms, some of which are highlighted below:

• Body (unconsciously communicated ie facial expression)
• Spoken
• Written
• Gestural (deliberate motion)
• Visual
• Auditory (ie environmental sounds or orchestra)

A CUI is Aware -> Back and Forth -> Shared Language(s) -> Understanding

Something to think about… What level of overlap is needed between shared language and understanding to function at an acceptable level?

What does symbiosis mean to us in this context?

After a lot of discussion we decided that in our view human computer symbiosis is a mutually dependent evolution of learning and information sharing. Symbiosis is a give and take relationship that requires cooperation on both sides. We saw this as more of a spiraling relationship rather than a cyclical one because we can program the computers/ CUI to know how to learn and help us with information, but it can’t do this unless we give it access to our data so that it in turn can learn from that data and give back better and more helpful information and interaction as a CUI. It’s a slowly building and growing process that takes time.

What is Wellness?

We’ve been asking this question to many of our interviewees so far to see what both workers and employers are thinking about and taking into consideration when they talk about personal wellness and how it can effect others. We have an initial brainstorm, but we decided that we want to do a bit more research and reading on this question before we can fully answer it.

Some of our ideas below:

All of these areas are going to change and evolve as we talk to more people and learn more from our research, but we wanted to give ourselves a good collective understanding of the concepts that we are working with and how we understand them.

1/28/16

Today we had an interview with another medical resident, named Sejal who is in her final year of residency at Case Western Medical Center.

Interview with Sejal

1/27/16

Today we had our second interview with an outpatient research coordinator, named Nikki who works with Lymphoma and Myeloma patients at Massachusetts General Hospital.

Interview with Nikki

1/26/16

After reviewing several different research methods in class today we narrowed down a list that we think will work well for our project:

• Background research reading
• Interviews — storytelling
• Tour — observation
• Shadowing
• Design ethnography — volunteering — full participant
• Cultural probe — survey or kit mailed

We also started scheduling our preliminary interviews and formulating our research questions:

Interview Schedule

Our first Interview was today with a friend of Lisa’s named Leslie who is a medical student in her fourth year at Georgetown Medical Center.

Interview with Leslie

1/25/16

In class today we were able to regroup after a weekend of research and talk to Peter and Bruce about our process and personal contacts who work in hospitals who we had started contacting about interviews. They also suggested that we get in touch with the reference librarians at CMU to find a few books about wellness and research into wellness to understand what exactly that means. We’ve also been reading several academic papers and watching TED talks suggested by Peter.

• Multiple Group Coordination in Complex and Dynamic Task Environments: Interruptions, Coping Mechanisms, and Technology Recommendations http://www.tandfonline.com/doi/abs/10.2753/MIS0742-1222250105
• Designing Information Hotspots for the Surgical Suite: How Architecture, Artifacts, and People’s Behavior Converge to Support Corridination http://reports-archive.adm.cs.cmu.edu/anon/hcii/CMU-HCII-09-100.pdf
• https://www.google.com/search?q=atul+gawande+ted+youtube&ie=utf-8&oe=utf-8

Additionally we took out the books that Peter had suggested earlier and are taking turns summarizing them:

• “Boys in White — Student Culture in Medical School” — Summary
• “Social Organization of Medical Work”
• “Collaborative Health Care: A Family Oriented Model” — Summary
• and a few others that we found in the same interest area

Some relevant article in the area that we’ve found are:

• http://www.nytimes.com/2016/01/12/upshot/silence-is-the-enemy-for-doctors-who-have-depression.html?smprod=nytcore-iphone&smid=nytcore-iphone-share
• http://www.medscape.com/viewarticle/851028
• http://www.nytimes.com/2010/10/07/health/views/07chen.html?mtrref=well.blogs.nytimes.com&gwh=628E5E5913B2CEE780CFD308A3D5B9EF&gwt=pay&_r=0
• http://www.nytimes.com/2016/01/25/business/employee-wellness-programs-use-carrots-and-increasingly-sticks.html?smprod=nytcore-iphone&smid=nytcore-iphone-share
• http://www.ama-assn.org/ama/ama-wire/post/one-program-achieved-resident-wellness-work-life-balance
• http://www.sciencedirect.com/science/article/pii/S0002961012005430

1/22/16

Literature that we are beginning to review for background research:

• Silence is the Enemy for Doctors with Depression
• Multiple Group Coordination in Complex and Dynamic Task Environments
• Designing Information Hotspots for the Surgical Suite
• Social Organization of Medical Work
• Boys in White: Student Culture in Medical School
• Better
• Complication
• Checklist Manifesto

1/21/16

Today we met during our research methods class. We talked about several different research strategies to begin thinking about.

Outline

We then began brainstorming as a team to think about different assumptions that we already hold about the hospital as a workplace that might help us to frame some of our upcoming research questions.

Preliminary Brainstorm of Background Assumptions

Next steps over the weekend:

1. Literature and background research example review (each team member posts relevant articles to our slack channel as well as reviewing prior research shared with us by Peter)
2. Contact direct contacts to see their availability and set up interview times for next week
3. Formulate preliminary research questions to ask interviewees and decide which interview/research methods will be most effective to use with each potential interviewee

1/20/16

Stage 1: territory definition presentation: Territory Map

Commentary From Critique:

• Interesting direction and territory
• Why would the CUI be best in this space and not some other type of interface?
• Think about what this product might actually look like
• Consider ethics and personal privacy issues with peoples’ wellbeing
• Review past research so that you’re not replicating things that have already been done
• Think about the fact that hospitals often don’t want recorded evidence of behavior to be around permanently for fear of lawsuits

1/19/16

Notes and comments to consider from yesterday’s in class preview:

• Get specific about the exact area we want to look at
• Make a list of people that we will have access to and think about both advocates and adversaries
• Reconsider certain elements of the graphic design of the territory map and presentation
• Why do we want to look at a non traditional office?
• Be sure to clarify workplace + wellness vs. a space that is providing wellness to clients ie in healthcare settings

Things that we want to keep in mind when scoping down our territory:

• What does it look like for someone who’s mobile on the job?
• What are the spaces that might fit well within the interests of Microsoft
• Interaction space — where is the opportunity?

We decided to make as comprehensive a list as we could of professions that work outside of a regular office building:

After much discussion and narrowing down the fields based on our interests and where we found the most opportunity for innovation and alignment with the project brief we decided to look at the hospital as the workplace.

We split up the different sections of the presentation and revised as necessary for our final presentation to Microsoft tomorrow.

1/18/16

Proposed Team Meeting Agenda

1. Share individual territory maps and integrate into one map
2. Complete presentation components (10 slides) including:

• The problem opportunity
• Territory stakeholders
• How we are planning to explore the territory
• Research methods we want to use
• Overall summary including key ideas and next steps

1. Define graphic style of the presentation
2. work to get to maybe 75–80% done with presentation before feedback session on Tuesday
3. If there’s time brainstorm the ideas of: what are the different spaces people consider a workplace? how do people communicate with each other? how do machines communicate with each other? What are the benefits for employers of understanding and caring about wellness of their employees, and how might CUI begin to be integrated in a broad sense?

Initial explorations of territory areas that our team is interested in:

After reviewing what each of us had thought about individually we began to integrate all of our ideas into one territory map and filter our ideas into a more cohesive problem space. We also spent a lot of time considering how we could make sure to integrate the strengths of the CUI into the territory maps as well as discussing the graphic style that would best represent these ideas.

beginning to consider the type of structure that our territory map will take on:

Focus areas of the territory map that encompass the workplace outside of a traditional office:

• Teaching
• Public Safety (police and firefighters)
• Hospital (doctors and nurses)
• Construction (including architecture and engineering)

1/17/16

I’ve been doing a lot more background research into the realm of conversational user interface in the workplace and what exists now to try and think of what this might be in the future, some relevant research that I’ve found:

Conversational AI: The User Interface of the Future?

• natural language = communicating with each other = access to services
• humans learn the computer interface to communicate with technology, but now technology is using the human NLP system to communicate with us as well
• dialogue robots are being designed to be specialized about any topic
• It’s a slow building process starting with human and machine working together and then the machine gradually learning to take over

How It Works — x.ai

• X.ai’s Amy schedules meetings for you as if she were a real personal assistant, and most people can’t tell the difference(again with the female persona… this is something that needs to be delved a bit deeper into…why?)

It’s a Baby Boy — Andrew Ingram — x.ai

Meet Luka!

• Luka recommends restaurants
• more simplistic easier version of yelp
• more personal — availabile for iphone

Beyond the GUI: It’s Time for a Conversational User Interface

10 Ways Conversational UI Will Change the Way We Live

Chatbots: The Rise of the Conversational User Interface

Workplace bots are accelerating, are you ready?

The Coming Voice Recognition Revolution — Workplace Strategy and Design — architecture and design

Workplace Strategy and Design — architecture and design

http://www.gensleron.com/work/2015/8/7/thank-you-for-sharing-how-social-media-is-reshaping-work.html

By 2020 it is estimated that 40 percent of the whole workforce will be independent contractors, or ‘micro workers’. They will typically work from co-working spaces, which are — no surprise — also founded on the principles of sharing. Our research on-co-working shows that 87 percent of people working in them are there primarily to connect with other people of the same mindset. They want to create new products, knowledge and projects, and they understand that these are collective endeavors founded on in-person collaboration with others and not alone.

Some areas I’ve been thinking about which I’d like to brainstorm more with my team are where do people work? and how do they communicate? What types of everyday conversations do coworkers have that could effect their lives and work habits? Some initial thoughts that I had are:

Where do people work?

• private offices
• open plan offices
• from home
• from coffee shops
• on airplanes
• in cars
• on the train
• field work in remote locations
• construction sites
• dinner/lunch meetings
• on the golf course
• over drinks after work
• networking events

How do people communicate at work?

• talking to one another
• instant messaging
• e-mails
• interoffice phone calls
• memos
• shared understanding of organizational systems
• shared documents and files
• meetings in person
• remote meetings via call in (go to meetings)
• facebook/social media
• text messaging
• eavesdropping
• leaving written notes

What conversations do coworkers have?

• how was your weekend
• frustrations with the boss
• frustrations with the client
• coordination on a project
• can you help me learn how to do XYZ skill
• wedding planning
• buying houses
• prepping for a baby
• visiting with family
• going on vacation
• stress about the commute
• how are the kids
• where are they going to school next year?
• I’m sick
• I have to plan/attend XYZ life event for a friend
• someone died
• someone has a date
• going out for drinks after work
• what did you make for lunch?
• shared diet tips
• selling crafts for a side business
• buying a new car
• My family member is sick and I need to care for them
• I’m taking classes at night and I have homework
• I’m studying for my licensing exam

1/15/16

After evaluating the ideas that we had written on the post-its and trying to group them a bit further within each category we realized we did not have a concrete idea/space that we were moving towards and these themes didn’t have a particular space that they were being applied to in a way that would help us scope and define our territory.

We began to talk more about the need to step back and define the space we wanted to apply these themes to as well as how both the themes and space would map to the requirements of the project brief. We started to discuss the idea of the workplace, something that’s very familiar to the average person. Most people spend between 40–60 hours a week in their workplace and with their co workers. We began to wonder how might the CUI realm be applied to the care taking of the employees of a company and interactions between people and the space and work. This could potentially have many benefits for both employers and employees.

We also reviewed several example territory maps during this session and defined a graphic style/direction that we all agreed on for the project. We then decided to each individually take the major themes that we defined and the realm of the workspace and work out our own rough draft territory maps

1/14/16

Team Meeting Agenda

1. Share new territory ideas and discuss
2. Pick territory
3. Begin rough mapping of territory map features

Important things to think about when defining our territory:

• Are you making any limiting assumptions or following a path of least resistance?
• Alternate between fall line thinking and hopping off
• Broad to narrow
• Divergent to convergent
• Has your area been researched heavily already?
• Will it be hard to contribute something new?
• Are there strong competitors?
• Do you have easy access to stakeholders?
• Does your team have expertise in this area?
• Are you staying true to the prompt?
• Can you get outside input?

We did a bit of initial brainstorming expanding on territories that we had previously looked at

Then we began to look at where our interests overlapped the most

The larger intersections that we found were in the realm of care giving or care taking, social infrastructure, access, and wellness. We decided to take a closer look at these areas and define some ideas within those space through affinity mapping:

Access

• To help
• Independence
• To specific people
• To experiences
• Remote work
• To data
• Makes you smarter, efficient
• Talk with other than humans
• Access more with different countries and cultures

Access/Care taking/giving

• Letting someone know you’re working on something for them
• Handing off between generations
• Communicating the situation to others
• Knowing when the one in your care is ok

Care taking/giving

• More patients with because of accessibility
• Seniors
• Kids. growth in kids entertainment
• Old vs. young vs. middle age
• Family unit
• Connected more with families and friends
• Stress
• Care taking, dogs, plants
• What to do after loss
• Loneliness
• Guilt
• Sharing information
• Burdens on others
• Connected with families and friends
• Burdens on yourself
• Shortcomings that are not health related
• Lowers the barrier between generations
• Disease
• Sharing the burden
• Knowing that there’s alternate options

Wellness

• Confidence
• Prevent disease
• Digital safety
• Safety in the physical world
• Financial safety
• Big data, more accurate information
• Scheduling appointments
• Share your health status
• Able to learn more about your body
• Mental wellness
• Having to use tech you don’t understand for….
• Health awareness of yourself and others
• Busy people taking managing their lives/work/jobs
• Access to knowledge, smarter people live longer
• Renewing prescriptions
• Compliance
• Of the space around you, how do your actions effect the environment — (physical, digital, auditory)

Societal Infrastructure

• Explore, visit places you have never been before
• More accessible information
• Automated machines
• Person to person expected interactions
• Structuring the home for aging
• Ride sharing systems
• Existing systems and standards of care
• Family unit
• Friend groups
• Public transportation
• Services we interact with ie. restaurants or coffee shops

Intersections between all themes

• Universal design
• Record access at the waiting room
• Independence
• Understanding treatments and considering options
• Community health center
• Campus health center
• Self care for caregivers
• Understanding what factors are affecting health and well being
• Communicating (pain, discomfort, feelings)

1/13/16

Important points from lecture

• Make sure to focus on a problem space that human machine symbiosis can be applied to
• Consider the future and think a few years out
• Think about systems, what goes well and what doesn’t and why?
• What are the largest market sectors for business?
• Think about who, what, where, when, why, and to what end?
• Consider trends, stakeholders, and themes
• Where is this technology being used already or not and why?

Territory Mapping Brainstorm

During our brainstorm we looked at a range of topic areas starting from both a micro and a macro scale and had a lot of discussion on which would make more sense to start building out our territory map from. We were gravitating toward something in the realm of care giving. After spending the second half of class brainstorming territory areas to explore, we decided that we needed to take some time individually to explore our full range of options and areas and decide what we were most interested in before we commit to an area of focus. We will reconvene tomorrow to discuss our findings and decide what our topic space/territory will be.

We also discussed the project brief and things we’ll need to consider for the best ways to design to it’s requirements. We will look back at this again to see how these ideas and technological context might fit in to our more defined territory space. Some things from the brainstorm that we need to consider in regards to this are:

• Design must span the digital and physical environment
• Interpret intent
• Fluid transaction
• Shows value and differentiation of CUI
• Human computer symbiotic relationship
• Demonstrate foundational elements of CUI

Things to remember for our presentation:

• Territory map should tell a story — what is the presentation about?
• Clearly define what the territory you are going to explore is
• Presentation should be approximately 10 slides at 1 minute each
• In the presentation make sure to discuss:
• The problem opportunity
• Territory stakeholders
• How we are planning to explore the territory
• Research methods we want to use
• Overall summary including key ideas and next steps

1/12/16

Team Dinner at BRGR

We had a team dinner at BRGR and discussed our goals for the project as well as our strengths and weakness in design skills and what we would like to focus on learning throughout the semester. We also created our team contract stating all of our goals, expectations, and team framework for the semester.

Team Contract