Process overview
Part 2: Handing over to a human
Our mission:
There are times when people need medical interventions for their mental wellbeing. In those cases, in-person interactions with psychiatrists are necessary. Your AI counseling service partners with psychiatrists in the referral of patients who the AI deems may need a medical intervention. Your client will be leveraging the information the system has on the client through their AI-led counselling sessions to provide a customized check-in interface and doctor-facing web portal.
Team: Ema Karavdic, Matt Prindible, Khushi Shah, and Jiasi T.
01 — Wednesday, November 14
Today was the first day for the second part of our AI project challenge. Based upon our experiences in the first part of the project (and the fact that Thanksgiving was coming up), we decided to map out a project timeline as our first task. This also helped us uncover initial questions about expectations regarding the deliverables that we could immediately ask to determine the amount of work we would have to do. It also highlighted some dependencies so we could pull forward some of the work we initially thought we’d be tackling later.
The next part we look into in the class was the storyboard. We started with an overarching process that we would need to cover end to end to see what it would look like. We then dove into each of the sections to unpack the narrative we wanted to present and the information points we may need to consider.
The first part of our storyboard was about the initial conversation that Mary would have with Joan, explaining that she wanted to see a human psychologist rather than continue speaking to Joan. The second piece was assigning a new counselor that would suit Mary. Third, we had the kiosk experience and finally we had the human psychologist view where they could see the data that had been created between Mary and Joan.
After going through each of the sections, we realized that the toughest part to design was probably going to be the psychologist view as it was going to be the most complex. We have decided that this is the one we are going to tackle first when we meet up on Saturday. Given its complexity, we also thought this would be best done in a group rather than individually, whereas some of the other processes were more simple and could be done over the Thanksgiving break when we weren’t able to all be together.
Next steps for us are to work towards creating concepts and presenting our storyboard next Monday for critique. After Monday, we should have a single concept for each set of screens to dive into and add more detail.
02 — Saturday, November 17
Today we decided to get into the detailed steps of our high level storyboard and begin to tackle the most complicated part of our design. We identified this as being the psychologist screen view as we would need to consider:
- What was important information for the psychologist to see
- Identify the kind of data being brought in from the app
- Establish information hierarchies
- Determine visual representation of information and style guide
We identified two psychologist screens that would be important to visualize for the second part of the narrative. They are the psychologist dashboard and the patient profile. We decided to start with the patient profile as it would have the most granular visualizations and information. After deciding on all the information needed at that level, we could then abstract it to the dashboard to only show the most important information.
To begin, we decided to list out all the information that we would need to show on the patient profile. This started to shake out some categories of information that we would need to create in the patient profile. Some of the information identified includes:
- General patient information
- AI history — area where all the data from the AI would exist
- Notes from sessions
- Medical information
- Documents/homework — could also include any prescriptions
We then broke up and allocated ourselves half an hour to individually sketch out what we thought this could actually look like. This allowed us each to collect our thoughts, do research and reflect. We came back together and shared our initial sketches.
They had many similarities in what was important to show and hierarchically quite similar. This made it easy to begin to bring our designs together in a cohesive set.
Bringing the designs together forced us to have a lot of important conversations and make a wide range of decisions about the direction we wanted our design to go in. It allowed us to map out a variety of options to show to other students as well as Q and Daphne in order to get feedback on some issues we weren’t too certain on. Sometimes, we also felt we would just need to see it in high fidelity to make a decision.
Next, we walked through all the steps in the process for the initial conversation with Joan and the kiosk. This should set us up well for Monday when we begin to sketch out these designs.
03 — Monday, November 19
So far in the project, our timeline has been working very well to keep us on track and hold us to the micro-deliverables that we need to produce in order to finish the project on time (and without many late nights).
Today we knew that we needed to focus on getting the style guide together in Sketch so that each of us could individually work on transforming our sketches into high fidelity designs during the Thanksgiving break. The aim is to have this to show when we get back next Monday and have peer critique. That way we can have critique on style, hierarchy and content of our designs. Khushi has had a lot of experience creating style guides while Jiasi is very interested in learning how to create them. Together, they took on the creation of our library in Sketch.
Meanwhile, Matt and I sketched out the processes that we had discussed on Saturday for the initial discussion with Joan and the kiosk.
Matt then wrote out a basic script to see how long it would be to run through the conversation that we had imagined Joan and Mary having about Mary moving to a human counsellor. It turned out to be quite long so that initiated a discussion on what was most important to show in the final video. We will definitely need to storyboard that soon so that we can include actual filming in our final video.
We also spoke to Q and Daphne to get some feedback on our designs and process from them.
Q:
- Think about how you might facilitate comparison with the data visualization: instead of having the doctor remember the last data point, find a new data point, and compare the two — what tool can be built to help do this?
- Appropriate hierarchy will be important to evaluate once the designs are in high fidelity form
- What information can be hidden and revealed contextually based on user interactions. Does everything always need to be visible?
- Look into surfacing more Joan data into the dashboard/Mary’s profile page initially so that the doctor doesn’t have to dig to get it. This is a key part of the assignment
Daphne
- On the high-level list of patient schedule, do they always need to see that someone’s used an AI counselor? Look into what is important to surface on the card
- Clear distinction between information received from the AI and then information that might be captured during regular sessions — look into the hierarchy and positioning of the groupings in the psychologist view
- Clarify the data that’s actually being mapped in emotional range — will need to have axis labelled in high fidelity design
- Will need to actually engage more with the kiosk rather than going ahead with the mobile only idea — important for accessibility and if someone is new to the actual psychology service system so doesn’t have the app
04 — Sunday, November 25
This session was predominantly to catch up after the Thanksgiving break and see where we were in terms of designs. Our aim was to have a first draft of all the high fidelity screens created so that we could get feedback on the functionality, content and the implemented style guide during our peer critique on Monday.
Khushi developed an initial style guide during the break to help us maintain a level of consistency in our designs while we worked separately. We will continue to iterate on this with feedback and also as our screens develop.
I also completed a first pass at the high fidelity mobile screens based on the sketches that Matt and I made prior to the break. This should give us a good foundation to start showing some motion.
Finally, we had a first pass at the high fidelity desktop screens. We found it challenging to determine what would be important to show as there was no real time allocated to doing user research. We conducted an interview with a practicing CBT psychologist, desktop research and made some assumptions. Of course, it is hard to generalize based off of this information but we thought it was important to at least have a solid starting point and get feedback from the class.
05 — Monday, November 26
Today’s class was a peer critique and was a great way to see the thinking that everyone else had done. We received some good feedback on the designs that we had done. It was helpful that we had the initial designs for all the major screens, setting up for some good progress in the coming days.
Feedback received:
Joan/AI history
- Include analysis on how effective CBT was
- Joan doesn’t need to be as prominent as it isn’t about her
- Joan isn’t a clinical psychologist — have to be careful of the language we use
Graphs
- Think about the types of data available from Joan from the consented data in the first part of the process
- Could display the range of emotions for each individual sessions
- Graph could be more visually interesting — looking into how this could happen
- Are there any other factors that could be interesting to display other than just length of session
General
- Add a goal/planning section as this is important in the counseling process
- Font size is a bit small
Before we add motion and the necessary transitions, we’re going to re-group as a team and spend some more time adding meaningful content based on the feedback and ideas we got from the critique.
06 — Tuesday, November 27
During our team meeting, we decided to re-visit our initial plan for finishing the project to see what needed to happen over the next week. We started by noting down all the tasks that needed to be done in order to finish the project and worked back from the final.
Next, we revisited the data surfaced on the dashboard and client screens based on our follow up expert interview and the feedback we received in the peer critique. We felt it was especially important to get the feedback from the critique in so that we could test our solutions with the class during our interim presentation.
In our screen revision, we focused on refining the content that would be brought in from the app to connect the two experiences. We also refined the navigation and created a clearer separation between AI and psychologist information.
Finally, we got our first draft of the mobile screens in motion. We used the old version of Joan to give an indication of how she will work, size and positioning. We hope to get feedback on the flow and interactions we represent in the interim presentation tomorrow.
07 — Wednesday, November 28
We presented our progress at the interim review to get feedback on mobile motion and the updates on the screens that we had done. Overall, we felt that the feedback was mostly about the details, which was fantastic as this meant that our overall structure was solid. We will be splitting up the work to apply the edits that we agreed were important.
Feedback was as follows:
App
- Summary of consented actions at the end
- Consent cards: undo gesture
- Consent cards: how does Mary know which side is accept or decline consent?
- Blur/lower opacity of Joan form when Mary looks into details — currently overlapping which is distracting
- Animate the accept/decline consent
- Include how many steps are in the process of booking an appointment with a doctor so that client knows where they are in the process
Psych screens
Dashboard
- Bump down shadows on card. Remove shadows
- Kill news x 2
- ‘Appointment’ doesn’t need to be repeated each time on the schedule
- Use voice to show doctor what they need at a specific time rather than showing everything at once
- Remove image/name of doctor — don’t need to see themselves
- Add icon to see who has used the app/AI. Put this in Mary’s profile header
Navigation
- Side panel reduced — takes up a lot of space but not much value. Add a few more people, reduce to narrow icons with first name and first letter of surname
- If keeping side panel, remove ‘Mary’ on the dashboard view
- Secondary navigation: ‘medical report’ should be ‘name of org report’
Goals
- Allow psych to edit the diagnosis/client personal details x 3
- Mary profile diagnosis to have small icon to indicate that it is AI generated. Edit button next to it. Once edited, the icon is removed
AI history
- Assessments could be in accordions rather than behind more buttons
Graph
- Could map frequency of sessions for graph
- Graph seems misleading as divisions are equal visually but the time intervals are not
Potential visualizations
- View themes/concerns over time
Therapist vs AI
- What is AI/therapist generated — is there a way to distinguish the visualization? Icon as before. Edit button to accompany
Visual design
- Can we push this further? Limited colors are good but could create greater visual connection between app and doctor system
- Use color to call out a primary piece of information on the dashboard — nothing jumps out at the moment
08 — Saturday, December 1
The primary aim today was to ensure that all the inputs were created for the final video so that we had enough time over the next couple of days to put it all together for the presentation.
The next two priorities were to implement the feedback we received and refine the visual form of both the screens and Joan. We created a variety of options to test different approaches.
Joan’s form has also had a significant evolution since her inception during part 1 of the project. We felt that Joan was too complicated, confusing the viewer rather than creating meaningful visuals. However, it was still a good foundation for the iterations that followed.
We concluded that the key feature that made Joan feel complicated was the mixing of the three allocated colors in the center of the form. We decided to experiment by removing the center and reducing the layers from three to two. The result was interim Joan. While this was a large improvement, the white space was too dominant and didn’t allow for the movement and colors to come out effectively.
We decided to experiment with radial gradients to see whether this could take the edge off of the aperture in interim Joan. Q also suggested that we try to use the gradients we already had in our design of Joan for form rather than style.
09 — Sunday, December 2
The main aim for Sunday was to start putting together all the assets that we had created on Saturday. We also wanted to go through a variety of visual design explorations to experiment with color, graphs and fonts.
It was very challenging to decide which colors worked best and how to use them most effectively. Some looked great as swatches but then just didn’t work well once we mocked them on the screen. When talking to Q, he suggested that the issue might be about balance of brightness and weight that each color combination had.
We also went for blue as our primary action color as we felt that this was quite common in health related branding that we had previously seen. We could leverage some of the contextual knowledge and feeling that this color naturally evoked. We decided to go for a deeper blue with a higher grey content as it was more neutral but eye catching.
After our interim presentation, we felt that we needed to introduce an accent color to make the screens more visually engaging and draw attention to elements that weren’t action points but still important to quickly note. One of the feedback points from the interim presentation was that people liked our very simple color scheme so we didn’t want to overload with too many accent colors. As a result, we decided to only go with one accent color. Our final decision was teal as we felt that the combination with the darker blue worked best. It created a nice contrast but didn’t compete for attention as much as some of the other options we explored.
During our interim presentation, we realized that we hadn’t resolved the way that we were going to visualize the data that we were bringing into the doctor dashboard from the app. We tried a variety of options and tested them with the class to get some feedback on what was working, especially in the context of our screens.
Eventually, given our color palette and general style is very clean, we decided to go for a clean graphical style as well. We introduced color into the screens through the graphs but didn’t add any additional gradients or tertiary colors in order to keep the focus on the functionality.
Finally, we also needed to come up with a name for our AI/psychologist system and consequently, branding.
After some exploration, we decided on Saluto as our system name. It comes from the term salutogenesis, that is a practice in medical sociology focused on supporting growth in human health and wellbeing rather than the causes of disease. It is also particularly focused on coping with stress and anxiety. We felt that this was very close to the intentions we had with Joan with Joan’s personality and generally with the offering of an AI service.
After much consideration, we decided not to design a logo, but use the font in a thoughtful manner as our logo. The service had both human and AI elements, so focusing on one or the other would be a misrepresentation of what was actually offered. Further, trying to meaningfully represent mental health in a logo would also be very difficult and fraught with interpretation issues.
As a result, we explored a series of fonts to determine what would be best. We decided the a sans-serif would be optimal as we felt that it fit our clean screen style and also depict the modernity of the system, particularly because it used an intelligent system and data transfers.
Next, we felt that a bolder font could act more as a logo than a lighter one, drawing attention and making a statement.
Finally, we wanted our font to feel approachable, clean but not too clinical and unique. As a result, we went for the top left font — Monserrat Alternates.
10 & 11— Monday & Tuesday, December 3 and 4
We spent these two days bringing together the rest of our content. It has been quite a challenge to align the video, audio and animations of the screen design that we’ve created. All the moving parts are necessary to create a cohesive story of the system that we’ve designed.
One of the key learnings we had was that Principle was exceptionally difficult to use if labels were not assigned to elements of designs in Sketch. Given that we were working across different screens and files, it was always going to be hard to create consistency. However, with some tips prior to commencing the build out of our screens, we could have paid more attention and built out a system that would make it easier for us when it came to animation.
As a reflection point in the future, we would like to explore how this process could have been set up to more seamlessly integrate the pieces together across the different technologies we used to bring this to life. It would definitely have allowed us to spend more time working on the micro-animations rather than tackling the complexities of the tools we were using.
12 — Wednesday, December 5
It’s presentation day! It was great to see how far the different teams had come from the interim project.
A few reflections after our presentation:
- We felt that we could push our major value proposition of combining voice and visual interface to the doctors screens if done again
- The thinking behind the hierarchy and content of the screens is very strong for us
- Our video could be more polished by aligning the video more closely to the audio so that it feels more realistic
- Update to the initial context setting piece so that it doesn’t sound like Mary doesn’t trust Joan
That’s all, folks!
