Using Midjourney to Illustrate UX Task Scenarios
Enhancing Stakeholder Understanding with AI-Generated Characters
Motivated by my recent volunteer duties at Cedars-Sinai Medical Center, I took the opportunity to create a mobile app concept for Cedars-Sinai Transforming Care at Bedside (TCAB) volunteers to log their shift tasks. (TCAB volunteers currently record their shift tasks on a paper form.) When it came time to present the concept, I felt illustrating various task scenarios with an imaginary volunteer in a side-by-side view with the corresponding app screens would tell a clearer story to healthcare professionals than just the app screens alone.
Coincidentally, the AI image creation service Midjourney had recently announced a new Character Reference feature, allowing users to generate consistent characters across different scenes. This caught my attention as a potential solution for creating cohesive task scenario illustrations. I decided to use Midjourney for this project, leveraging its new capability to maintain consistency in my volunteer character across various task visualizations, making it easier for non-designers to understand and engage with the proposed solution.
I found the process successful, and I’d like to walk you through it.
Getting Started with Midjourney
A quick note: This is not a comprehensive Midjourney tutorial. This post assumes a basic familiarity with Midjourney: subscribing to the service, interacting through Discord and/or the web interface, and executing basic commands. There are numerous tutorials online for getting up and running, including Midjourney’s own reference materials which are excellent: Midjourney Quick Start
Creating the Reference Character
A typical Cedars-Sinai TCAB volunteer is either in college or just out of college and working toward a career in healthcare. TCAB volunteers also wear a specific uniform on shift — black scrubs and a teal colored jacket identifying them as a hospital volunteer. To create the reference character I wanted to use for the different shift scenarios for my TCAB volunteer, I used this simple Midjourney prompt:
/imagine A comic style cartoon image of a young woman in her early 20s, muted colors, wearing a black shirt and pants with a teal short-sleeved volunteer jacket isolated on a white background --ar 2:3
Style Choice and Initial Results
Why comic style? Two reasons. Primarily, I wanted the images to be a supplement to the app concept screens, not a distraction. While Midjourney is more than capable of generating photorealistic images, I felt that photorealism could be overwhelming in this context. Secondly, it would be more difficult to successfully control the details of each scenario using photorealism. Also, using a comic illustration style would potentially cover slight inconsistencies when the character was rendered in different scenarios. The overarching tasks the volunteer is performing are paramount to the story, not the minute details.
Here is a sampling of the early output from the Midjourney prompt described above:
Refining the Reference Character
In the early attempts, the look of the volunteer uniform — especially the jacket — wasn’t quite right, so I needed to iterate. Fortunately, Midjourney makes it easy to generate variations on an initial image rendering. After generating an image, just select ‘Vary (Strong)’ for a strong deviation or ‘Vary (Subtle)’ for a minor deviation from either the Discord or web interface. It’s also easy to make changes and tweaks to the prompt before committing to the variation.
I continued to generate variations in this manner until I achieved the result that got the uniform right and also had the visual simplicity to work successfully in multiple task scenarios:
Mapping Out the Scenarios
With the reference character established, it was time to get busy generating the scenarios I needed to illustrate the user flow. The first step was to create a list of tasks and interactions a volunteer performs during a typical TCAB shift:
- Sign into the shift
- Review the shift checklist
- Ask the charge nurse if she needs assistance
- Swap out nurse information cards on patient doors
- Respond to patient needs via telephone at the nursing station
- Create patient discharge folders
- Disinfect the nursing station
- Restock supplies
- Check in on patients to see if they need water, ice, blankets, etc.
- Walk patients that can ambulate
- Enter any important notes for the shift
- Sign out of the shift
Using the Character Reference Feature
Once I had the task list, I was ready to use the Midjourney Character Reference feature to place my volunteer character in those scenarios. The first step in order to use the Character Reference feature was to acquire the URL for the reference image — I’d need it for the prompt later. To do this, I used the Midjourney web interface to grab the URL:
After that, it was a matter of creating a prompt using the Character Reference --cref and Character Weight --cw parameters. Here’s an example how I structured that prompt for one of the task scenarios above.
/imagine an illustration of a volunteer answering a white multiline telephone at a busy hospital unit nursing station desk, the volunteer is wearing a black t-shirt and pants --cref https://s.mj.run/q53e6wOymnM --cw 100
There are three parts to this prompt: 1) a description of the scenario for the character, 2) the Character Reference --cref parameter followed by the URL of the reference image, and 3) the Character Weight -- cw parameter.
The Character Weight parameter,--cw, combined with the --cref parameter sets how strongly to interpret the reference character. Setting the --cw parameter to the maximum value of 100 instructs Midjourney to use the reference character’s face, hair and clothing absolutely verbatim. Since I wanted the character consistent in each setting — including hair and clothing — I set --cw to 100. However, say you want to have your character dressed differently throughout a period of time (at work, cooking, playing tennis, etc.) you can set --cw less than 100. For example, specifying --cw 0 will only make the character’s face consistent, which will then allow you to describe alternate hair and clothing in the prompt. --cw 50 would be somewhere in between. It takes some experimentation. Here’s the full description of both the --cref and --cw parameters in the Midjourney documentation: Midjourney Character Reference.
You may have observed that I still noted “a black t-shirt and pants” in my prompt, even though --cw was set to maximum. That is because Midjourney would occasionally render the shirt and pants as slightly different colors and it was important to me to keep the volunteer uniform consistent. Adding the clothing description to each prompt took care of that problem.
Here’s the image Midjourney generated from the prompt above:
Iteration, Iteration, Iteration
Or, I should say, one of the many, many images Midjourney generated. Rarely will you get what you want from Midjourney right away. In my case, the image above was the product of quite a bit of iteration and variation. For example, below is a sample grid of the images I went through before I landed on the image I wanted to use. Fortunately, it’s easy to generate variations in Midjourney until you get the right result.
Then it was a matter of rinse and repeat — going down the list of scenarios, crafting the right prompt using the --cref parameter to generate a series of images that would fit that scenario and then choosing the best image to illustrate that particular scenario. Here is a sampling of images from the presentation with their corresponding Midjourney prompts below them:
/imagine an illustration of a volunteer walking down a long hallway on the floor of a busy hospital unit --cref https://s.mj.run/q53e6wOymnM --cw 100
/imagine an illustration of a volunteer knocking on an open hospital room door on the floor of a hospital unit, the door is slightly agar and the volunteer is about to enter --cref https://s.mj.run/q53e6wOymnM --cw 100
/imagine an illustration of a volunteer walking into a hospital room bringing a patient a folded beige blanket --cref https://s.mj.run/q53e6wOymnM --cw 100
/imagine an illustration of a volunteer walking into a hospital room with a cup of water for a patient --cref https://s.mj.run/q53e6wOymnM --cw 100
/imagine an illustration of a volunteer wiping down the counter at the nurses station of a hospital unit with a disposable wipe --cref https://s.mj.run/q53e6wOymnM --cw 100
/imagine an illustration of a volunteer at the nursing station of a hospital unit filling out a form --cref https://s.mj.run/q53e6wOymnM --cw 100
/imagine an illustration of a volunteer restocking supplies in a supply cabinet on a hospital unit --cref https://s.mj.run/q53e6wOymnM --cw 100
/imagine an illustration of a volunteer entering notes on her mobile phone on the floor of a hospital unit --cref https://s.mj.run/q53e6wOymnM --cw 100
Conclusion
I found that using Midjourney to create consistent characters for task scenario illustrations enhanced my ability to tell a compelling story and improve stakeholder understanding. By visualizing specific use cases with a consistent character, I was able to create a more cohesive narrative that helped healthcare professionals better grasp the context and value of my proposed solution. For me, it opened up new possibilities in how I approach UX-based presentations.
