Creating Hyper-Realistic “UX Simulations” using Open Source Software
UX simulations are a new idea for automating the production of lifelike user interactions within a web application. With a wide variety of potential uses, these pre-scripted demos and videos can help inform product design, serve as marketing collateral, and even improve customer success.
In this 3 part series, we’ll explore how to use a free and open source product called Cypress.io to create these valuable resources.
- Part 1 — Introduce Cypress and the concept of UX simulations.
- Part 2 — Explore the value of UX sims and how to operationalize them.
- Part 3 — Dive into the code and show how to make it work.
What’s the (Intended) Purpose of Cypress.io?
For technical readers, Cypress is the latest and greatest in end-to-end automated testing based on Electron.js and Chromium based web browsers (with more coming in the future).
For less-technical readers, Cypress is a web browser that you control with code — No mouse or keyboard required. Clicking buttons, navigating between pages, and generally replicating user interactions are all scripted events. These scripts perform user-like interactions on working applications to ensure that everything is working as intended.
[Note: I’m not affiliated with Cypress in any way — Although I did try to answer one issue on their Github]
A developer working on a new feature can use Cypress for Test-Driven Development (TDD) which helps ensure their code is robust and maintainable. The standard procedure is “Red-Green-Refactor”. I haven’t found any TDD diagrams that I like so I made this:
“Red” is writing a test and intentionally failing it immediately (a passing test would indicate a problem at this stage)
“Green” is writing just enough code to make it pass.
“Refactor” is any improvements to the code from that point forward.
With a passing test in place, developers can confidently work anywhere within the codebase knowing they’ll be alerted by Cypress if they’ve accidentally broken something.
After a new feature has been developed, it needs to be deployed into production. Many/most/all modern web apps are using some kind of Continuous Integration (CI) pipeline. This is a process that runs on a remote server to automatically stage, test, and either reject failing or deploy passing code. In this way, Cypress acts as the last-line-of-defense before code goes live.
Cypress is designed to be fast. As fast as computerly possible — Which happens to be really, really fast. This is ideal for both TDD and CI because it provides rapid feedback and lowers deployment times.
The often quoted factoid is that Amazon deploys new code into production every 11.6 seconds (circa 2011). That’s a feat only made possible through the use of high speed automated testing.
Cypress.io is NOT Built for Humans!
Cypress bills itself as “a test runner built for humans”, and after several days of use, I can really appreciate where they’re coming from. The ease of use and overall utility it provides for development is admirable. I can honestly say it was super intuitive and a lot of fun to work with — At least until I tried making it do magical things (more on that in Part 3).
The unfortunate part is that Cypress is really only built for the rather narrow subset of humans (and computers) at the very end of a much longer product design process. Anybody outside of that small sphere of specialists will probably feel more like this:
While it was never designed for our purposes, at its core, Cypress does simulate user behavior. If it could be throttled back to the speed of an “actual user” [Spoiler: It can], those simulations suddenly become something we can all have a conversation about.
In addition to its general ease of use, Cypress is also capable of rendering full HD video right out of the box. This makes it an ideal product to begin experimenting with the very first generation of “Hyper-Realistic UX Simulations”.
What’s the Point of Creating UX Simulations?
In my view, UX simulations have the potential to both inform the product design process as well as form part of the product itself. I’ll be diving deeper into this in Part 2, but here is a quick list of possible use cases to get us started:
- Pre-alpha user testing
- Sign-offs and approvals
- Informal peer feedback
- Meetings and huddles
- Demos and presentations
- Redesign references
- Variant design evaluation
- Style guides
- Marketing collateral
- Evergreen documentation
- User/employee onboarding
- Customer success
- Posterity
Some of these may be impractical, but others are definite wins. Also, similar to any new technology, unanticipated opportunities will start to emerge over time.
When you see how easy it is to produce UX simulations, simply as a natural byproduct of the product design and web development process, you’ll understand that almost any value at all could be worth the effort.
Coming up…
In Part 2, I’ll dive a little deeper into the value of UX simulations in the product design process. I’ll also discuss how you might operationalize their production.
In Part 3, we’ll start to build a bridge across the uncanny valley and discuss some of the current limitations to achieving truly hyper-realistic UX simulations.
BONUS: Who am I and Why Does it Matter?
My somewhat unorthodox approach to Cypress is informed by my equally unorthodox career path. I studied Marketing-Entrepreneurship before supervising a large volume of technology meetups at a coworking space on the edge of Vancouver’s startup district.
This inspired me to jump into a Technical Web Design program at the British Columbia Institute of Technology (BCIT). At the end of my program, I was recruited by the school to support online education. It was working at BCIT where I started introducing tools similar to Cypress (jsdom, Cheerio, PhantomJS, Puppeteer, and Electron) to automate my job away.
My growing interest in Product Management helped inspire the idea of using Cypress to generate UX simulations so I’m looking forward to sharing my thoughts on their potential value to the product design process.
It matters because I’m currently looking for work.
