A user encounters a JavaScript error. You’ll never guess what happens next!!
One of the hardest things a developer must do is release their bug-riddled code out into the world, knowing they may never hear from those little critters ever again.
The savvy developer, then, should have some sort of mechanism so that those bugs can call home and tell you about their adventures in user-land.
Maybe your bug reporting setup is just this:
Now just ask your friendly network engineer for a spreadsheet with all the 404s starting with /errors. They’ll love that.
But that ‘code’ probably isn’t going to help you track down where the error occurred. You’ll want to go one better and send a message containing the file and line number:
That’s borderline legitimate code, but like a attractive grizzly’s skeleton, it’s still pretty bare bones.
If the error is related to some specific data then you won’t have much luck replicating it.
Wouldn’t it be neat if you had a full report of the user’s activity so you could retrace their steps. Something like this:
The interesting parts being that the user navigated to a product details page (step 4) and clicked the buy button (step 5, the final step).
Straight away I can guess that there’s probably something iffy with the data for that particular product, so I can go to the URL and hit the button labelled “Buy this for $”.
Sure enough, when I do this, I get the error; this particular product had no price, so calling toLocaleString threw an error. Classic rookie mistake.
But … what if the flow was more complex? Maybe the user was on one of many tabs not reflected in the URL, or the error occurred when validating data from a form.
Following a URL and clicking a button isn’t going to cut it.
What I really want is to be able to play back all of the user’s steps up until the point of the error. Ideally just by clicking a crimson “Next step” button again and again.
This is how it would look in my imagination if I didn’t have aphantasia:
(The error showing on the screen and the file opening in my editor are courtesy of the Create React App setup, nothing clever that I’m doing.)
Side note: I actually made this as an experiment to be able to carry out ‘unmoderated usability testing’. I wrote all the code that tracked the user actions and played them back, and then asked a clever chap called John what he thought. He said it was a stupid idea, but that the code might be useful for replicating errors. So, that’s what this blog post is about now. Thanks John.
The setup
The code is all here if you’d rather read that than my words. I’ll show simplified versions of the functions below and link to the full versions as I go.
I have a module record.js that contains a handful of functions for capturing different user interactions. These are all captured in a journey object that is sent to a server somewhere if an error is encountered.
At the entry point to my app, I’m going to kick things off by calling startRecording(), a function that looks a lot like this:
And when an error occurs, I’m going to send that journey object off. I’ll do this when I start my app:
With sendErrorReport (in the same module that journey is defined) looking like so:
So far this is not super useful. But we now have the framework to start capturing more interactions.
If you have a state-based app (that is, the DOM is rendered based entirely on some central state) then your life is going to be easy (and dare I say you’re less likely to have bugs in the first place). When trying to replicate an error, you can simply re-create the state and you’ll probably be able to recreate the error.
If your app is a little bit retro and has things hiding and showing based directly on users clicking on buttons, things are going to be more difficult. You’ll need to replay click and focus events, and I guess each key press too. Not sure what you’re going to do about users pasting data. Best of luck to ya.
I’m both lazy and selfish, so I’m only going to address the scenario of a site using React and Redux.
There’s a few things I want to capture:
- All dispatched actions (so I can ‘play back’ the store state)
- URL changes (so I can update the URL)
- Clicks (so I can see with my eyes the buttons and links the user was clicking on)
- Scrolling (so I can see what the user was seeing on the page)
Intercepting Redux actions
This starts with the totally-easy-to-comprehend = () => next => action => {. You can go read the docs if you’re interested in what that’s all about. I’d rather spend my mental bandwidth on more important things, like practising my facial expression for when people are singing happy birthday to me.
All you really need to know is that this will push Redux ‘actions’ into that journey object as they happen.
I then reference this middleware when creating my store using Redux’s applyMiddleware function.
Recording URL changes
The point at which you capture a URL change will depend on how you do your routing.
React Router doesn’t make it easy to detect URL changes so you’re going to want to do something like this or maybe something like this. (I wish I could just define an onRouteChange handler with React Router. Surely things like logging virtual page views to Google Analytics is a pretty common thing.)
Anyhoo, I prefer to roll-my-own routing for most sites because it takes, like, 17 minutes to write the code and is significantly faster.
To capture a URL change I have the following function that I call whenever the URL is changed.
I call this in two places. At the same point where I do history.push() to update the URL, and also in a popstate event, which will fire if the user hits the back button:
Capturing user interactions
This is the most ‘intrusive’ of the capturing mechanisms because you need to interweave it throughout your app, so I personally would not bother doing this unless I was getting errors that I thought I couldn’t replicate without knowing what the user had clicked on.
Nevertheless, it was an interesting problem, so I shall document it here. When working with React, I always have a <Link> and a <Button> component, so centralising this is fairly simple. Let’s take a look at a <Link>:
The two lines relevant to this blog post are data-interaction-id={props.interactionId} and captureInteraction(e);.
When it comes time to play back a session, I want to be able to highlight the thing being clicked. For which I will need a selector of some sort. I could insist that elements to be tracked have an id, but for a reason I now forget, I decided that something specific to Peeping DOM (I still wince) was more appropriate.
Here’s the captureInteraction function:
As with all the others, I gather a bit of data then journey.steps.push.
Scrolling
The last thing I want to do is keep track of where a user is on the screen. If they scroll to the bottom of the page and start filling out a form, playing that back without scrolling isn’t much use.
I will batch all sequential scroll events into a single event. I’ll debounce for performance (using Lodash because setting and clearing timeouts in a loop confuses me).
startScrollCapturing() is called when the app first starts.
Some ideas I didn’t build
- You might want to capture keystrokes like escape and tab and enter if that’s going to be helpful.
- You will also want to capture screen resizes (if replaying scroll position is important).
- Rather than capturing scroll position, you could instead call
scrollIntoView()on an element at playback time when highlighting it. - You might want to take a copy of
localStorageandcookiesif they affect the behaviour of your site. - Lastly, users tend to get a bit complainey about people intercepting and saving everything they type, particularly credit card numbers, passwords and the like. So it’s important that they don’t know you’re doing this. Wink.
Edit: contrary to what a few commenters have said, the methods described in this post bring no additional security or privacy concerns. If you’re currently collecting sensitive user data, then whatever requirements you have for the collection and storage of that data must be applied to data collected when you submit an error report. If you wouldn’t auto-save a form without asking, then don’t auto-send an error report without asking. If you must expressly require a user to tick a tick box before submitting a form with personal information, then you must expressly have that user tick the same tickbox before sending an error report. There’s nothing fundamentally different between sending a payload to /signup or /errors — as long as you treat the data in the appropriate and legal manner in both cases.
You might think we’re done now, but that’s all just to record the steps. The sexy part is being able to play those steps back…
Playing back a user’s steps
There’s two parts to this:
- The interface that I open to investigate errors (by replaying the user’s steps)
- Putting some code in my site that allows it to be controlled by external forces
The playback interface
This page will display my site in an iFrame and the me ‘drive’ the site by replaying the steps from a user’s journey.
The page will load the details of a session that resulted in an error, and send each step ‘down’ into the site to change its state.
When I open this page I see some basic, ugly UI, and the site loaded in an iFrame (wrapped in an iPad image for a bit of pizazz).
Here’s the same gif from further up the page.
When I click “Next step” it’s going to send a message to the iFrame using iFrame.contentWindow.postMessage(nextStep, '*'). There’s one exception to this: URL changes. In that case it just updates the src of the iFrame. This is essentially a full page refresh for the app, so might not work depending on how you carry state between your pages.
If you’re not familiar, postMessage is a method on the window object designed to allow communication between different windows (in this case, the main page window and the window in the iFrame).
That’s about all the cleverness that this playback page has.
Allowing external control of the site
The logic for allowing playback is in playback.js.
In the entry point to my app I call a function to start listening to for messages, I pass in the store so I can call dispatch on it later.
I only do this in development mode.
That function looks something like this.
For Redux actions it just dispatches them to the store — nothing more to it. How great is that?
Scrolling does exactly what you might think it does. This is where having the correct screen width is important (and the setup as you see it in this repo will be incorrect if the user changes screen width/rotates their device, but I think calling scrollIntoView() is probably smarter anyway).
The highlightElement() function just adds an outline to the element. At it’s heart that’s just this:
That’s all there is to it.
Debrief
So, is this useful in the real world? I guess it depends on how many errors you get, and how much trouble you have replicating them.
Maybe just capturing URL and store changes will get you far enough to track down an error.
Or maybe the recording logic is useful to you, but the playback interface not so much. If you’re seeing errors that are only happening in, say, Safari 9 on iOS, the playback page is of no use because you won’t see the error in question.
As always, the litmus test (for me) is if I plan to incorporate this experiment into the site that I’m currently working on, and the answer is no.
Anyhoo, as an experiment this was kinda fun, I learnt a few things, and I still hold hope that it might be useful one day as an el cheapo unmoderated user testing tool.
If you find this useful, have some more ideas, or would like to be mean for no real reason, hit me up in the comments.
Farewell.
