See what your users see
This is the first in a series of posts about Elm’s exportable app history. You can read the second post here.
If you’ve ever received a support ticket, you’ve seen something like this: “Carmen M. can’t complete her quiz, it’s not affecting any other students”. Great. Was she unable to select answers to the questions? Able to answer but the submit button stays disabled? Able to tap submit but to no avail?
Users don’t submit detailed error reports because they don’t experience an app the same way its developers do. Nor should they! They have better things to do than learn our lingo. We can partly bridge this gap with technical data, but for difficult bugs you often need more to understand the state a user like Carmen has found herself in.
At eSpark, we have a pretty extreme version of this understanding gap. Many of our users are still learning to read and their issues must be intermediated by a teacher, who is unlikely to have seen the issue firsthand and whose twenty or more students won’t afford him time to explore software issues.
These can be hard problems to solve, and so we’ve put effort into gathering better data when issues occur. One of our first support tools pulled together log data from various sources in to chart users’ paths through our app. More recently we’ve investigated ways to improve the resolution by capturing every every action with meaningful contextual data. These tools are helpful, but they still don’t allow us to really see what happened to our users, the way we could if we’d been in the classroom watching over their shoulders.
What if we could see our apps through our users’ eyes? See what our users saw and retrace their steps through our application?
Enter Elm
Our eSpark app assists teachers with students learning fundamental math and English skills. As we started on a new version, one question we asked ourselves was how we could gain deeper insight into the experience of our users. The story of how we chose Elm is both long and worthy of a future post, but one significant motivation for our choice is the powerful debugger that ships as part of the Elm framework.
In a normal Javascript application, every function can store its own state and change it at will. In an Elm app, by contrast, all data is defined in one central model and can only change through specific messages. If a particular state can’t be represented through your application’s model, your application can never end up in that state, no matter what your users do. And since all that data is clearly defined, Elm can easily show you the full state of your application.
Here’s an example from a side project of mine related to books:
This view, provided by Elm’s debugger, shows you both the complete state of your application and also all the ways that state has changed. Elm (compiled in debug mode) tracks every event that occurs, and thus all possible changes to state. Import a history that a user exported and you’ll see *exactly* what they’ve seen _and_ each step they took along the way.
Mission accomplished? Not quite.
A tool is only as powerful as it is usable. Elm’s history requires a user to export a file and then email it to you. That may be plausible in many cases, but it’s a tall order for either a kindergartner or her teacher, both of whom have better ways to spend their time in the classroom.
Unfortunately, Elm doesn’t expose an API to access this history. Fortunately, that’s not the end of the story.
Scripting History
The Elm history export button is just HTML rendered onto the page. There may be no formal API, but automatically clicking on a button and capturing the result? That we can do.
When you click on the Export button, Elm behind the scenes creates a new <a> tag (in virtual-dom -> Debug.js) whose href contains all the data for the file:
Pretty nifty.
Because the <a> tag flickers info and out of the DOM, we can’t listen to it directly; instead, we can subscribe to all click events for the page and look for ones representing history exports.
Specifically, when we see an href that’s a long JSON string (matching, for instance, the regex /{“elm”:“0.18/), we know we’ve got Elm history data. We can then capture the href string, extract its data, and pass it on to whatever handler we want.
The code is actually pretty simple:
From here, keeping track of a user’s history is pretty straightforward. Using whatever interval or trigger you want, generate a click event on the export button:
ElmRings!
I’m excited to announce elm-rings, a flexible, open-source Elm history recorder! Drop it into your Elm project and you can start capturing user history whenever debug mode is on:
It’s up to you what to do with the data you gather — at eSpark, we send it back to our server and store it for the user after sanitizing passwords and other data. (This requires some extra work because of the format.) When we get error reports, our support team can grab the latest state, look for errors, and pop it into their own session to see exactly what the user experienced.
Future improvements may include logic to highlight error messages for our support team and (to the extent possible) automatically filling in the user’s current access token or other filtered data to allow us to fully use the app as the user.
The Inevitable Caveats
There are caveats, though.
- Data security: every keystroke of a user’s password and all the sensitive personal information they enter in your app go into Elm’s history. It’s no good hashing passwords on the user model if another table contains them in plain text — you need to sanitize and secure this data carefully if you store it. (This is discussed further in the ElmRings readme.)
- Performance: an app that generates a lot of entries may well run into performance problems eventually., especially on lower-powered hardware (such as the Chromebooks or iPads schools use). I haven’t measured when those would occur, but if you’re storing a lot of data or generating a flood of events, keep that in mind. (I’d be grateful for any data!)
- Exposing your internals: with debug mode enabled your users (and, in theory, any Javascript on your page to see exactly what data your app stores and how it’s structured. All front end applications have to assume any data is open to the world, but this makes it unusually accessible.
Given those caveats, you may decide (like us) to only capture state for particular users for whom a flag is enabled. The big drawback of that approach is that you can’t proactively know who will encounter an error — it’ll be difficult to capture hard-to-reproduce problems.
There’s one additional limitation that’s both obvious and worth stating explicitly: the Elm history only captures what happens in Elm. If your app is all Elm, you’re golden. If you’re integrating with Javascript libraries to do fun things like in-browser video recording with Javascript and WebRTC (more on that soon), you’ll have blind spots in your log. For this and many many other reasons, the more you can put in Elm the better.
Could we do even better?
What would an ideal implementation of tracking Elm history look like?
What I’d most like to see would be direct and exclusive access to the debugging history via the application’s Javascript. That would close the security hole of having export button in the DOM for curious users or malicious scripts to peek into. Something like:
Another great feature would be the ability to import history programmatically. Your team downloading files with sensitive user history presents a risk — how much better would it be to redirect them to the app with a one-time code in the URL params, where the history would be automatically downloaded and imported with no files ever hitting a hard drive?
What’s Next
Both our engineering and support take have been excited to use of history tracking in our new eSpark. It’s still early days, so as we get more experience we’ll share how it and other aspects of Elm work in classrooms and for our team.
It’s been an exciting project and we’ve learned a lot — keep an eye out for more blog posts in the not-to-distant future!
Check out the next post in the series, Understanding an Elm History Export.
