I. The preparations

The first step in implementing saving and sharing was… giving it some time instead of rushing into things. We did it just so that the team itself became comfortable with the project, and so that we rejected the pull of choosing the easiest (but not necessarily the best) solutions.

The GitHub issue asking for street saving was the second one ever filed, and you can see some of the early process there, slowly discussing away the most easy/obvious solutions (using URL fragment to store the street, or using Save/Share buttons).

II. The tenets

After waiting some time, I approached the problem holistically — trying to think of the entire saving/sharing system in broad strokes. Since there is no One True Way solution, it felt more important to pick one that was internally consistent. What informed my particular approach was internal drive towards extreme simplicity, and Code for America’s great attitude of defaulting to public and open.

Here are the tenets I arrived at:

1. Auto save

Ask for forgiveness, not for permission. There should be no explicit save button; the street you’re editing will be saved automatically as you go along. If you make a mistake, the multi-level undo will cover your back.

2. Auto share

Open by default. Every new street will be public and shareable from the very second you create it. There will be no explicit “share me” button.

3. Collaboration

You will be able to edit (“fork”) other people’s streets without needing to ask anyone. (We dubbed this “remixing.”) There will be no explicit fork button.

4. Signing in

We won’t require you to sign in, and we will provide all of the core functionality if you don’t. (An account requirement would be a huge barrier to entry, and we wanted people to be able to pick up Streetmix straight away.) However, we will encourage you to do so, and provide extra benefits if you do.

5. Persistence

Streetmix behaves like an app. Going to streetmix.net will put you back in the exact same state you have seen it last (e.g. the last street you edited).

6. Protecting your data

Streetmix will make it very hard for you to lose your edits or data.

7. Great URLs

We will use nice, simple, short, human-readable and -editable URLs. We will also make sure that our URLs don’t ever expire.

8. Thinking of the future

We won’t overthink it (“premature optimization is the root of all evil”), but we will spend some time thinking through some future possibilities so that our solution is reasonably future-proof.

9. Going the extra mile

We will build a solution that goes an extra mile; we won’t be happy with just mediocre saving and sharing functionality. There will be at least one thing in our solution that’s best of class. Ideally, more than one.

III. The flows

After establishing the basic tenets, I created some quick mock-ups and put them together in a slide deck. (The mock-ups could conceivably be just paper sketches, except it was easier for me to prepare it all in Photoshop.)

With the deck in hand, I could present the basic ideas and all the important flows to team members:

The basics

Every street will have an editable name (mandatory) and a little description (optional).

The street will be saved automatically reasonably quickly after you make edits. If you will try to close the browser before the save occurs, we will put up a dialog box warning you that you will lose the latest edit.

Moreover, the undo stack will also be saved along with your street — if you close your street and open even months later, you can still undo it many times over.

We will remember the last street you worked on and your preferences (e.g. metric vs. imperial units) using HTML5 local storage (otherwise known as “modern cookies”) if you are not signed in, and also keep it in your profile if you are signed in.

There will be a pretty standard command menu in the upper-right corner prompting you to sign in if you are not — or showing your avatar/name, a gallery of your streets, and a sign out link if you are.

There will be a simple sharing menu allowing you to take the URL and post it to Twitter or Facebook. We decided to limit ourselves to only those two options as they are the most popular. (Note that all the sharing options are just vehicles to send the street URL to a given social service… they don’t actually change the street in any way.)

If you’re not signed in…

Going to Streetmix for the first time would create a new street and assign a unique new URL to it, e.g. streetmix.net/-/125. This URL will always point to this street and never be deprecated. You can immediately share or bookmark it. I chose an incrementing, integer counter for streets over UUIDs or hashes for mostly aesthetic reasons: nicer-looking, shorter URLs. (That had eventual latency consequences—every time a new street would be created, we’d have to check with the server to “secure” a new URL.)

The option to see all of your streets won’t be available, since we won’t have any idea who you are. We will nudge you to create an account/sign in so that you get it.

If you’re signed in…

We will use Twitter’s authentication mechanism, and their namespace.

Creating a street would assign it a URL based on your name and the street name, e.g. streetmix.net/mwichary/post-st. A good-looking URL with your Twitter handle in it was a deliberate choice to promote signing in and make people feel closer ownership of their streets. (The hyphen in the non-signed URL signifies the default/unsigned namespace, and is meant to make you feel slightly uncomfortable.)

You will see a gallery of all your streets you created, with an ability to delete them.

If you are looking at someone else’s street…

The UI will look a bit different, showing the street creator’s avatar and name. In order to edit their street you simply start messing with it, as you would with your own street. At the moment of the first change, Streetmix will automatically make a copy of that street (“fork” it) and assign it to you (if you are signed in) or put it in the default namespace.

Contentious/challenging parts of the proposal

Street pollution. Since visiting Streetmix for the first time automatically creates a new street with its own shareable URL, this means there will be tons of stray/unfinished/barely begun streets floating around. Forever. We decided this was okay, since disk space is cheap.

Starting a new street. If Streetmix behaves like an app, and every visit to it puts you in the same state you’ve seen it last, how do you start a new street? I’ve decided to create an explicit New street link visible in the menu.

Twitter as authentication. Not everyone has a Twitter account. Since Twitter is free, I thought that there’s much more benefit of the simplicity of keeping to one authentication method, rather than providing more options and the resulting user’s confusion (“what did I use to sign to it last?”) and resulting URL messiness.

IV. The discussion

I presented the above deck, individually, to all the people on the team. That allowed me to gather opinions and feedback from people, and find out about holes in my thinking. Here are some of the problems unearthed during this process:

Street renaming

If you are signed in, the URL will contain the street name (e.g. streetmix.net/mwichary/post-st if my street is Post St).

Let’s assume you then rename the street to Geary St. The URL then becomes streetmix.net/mwichary/geary-st. What happens to the old URL, though? We could either keep it forever, quietly redirecting to the new street — but that pollutes the namespace, and necessitates things like streetmix.net/mwichary/post-st-2 if you later create the “real” Post St.

After some deliberation, we decided to amend the URL scheme to use streetmix.net/mwichary/5/post-st instead. The number 5 would indicate that this is my fifth street. Everything after this would be a slug, effectively ignored by the application, so even ridiculous addresses like streetmix.net/mwichary/5/ponies would work, and Streetmix would quickly rewrite them to the proper URL based on the street name upon arrival.

(Amazon does something similar, having separate machine-readable and human-readable parts of the URL. However, they never rewrite their URLs, so you can have fun with fake addresses such as www.amazon.com/I-Have-A-Huge-Mancrush-On-Will-Smith/dp/0394720245).

The only downside to our approach would be that streetmix.net/mwichary/5/post-st is not as clean as streetmix.net/mwichary/post-st. However, that seemed like a small price to pay. And the extra bonus? It’d be easy to navigate people’s streets by just changing the number.

What happens if you want to re-edit your own street, but you’re not signed in?

The only person allowed to edit your street is you (other people can only remix/fork/copy your streets). If you are signed in, we know exactly who you are, so that’s not a problem.

The situation becomes more complex, however, if you are not signed in. Any street created by a non-signed-in user could conceivably belong to any other non-signed-in user. As long as you keep editing the street in the tab, we know it’s yours. If you close and reopen it, though, that link is gone. We could keep some information about the streets you edited in your browser, but that seemed frail, complicated, and unsafe.

So, opening a non-signed-in street as a non-signed-in user always creates a new copy of it. The upside is that it protects people from editing the streets of other people. The downside: editing your own street if not signed in would create new copies of the street (with new URLs) if you close the tab and come back to it later.

What happens when you open a new tab?

Given the requirement of persistence, opening a new tab would just point to the same street you were just editing. This is less than ideal, since a second tab might indicate the user is interested in creating a new street.

My solution was to… do nothing. I assumed that this will be a rare instance, and we already had a New street option that would override the persistence and always create a new street. After some thinking, I decided to actually create an explicit URL (streetmix.net/new) that would always start a new street, so that we could link to it in different places also.

V. The implementation

The deck would be updated a few times and serve as a first version of a spec. Later on, our back-end engineer, Shaunak Kashyap, created a proper spec in a Google document that a few of us on the team commented on so that we cleared up a lot of misunderstandings and ironed out some details:

Saving timing

We needed to fine-tune some of the saving timing. We wouldn’t want to save things immediately after changes are made since that would tax the servers… but the delay too long would make it too easy to try to close the tab or otherwise move away from the page before the changes were saved.

After some experimentation, we decided that a 500ms delay seemed good enough, with an exponential back off (500ms, 1000ms, 5000ms, 10000ms, and then a random number from 0s to 60s) if the connection was down.

Dealing with failures

I realized that some of the server failures are non-blocking and we should try to resolve them quietly (e.g. transparent saving), whereas others should block the UI until the connection is re-established (e.g. trying to make a copy of someone’s street, since you need to obtain a new URL from the server).

For the blocking UI, I designed a “shield” that blocks access to the UI and slowly fades out the screen while showing an error message.

For the non-blocking UI, I decided to forgo any warnings until a few seconds pass, and then show a non-intrusive message at the bottom, informing the user about connectivity problems.

Both of the UIs present a Try again button giving the user the power to immediately ask Streetmix to try again if they believe Internet connection issues are resolved.

New street menu

I hypothesized that sometimes you want to start with a new, empty street… but other times you might be interested in taking an existing street you have, and making a variant of it. How do we put together a UI that allows for both?

The first solution I thought of was making the New street button a pull-down menu with a few options (empty street, example street, copy of the street you’re looking at…). However, that seemed not great in that the user would need to make to a choice without fully knowing what it entails.

So, I reversed it. The New street option would take you to a new street, but then present a little temporary menu where you could switch between all the options, getting an immediate preview below. The moment you started editing the street, the menu would go away. (Plus, your choice was remembered as a default for the future.)

Namespace collisions

The streetmix.net/mwichary/7/main-st URL scheme using Twitter’s namespace looks clean and great, but it forbids Streetmix itself from having nice URLs — if we wanted streetmix.net/error or streetmix.net/help/about for internal pages, it would prevent @error or @help from ever using Streetmix! As unlikely as it is, we couldn’t risk it.

We decided to special-case those URLs so that we could still use streetmix.net/error, but a hypothetical user named @error could access their streets via streetmix.net/~error. We now do it for a number of URLs (new, help, gallery) — some already existing, some aspirational.

Street naming

We spent some time thinking of the default street name, arriving at Unnamed St. Defaults are precedents, and with this one, we wanted people to understand that a) the street name is editable, and b) it’s a street name (the suffix St) rather than something else. (We also tried to communicate it with a visual style somewhere in between European and American street labels.)

Originally, editing the street name was part of the undo stack. This proved to be a mistake (imagine making some changes to the street, then renaming it, and then trying to undo the changes). For a while, I fought for special casing it so that if you change the street name, you can undo it if it’s the last thing you did… but it felt too cumbersome.

Manual saving

If you try to save manually by pressing Ctrl–S or Command–S, Streetmix will suppress the browser save dialog box, and instead display a friendly transient message: “No need to save by hand; Streetmix automatically saves your street!”

Street name UI

We implemented a few nice touches when editing your street UI.

The name tries to get out of the way of editing, but drops down if it would otherwise start overlapping the header:

And, while we wanted to use a stylized typeface that reminded people of street signs (some discussion we had over it), we will drop down to a standard browser font the moment we discover any of the entered characters is not available in the stylized font. We specifically wanted to prevent having different letters in different fonts, which is what happens in browsers by default.

Error messages

We tried to make our error messages as friendly as possible.

One example: if a user deletes their street (one of the benefits available to signed in people), we will show an error message that’s slightly more informative than a usual 404 — it will actually identify the street as having been deleted, and also include a link to the gallery of streets by that person.

Omissions

If you look at the original mock-ups, each street was supposed to have a description alongside the title. We deprioritized this — mostly because I never found out a way to keep it there without cluttering the rest of the page.

Likewise, Streetmix before saving and sharing had a little sizing chart summarizing how much space on the street was devoted to pedestrians, cars, nature, etc. The street name and other UI took over that space — and we realized we didn’t miss it all that much.

VI. The edge cases

In addition to all of the details, we spent a significant amount of time thinking of all the possible edge/corner cases, and how Streetmix would deal with it.

Here’s a summary of the problems and solutions we arrived at:

Street saving fails intermittently (possibly due to Internet connection being flakey or down). We made it harder to close the page by showing a warning dialog box if changes were unsaved. We also kept trying to re-save automatically with increasingly bigger time distances (using an exponential back-off algorithm). Eventually, we would show a transient UI as discussed above.

Street saving fails with server returning 500 errors. Same as above.

Server failure when creating a new street. Don’t allow to proceed (but allow to retry), since we need to get a proper unique URL from the server to get going. We considered allowing to play offline with a warning that changes won’t be saved… but that seems like a lot of complexity for a very little payoff.

Server failure when remixing a street. Same as above.

Server failure when switching streets using the gallery. Decided to show a timeout, and allow to retry after some time of inactivity. We don’t allow to close the gallery in this state.

Server failure when deleting streets from the gallery. Similar to saving the street… Transparent failure. Keeping all the delete requests cached somewhere, and trying to repeat them silently in order. Don’t make it easy to close the window before saving changes.

Server failure when telling the new street to be a copy of the last-viewed street. Blocking. Allow to retry after some time of inactivity.

When resuming (loading the homepage), local storage points to a street that no longer exists (or local storage is corrupted). Silently start a new street, as if local storage variable wasn’t there.

Signed-in user signs out in another tab (same browser). Immediately show a warning dialog box covering the entire page that tells the user “You signed out in another window. Please reload this page before continuing.”

Signed out user signs in in another tab/window (same browser). Same as above. We considered silently reloading the page and “promoting” the street automatically, but it seemed simpler to just be explicit about it.

User opens same street in two browsers and makes changes in one. Whenever the user re-focuses/returns to any Streetmix tab, we force a non-blocking check-in with the server, fetch new street and silently update it if it’s different, while showing a transient warning (“Your street was reloaded from the server as it was modified elsewhere.”). We can do it because querying the server is not expensive, and we want a non-blocking call since otherwise Streetmix would feel slow.

This introduces a possible problem: Let’s assume there are unsaved changes on the client that are newer than changes on the server. Reloading the page from the server will result in “clobbering” the local changes. We’re okay with it since we assume this is rare and not very painful.

User opens same street in two tabs and makes changes in one. Communication between tabs is easier than between browsers since it doesn’t need to involve a server. We considered keeping a local storage entry with identifiers of streets in all opened tabs, and forcing other tabs to reload the data from server if it’s the same street. But in the end, we decided to do the same as above for simplicity.

User deletes the street in another tab/window/browser, and then comes back to the original tab where that street was opened. Version of the above. When the non-blocking check comes back with a deleted street, we show a blocking/full-screen message saying “This street has been deleted elsewhere.”

VII. The testing

We proceeded with a two-phase testing.

Phase 1 (internal)

After having implemented the core functionality, but before most of the niceties/edge cases, we first unleashed the new version on the internal team. This resulted in a lot of great feedback, and some hilarious situations (see screenshot above, where a team member was able to edit someone else’s street by undoing through fork ownership change — a huge security problem).

Phase 2 (external)

Eventually, after we fixed all of the big errors we knew of, we opened up the trial to a couple dozen trusted external testers. Since simple non-saving streetmix.net was already live, this presented a problem. The easy solution would have been to launch it under a separate, “secret” domain (e.g. streetmix-experimental.net), but that’d mean all of the URLs would cease to work after we eventually launched the new version and switched back to streetmix.net.

To solve this, our engineer created an ingenious cookie-based solution, so that people who would receive a secret URL would opt into the experimental Streetmix, and other users wouldn’t know any better… even though both groups would use streetmix.net URLs.

This second phase helped out find out a few more bugs, and would allow to test the server with a bit more load. As we didn’t have time nor resources to do usability testing, we also hoped to learn a bit about whether our user interface was understood.

VIII. The launch

We quietly launchde the new system Thursday night, and announced it publicly on Friday morning:

It’s fun to compare the original mock-ups with the eventual outcome. They are widely different visually, but surprisingly similar conceptually:

The launch went surprisingly smooth. Our engineers struggled with one or two hard-to-reproduce problems somewhere between our server and Twitter authentication API — tough enough to actually delay the launch by a week.

They eventually solved them, though, but not before our team arbitrarily naming them Error 15A and Error RM1 (for “Robert Moses”) so that we could ask our users which specific error they’ve seen. (We ourselves have had nightmares about 15A for weeks.)

None of the other issues we noticed were launch-delaying or -breaking. Here’s a list:

No user testing. The biggest omission in the whole design process was our inability to talk to people — it was a side project, after all, and we had very limited time and resources.

Saving being too transparent. We did a pretty good job at making saving invisible… too good of a job, in the end. Unless you already know about it, it feels like your street is not saved. Even then, the system feels . Google Docs and Medium show little ambient saving indicators to keep your mind at ease, and we should have done something like that.

Only supporting Twitter. A small number of people complained about only supporting Twitter as authentication.

Sharing a point in time. We noticed that some people expected sharing to freeze the shared street in time — they were surprised that the changes they made after sharing would show up in the links they sent out to people.

Entry point. If you arrive at Streetmix for the first time via streetmix.net, you will get a nice welcome menu:

However, an artifact of how we do things is that you will never actually see that naked URL in your browser — it will immediately get replaced by a link to your street. And, indeed, most people talking about Streetmix would share the URL to their streets. This prevented newcomers from seeing an introduction.

We tried to solve this predicament by adding a variant of the welcome menu for people arriving through a side door.

Persistence. Coming to Streetmix after a week and seeing a random street you just happened to view most recently last time, is not the greatest of experiences. Ideally, Streetmix would have a start page with a few options (start new street, continue where you left off, streets you last edited, etc.).

Undo stack. We decided saving the undo stack was not worth all the trouble — it made things much slower and quickly started burning through our disk space. Truncating the undo structures from literally dozens of copies of the streets as you go along, to something more clever like deltas, would allow us to bring this feature back, but we never thought it was a high priority.