Overland — Behind the Scenes — July 2017
As we’ve finally gotten our act together a little bit and moved the bulk of the actual mailing list content to the oh I don’t know mailing list, this month’s Overland article will be just the behind-the-scenes bit. And on a scandalously controversial topic sure to arouse the interest of passers-by: user interface design.
~ wave of ferocious desire passes through said passers-by when they hear this ~
Hang on though, you just did a whole thing on UI back in January, so what pray tell is the meaning of this then? How many times are you going to redesign the stupid buttons for this game? Well, my friend, a lot of times, it turns out. Because it turns out the stupid buttons are pretty important. They’re the pages of our book, as it were — the knob for our door, the zipper on our hoodie. It’s the whole thing of how you play a strategy game, especially a turn-based strategy game, especially a squad-based turn-based strategy game, and most recently, a squad-based turn-based strategy game that supports gamepads.
~ wave of ferocious revulsion passes through coders when they hear this ~
Hang on though, you built the game to support gamepads right from the start right? Because otherwise re-rigging everything this far into production is a complete nightmare. Well OK sure, but the opposite is true too. The first couple major UI revisions were all built around ensuring that we did a good job of supporting a variety of input devices — touch screens, mouse and keyboard, controllers, whatever. But UI already has so many constraints and parameters that it started to feel like an unclimbable mountain. Every small change had to be parsed through so many different lenses in terms of interactivity and moment to moment control and it was just overwhelming. When we started ignoring the alternate inputs and just tried to get decent mouse controls, development and iteration time drastically improved. My point is that making your game, even your turn-based game, feel good to control with *completely* different physical pieces of hardware is pretty tricky no matter when you start taking it into account.
Ok Let’s Talk About Buttons, Nerd
Ok so anyways a month ago Overland was a mouse-based strategy game, and as of a couple of days ago it’s a mouse-or-keyboard-or-gamepad-based strategy game, and they all feel equally fine to use somehow. Since I covered a lot of our past efforts / fiascoes back in January, what I wanted to go over this time was two relatively straightforward ideas that got us over this next hump.
Also all the usual disclaimers here that while I’m the programmer and the person writing this article thing, the systems that I’m writing about were designed in collaboration with Overland team and especially Heather Penn, who it turns out hates writing things.
Ok!
Idea One: Modality
So here’s a thing I was extremely against for a long, long chunk of development, not the least because some previous implementations of it were extremely inadequate. And I’m so disinclined to build things this way anyways that… well hang on. Basics first!
Modality is the idea that the whole piece of software or whatever you’re using sort of changes “modes” around certain sets of actions. Think popup windows (“are you sure you want to close this document?”) or right-click menus (“copy? paste? copy link?”) or a settings menu or the many sort of swiping layers of a menu in a mobile app or the little black formatting popup in Medium here when you highlight a word or phrase. It’s any time when you’re looking at just a small section of the whole thing and making some decisions in isolation. It’s any time you can only do a few specific things instead of all the normal things.
The attraction of modal interfaces is they tend to be a more focused and less cluttered. Related decisions and settings and parameters and choices can be grouped together in intuitive ways, and handled one chunk at a time. The disadvantage is that they can obscure the broader context in which you’re operating, and/or risk moving the human using the dang thing from the familiar set of circumstances they were in to a new and unfamiliar set of interactions. In video games especially where the input barrier can be so high, this is not a risk to be taken lightly.
For Overland in particular, I was thinking about this in terms of some of our other design goals, especially a kind of ongoing target of what we call “legibility”, the ability to sort of glance over the entire game level and be able to tell roughly what is going on, even if you haven’t actually played the game before. Trying to help as many players as possible tell who is in danger, who is safe, and imagine what might happen next. A modal interface seemed like it might be in tension with that. What are the popups going to obscure? What sorts of really basic interactions will be disabled in order to enable a more focused set of interactions?
And the first few times we tried this, in fact, exactly those concerns ended up being true. So uh get out of town, modality. Adios.
What we did instead was moved on to a system where at any given time, every possible action you could do would appear as a floating icon near the thing you could do the action on. This was frequently great, as you could view not just the whole game scene but also see exactly what sorts of things you could do next. That is until you were standing next to one or more heavily interactive objects, in which case the number of available actions would balloon out into a kind of floating cluster-mess from which no sane human could recover.
I chose to not share a screenshot of this situation with you. You’re welcome.
So naturally we hid all those things unless you were already mousing over or highlighting the object in question. Basically, where before, you might see 20 buttons on screen at once, we could use this extra bit of info of “location of mouse on-screen” to limit this to four or five buttons at a time. This was kind of a cheat, since technically that kind of is a modal popup, but since it appeared and disappeared based on highlighting things rather than clicking on things, I thought maybe we could get away with it. It was the perfect compromise. Well except for the few days we spent figuring out how to draw outlines around 3D things, lordy. But otherwise yea, good system.
The thing I liked most about our new faux-modal approach was that players had a way of naturally drifting the mouse over a thing they might want to interact with, and when they did, voila — there were the actions. No clicking necessary.
Except of course that mousing over the correct things was often a matter of intensely frustrating pixel hunting, no matter how forgiving the raycasting hulls were. Then there was another guessing game of exactly how far outside each button you could go before the popup menu disappeared again and so on. Even the most basic actions began to feel subtly (and not so subtly) like defusing a bomb. Don’t cut the green wire, do cut the red wire, etc. And hey guess what, it also maps really poorly to touch screens and gamepads, neither of which are particularly accommodating of mouseovers or rollovers or temporary highlight states.
So what now?
Well, one nice thing that happened over the course of these faux-modal phases of UI design is we introduced some new UI elements in other parts of the game that opened some new doors. For example, we had added the ability to draw a little outline around certain 3D objects, to encourage players to mouse over them to reveal those popup menus. When we started to circle back around to the idea of well maybe modal UI really is the ticket here, it was with the knowledge that we had a simple and intuitive way of calling out currently-clickable objects. We had a way of telling players “hey, if you click here, you’ll get a little modal thing”. Setting expectations so that players can cognitively prepare for a mode change is pretty great.
So finally we were asking ourselves the question we probably should have asked a lot earlier: what if instead of this sort of halfway-fake-modal-mouseover behavior, you just clicked on a thing, and we treated our fake modal popups as actual modal popups?
After rolling the idea around for a few months while we worked on other parts of Overland, we started looking at what we wanted to have ready for PAX West this September, and having the game on gamepad was at the top of the list. Something about pixel-hunting with a mouse and trying to play 600 games in four days just doesn’t mesh well (weird i know), and we wanted to make a better connection with players. We started looking at this modal stuff more, and looking at our UI source code more, and finally seeing some cool opportunities.
Similar to the way we could use that extra data about “well where’s the mouse pointer”, these modal popups could be cues for a lot of little things. We could hide some of the inapplicable system buttons (you probably don’t want to end your turn while deciding whether to push or pull this obstacle, for example). We could maybe procedurally push the camera in a little bit. We could fade some vignettes in to focus things more. We could turn off a handful of other unnecessary UI elements to get clutter down more. And so on and so forth.
Modal popups aren’t a magical solution for every interface problem, but here are some of the ways that we ended up making them work for us:
- Clearly show whether clicking something will pop up a menu
- Keep the same controls as normal
- Keep most of the same context / view as normal
- Find the most humane order of operations
That last one was a minefield for me. For example, we wanted players to be able to trade inventory items between squad mates, but with gamepad in our sights, drag-and-drop seemed out of the question. The first idea I had was to make a kind of second-level-down extra modal state, where after clicking a character, you’d then click an item, then click a destination. This was a great idea until I realized it wasn’t.
The second idea, which I actually implemented and shipped in a patch last week, was to make clicking on the currently selected character enter a kind of inventory mode. In inventory mode you could click on your own items to drop them or pick them up, and click on neighboring survivors or vehicles to give them items or store them in the vehicles. It was cool, didn’t require that like weird third modal targeting whatever, so I committed the code and went to bed.
The next morning, Heather immediately pointed out that this was a really jarring and weird way to trade items around in this game, because all the other modal situations in the game were based solely around the thing that you’d clicked on. E.g. if you click on a dumpster, you can search it for items. You don’t click on your character and THEN the dumpster. So we switched it around — now, if you want to store an item in a vehicle, for example, you click on the vehicle, and then click on the item you want to store. No third mode necessary, and no more diverging from the clicked-thing-centric approach of the other interactions. Probably you could do it another way in another game, but for Overland this was clearly it.
Anyways, that’s how some solid little modal popups totally saved Overland last month, which was about the least likely scenario I could imagine for most of development. Never forget to go back and question those old assumptions — if enough time passes, your project will have shifted out from under some of them, and you might have some new (old) tools at your disposal.
Idea Two: Unified Screenspace Selection
Oh lordy that sounds like the title of a computer graphics paper, but I promise you it couldn’t be farther from that. Well actually no it could of course obviously but still.
So when we were vetting switching to modal menus, the go/no-go point was whether or not we’d be able to simplify and unify our game controls, both from the dev side and from the player side. Remember the goal here is to make Overland easier to play, not harder, even though we were adding new inputs and buttons and whatnot to the mix. So as much as possible I wanted a gamepad’s directional inputs to act a little bit like a computer mouse, but not in the reverse-touchscreen-dual-sticks twist of “hey you can push literally a mouse pointer around with a flippin thumbstick” thing.
But also the goal was to kind of make it feel a little like a mouse, albeit a smarter or more magnetic mouse. This was inspired by the new Apple TV of all things — nobody plays games on that thing bless their hearts, but the basic interface is this kind of swipe-y / trackpad-y thing that just pops from button to button, and it mostly works great (we won’t talk about their alphanumeric on-screen keyboard widget). I don’t know who Apple lifted this design + approach from, so apologies to whoever actually came up with it!
But how’s that going to work in Overland though, in practical terms? In terms of what literally do I have to program next to get this to work? Because sometimes you’re trying to select grid spaces (more on this isometric nightmare in a bit), sometimes you’re trying to select a 3D object in the world (like a stick a character is holding), sometimes you’re trying to select a UI button floating in space way up above the 3D game space, and sometimes you’re even trying to select buttons that are projected onto a 3D mesh (like on our map UI). Sometimes you’re trying to navigate between three of these things at once. And it needs to feel seamless.
On mouse this isn’t really that difficult. You do a math thing called a raycast (pew pew), which is essentially shooting a laser (pew) into your 3D scene, and checking to see what it hits. For small objects that are annoying to laser, you can put them inside a kind of invisible bubble or hull, and the raycast might hit that instead. Either way, you can figure out relatively cheaply what 3D object a player is mousing over. UI buttons are even easier. Grid cells are basically the same as the first method there, with a extra step or two. Buttons projected onto a mesh use an extra like 11 steps, but still it’s basically that.
There are a couple problems with this approach though. One, again, it maps very poorly to gamepad and touch-screens. Not the raycasting math itself, that’s fine. It’s that constantly tweaking these bubbles or hulls to give players a little click-slack but not too much, combined with the fact that usually most of the screen is just unclickable, ends up feeling like a hassle, for developer and player. And at least for Overland it never really worked all the way; mis-clicks were absolutely rampant, especially at public shows.
So maybe just do different selection systems for different kinds of things right, like play to the strengths of the different layers, essentially, and just make it custom. For example, just let players use a d-pad or something discrete to select which grid cell they want to select. Except Overland is isometric, so which equally-diagonal direction do you count as up? Players are notoriously 50/50 allergic to whichever direction you say is up (unless you say down is up, then well it’s pretty much 100% allergic. Why would you even) and to make matters worse, maintaining all these different systems is a recipe for a slow painful death on a small team with limited resources. Plus it goes against some of our modal concerns; remember, we want the controls there to be as consistent as possible so switching modes isn’t too jarring.
Fortunately there’s another option. What you can do is kind of the laser pointer thing but in reverse. Point a laser at the thing, but mark off where the laser crosses the actual screen. And do this for everything — buttons (easy, they’re already on the screen), 3D objects (also easy, a couple matrix transformations), grid cells (same, basically free), and those lousy buttons that are projected on a mesh (this is actually really gross and I won’t go into it here). So, mostly pretty easy to implement. Except the projected things. Those were bad. But anyways you end up with basically a bunch of points on the screen that are really easy to work with.
Basically you check which of these screen points the mouse pointer is closest to, and that’s it. Everywhere. On every interface in your whole thing. Then you hook up the gamepad’s direction input to push an invisible mouse pointer around until it finds the next closest thing to select. Or let players use WASD to do that for whatever reason. By mapping pretty much anything and everything in the game to a point on the screen, you can unify the way things are selected and clicked on, but still have some control over exactly how the reticle moves or the selection itself occurs, depending on what hardware players are using. And we also avoid the nasty isometric directional thing — diagonal thumbstick pushes naturally select the next cell along that diagonal track and so on.
Some things to keep in mind if you’re considering a similar approach: Overland is a basically top-down game where things rarely overlap each other on the screen. And we use these modal… modes to constrain the kinds of things you can select at any given time. Also a lot of our selectable elements are basically square — this approach won’t map quite as nicely, especially for mouse users, to shapes are that are super elongated. For mouse users, it can feel weird to always select the closest object, especially if it’s quite far from the mouse — you might want a kind of maximum effective radius there.
And Now All The Things Are Good
Well not all the things, certainly, but many of the things are pretty dang alright now. Under the Overland hood, anything selectable gets dumped into the same mechanism and converted to a screenspace position. Those screenspace positions can be highlighted or selected by a mouse or keyboard controls or a gamepad or a trackpad or whatever. The only raycasting we do is to figure out the worldspace position of the center of each map tile. This new selection layer of Overland can’t even tell the difference between a UI button and a 3D model, and gives us a level of uniformity in interactions across different hardware interfaces that I’m very, very pleased with. Plus, it’s code that we can maintain easily despite our small team.
Anyhow, the easiest place to follow dev right now, in between these beefy newsletters, is on the official Overland twitter. We’ll be keeping a relatively low profile for the next few months while we build our next big milestone, but patches and release notes should be pretty steady over the next few months if that’s your jam.
Thanks again, and as always, safe travels ❤
