Decluttering the HUD: the case for audio-driven navigation

The GPS technology that’s commonplace in today’s phones and cars creates a case of life imitating art. Much like the mini-map in Super Metroid, which would chart Samus’ location in real-time and mark places of interest like save points, services like Google Maps let pedestrians, motorists and cyclists see where they are, what’s around them and how to reach their destination on a digitised lay of the land. Not to be outdone, many modern games use far more abstract means of directing players, with brightly coloured arrows floating through the air (but magically unseen by NPCs), pointing at anything worth paying attention to. Not exactly subtle wayfinding…

Granted, Google Maps is quite an invasive system itself, having access to the location data of millions of users around the world, but at least it doesn’t insert gigantic floating numbers and symbols onto your eyes (a role reserved for the ill-fated Google Glass). It’s all too common for games, however, to clog the screen with a busy HUD, making gameplay start to feel like you’re simply following blips on a map — just look at every Ubisoft game from the past five years. Worse yet, check out this screen from Horizon: Zero Dawn.

The horror.

My guess as to why HUDs have become so crowded in recent years is because game worlds have enlarged drastically thanks to the popularity of open-world titles. Having developers ask players to familiarise themselves with a region the size of Assassin’s Creed Syndicate’s London would likely be too large a demand, so instead they rely on non-diegetic visual cues. I think this is a cop-out, though: if a game world can’t be remembered, perhaps it’s… not very memorable. Maybe the streets are too derivative because of the endless reuse of assets — a necessity due to the expectation of gigantic maps.

Given this expectation, I can hardly blame developers for having to rely on blips and arrows to guide players and ensure they don’t get lost. But why only visual cues? We don’t traverse the real world using sight alone; our other senses, especially hearing, also play a vital role. Yet for some reason, audio is rarely used as a means of navigation in games.

Note: this article is the second in a two-part series on listening in video games. You can also read the first part, ‘In Persona 3 and 4, sometimes the wrong answer is right.’

It’s strange, considering that game and sound designers largely seem to be aware of how they can use audio to declutter the screen. For example, it’s incredibly common to find games that eschew explicit health indicators, instead communicating damage by draining the colour from the visuals and drowning the music with the thumping of a heavily beating heart. The Bioshock titles use music to hint at enemy presences — the more intense and busy the arrangement, the more outnumbered you are, somewhat eliminating the need to show opponents on a radar. Game creators know how to replace UI elements with audio; they just don’t do tend to do it when it comes to navigation.

That’s not to say that none have tried, but sometimes the approach has been flawed. Look at The Last Of Us, for instance: sound plays a large role in the game, requiring players to move around silently to avoid detection, and to track enemies via the sounds they make using the protagonists’ “listen mode” skill. Naturally this has a use in stealth-oriented combat, but it also has a navigational function, allowing players to determine routes that are free of attackers based on which areas are free of noise. This wound up still being a visual cue, however, because sounds were communicated onscreen in an echolocation-like fashion. (Thankfully, the game’s most brutal difficulty settings disable listen mode, making your own hearing a necessity).

Metal Gear Solid 3 also missed the mark. Director Hideo Kojima removed the “Soliton Radar” mini-map that had been a staple in the previous two titles, partially to avoid a plot inconsistency (the technology isn’t meant to exist in that game because it’s a prequel), but also because it would force players to rely on their own spatial awareness — a great opportunity to make listening an essential component of moving through the game world. However, of all the navigational gadgets made available to the player — such as the motion detector, the anti-personnel sensor and the active sonar — only one was built around following noises: the directional microphone, which could be used to determine the general direction of an enemy based on their footsteps or voice. Sadly, that gadget was so clunky to operate that players tended to only use it when it was absolutely necessary.

Even when being used effectively, though, the directional microphone would underscore a glaring issue with sound-driven wayfinding: perhaps the reason audio cues aren’t commonly used for navigation in games is because it’s difficult to convey a specific location through sound alone. If you hear the roar of a motorcycle through your television speakers but can’t see the motorcycle onscreen, how will you know which direction it’s coming from?

While many modern home entertainment layouts are designed to surround a person with speakers, making it easier to create the illusion of a three-dimensional acoustic space with sounds generated from specific directions, not everyone has such an elaborate setup. Even for those who do, what if they don’t place the speakers in the correct spots? And while positional audio can be created through headphones by using binaural recordings, forcing players to exclusively hear the game through headphones and not speakers would be overly restrictive.

Ultimately, navigation using audio alone is impractical, just as using HUD elements alone is messy. But just like when moving through the real world, our senses should be used in tandem. Sounds may not entirely remove the need for maps or signs, but they do highlight things we can’t see — the whirring of air in a tunnel suggests a train’s impending arrival, and the blaring of music means you’re getting close to the location of that birthday party. It’s easy to imagine how such an idea could be applied to games: if you hear running water, you’re close to a river; if you hear traffic on a freeway, you’re approaching a city; if you hear rustlings in the bushes, there’s a wild animal nearby.

The Sly Cooper titles of the early-to-mid 2000s managed to pull this off, even without the wonders of modern home theatre arrangements or binaural recordings. They instead used a simple but effective approach resembling a game of hot or cold — the closer you got to the location of a collectible clue bottle, the louder its wobbly sound would become, making players chase down increasing volumes to find treasure. In this case, sound was used as both a tell and a means of guidance.

The beauty of these sorts of audio cues is that they’re diegetic. Unlike HUD elements, they don’t break the fourth wall through their mere existence; instead, they simply add to a game world’s realism by building off sounds that should exist no matter what, potentially making the experience more immersive.

It would be remiss of me to ignore the fact that making games more reliant on sound could make them less playable for the hearing impaired. The idea that games should be less practical for any group of people seems a bit tone deaf (pardon the pun), given that the video game industry seems to be becoming more mindful of players with disabilities. Many titles now provide options to alter colour palettes (a godsend for those suffering from colour blindness) or remap buttons (which can help anyone who is unable to play a game with its intended control scheme). And perhaps that’s the answer: options. While I might have found The Last Of Us’ visualised sounds to be purpose defeating, I can imagine they were incredibly helpful to those with hearing issues.

In no way do I think the HUD should go away — audio can’t completely remove the need for onscreen elements, and there are obviously people who rely on them. But that doesn’t mean developers should avoid audio cues altogether when it comes to navigation, considering their potential to reduce clutter and increase immersion. The world is full of signs and symbols that aren’t strictly visual in nature — it’s about time games start catching up.