Music and Experience Design

There are many good articles out there on the internet that talk about designing music for experiences, the influence of background music on video games and how people play better or worse with certain sounds or music. This is NOT going to be one more of those.

I have been playing the violin for as long as I have known. As I started giving stage performances, many a time, it would be frustrating to see people leave in the middle of the show. Although the tickets are pre-bought, it would affect you in the long run as they wouldn’t recommend other people to go to your next concert. Over time, I realized that, to make it as a successful musician, it is very important to read your audience continuously and cater your performance to each of them. There are connoisseurs of music who understand every technical aspect, senior citizens who just want to relax and have a good time and restless kids who have no clue what’s going on. You have to get the pacing and timing of your pieces right so as to keep everybody interested through the concert. Thus, eventually, you develop a sense of understanding as to how to sustain attention.

As an aspiring designer, I see a lot of relatedness between music and experience design. Both composition and performance aspects of music have a lot of learnings that can directly be translated to designing an engaging experience. This blog is about what I think, are a few parallels between the two and how some games have used elements from music to create engaging gameplay.

Rule of threes:

Keeping your music interesting is a much more difficult task than most people realize. There is a very delicate balance between a piece of music so boring that people start to doze off, and something so complex that the listener can’t keep up with. The trick is to find a balance between repetition, so that the listener has something familiar they can grab onto and follow, and variation, so the listener feels challenged to keep up and surprised by the direction the music is taking. Western Classical music employs the Rule of Threes to address this. Rule of threes gives us the smallest number of elements to establish a pattern and then keep the listener engaged by breaking it. Legendary musicians like Mozart and Vivaldi have made use of this. Let’s look at an example:

Listen to this: Spring-Antonio Vivaldi

Vivaldi

Vivaldi’s Spring starts with a simple one bar motif (1), then repeats it verbatim (2). This establishes a pattern in our minds, and unconsciously sets up our expectations that we’ll hear the pattern a third time. He defies our expectations, however, and takes the melody in a new direction (3). By the end of those (2) bars we’ve heard the same thing twice. So Vivaldi waits just long enough to take us in a new direction.

This has been used successfully in Games like Mario and in many other games created by Nintendo. Look at the last corridor of the first castle, before you meet Koopa for the first time:

Mario — Rule of threes

Rather than just let the player deal with Koopa and meet instant death, the player is given a short training course on fireballs. First, a fireball comes straight at you in isolation. Then, the terrain changes and the fireballs come at different heights. Finally, the fireballs start to come more quickly. The player has been “primed” for battling Koopa before the fight has even begun.

Four Level Structure:

Indian Classical Music has a four level structure in its improvisation component. Improv generally starts with an introduction (usually a grand opening that sets the stage), then there is a development phase (sustained pacing), there is a twist in the middle (change of pacing / timing / progression ) and the piece goes to conclusion. By doing this, the piece starts by setting the expectations, brings people to a comfortable state, breaks it and then eases them back towards the end. This provides for a great interest curve and keeps the listeners engaged throughout the piece. This is very similar to Chinese poetry “Kioshotenketsu” which uses the same four-line structure.

This structure has been used in many games to introduce players to new interactions. For example Super Mario Run has used this structure to implement in World 1–2 “Wall-Kicking It Underground”. You can read more about how it’s done here

Concert Structure

In the ancient times, it is believed that Indian Classical concerts used to go on for ~ 6- 8 hours with no definite structure, as they used to be side events during festivals and celebrations. As concerts began to take the main stage, with people paying for them, musicians realized that there needed to be some structure to keep the listeners engaged. In the 18th century, one particular gentleman from South India, named Ramanujan Iyengar, put forth a structure with appropriate pacing / timing to maintain interest throughout the concert. His format builds up expectations , culminates in the improv section and eases towards the end. However, at the very end, he ends with a fast paced grand piece that raises the audience’s expectations again, ensuring they come back to his next concert. This structure proved to be so successful that musicians started using this a basic template for all their concerts. Even now, after more than 200 years, this format is being used by Indian classical musicians.

Indian Classical Music — Concert structure (Interest Curve)

Alternating between moments of tense and release, building up the experience to highlight certain moments, ending it with a bang, to set expectations for possible sequels are all learnings that could directly be translated to designing an engaging experience.

Making the Interface Fun:

Various musical instruments have completely different interfaces and have totally different learning curves. But the one common thing about all of them is that they are inherently fun to just dabble with. For example, you don’t really have to know the intricacies of playing a piano to have fun with it. Just tapping the keys and listening to the sounds is fun by itself and makes people want to play with it all day long. This directly applies to design a game. Making something that is fun in and of itself and allowing the players to have fun, even while not following the goals of the experience, would make it accessible to a larger portion of the audience than we intend to reach. The best example of this is rubik’s cube. The toy in itself is so much fun to just rotate the colors around.

“All our knowledge has its origins in our perceptions.” — Leonardo da Vinci.

In my opinion, there’s no such thing as “correct” or “ideal” design; there are just rules of thumb, which may or may not apply to solving the problems of the specific game we are designing. Based on the specific experience we’re trying to give people, it would benefit us designers to look everywhere else for inspiration to create something that leaves a lasting impression on people.