What Are Virtual Instruments
When code and music overlap: how they are built and played in order to create realistic sounding music.
A lot of the music you can hear in today’s tv series, film trailers and video games is composed and recorded using virtual instruments rather than being played by a live orchestra. But what are these so-called virtual instruments and how does that work? If you are wondering about things like MIDI, sampling, and more, I’ll try to walk you through these concepts in the following article.
Synthesis and sampling
When talking about virtual instruments, we usually mean software instruments. They are software programs that emulate the sound and characteristics of real musical instruments and are played using a computer. They come in various shapes, sizes and forms but generally speaking, there are two major categories:
Synthesized instruments
These are a collection of algorithms and mathematical formulas that will attempt to recreate the sound of a real instrument. Although being really flexible and frequently used for sound design scenarios or modern electronic music, they don’t quite sound like the real thing. Most people will be able to hear that they are not physical musical instruments played by an instrumentalist. Here is what a synthesized strings section sounds like:
You’ll agree that this doesn’t sound very convincing. Synthesis definitely has its uses in modern hybrid orchestral music (look up Hans Zimmer) but this is not what this article is about. We will concentrate on the second category shortly instead. Here is what a synthesized instrument can look like:
These instruments are usually called synthesizers or simply synths.
Sampled instruments
This second category is a whole different beast. Software sampled instruments are built using real instruments played by real musicians and then assembled by software developers. They are usually called sample libraries. Their main goal is to emulate musical instruments as close as possible and give the listener the illusion that they are actually hearing a real player. We are going to look at these in more detail in the following paragraphs but just have a listen to the following example of a sampled strings section playing the same chords as before:
Now, that sounds better, don’t you agree? This is what the interface of this particular virtual instrument looks like:
Let’s see how this magic happens…
First step: recording
When a software instrument company sets out to create a new virtual instrument, they first need to record one or more real players. They’ll go in a studio and ask the players to play one note. Just a single note. Let’s say a C note. The recording of that note is what is known as a sample. Here is a C note sample taken from a cello section sample library, 4 players playing simultaneously, (8Dio Century Ensemble Strings, $448):
But a musical instrument can usually play more than one note, right? So we need samples for them as well. That’s why each note of the range of the instrument will be recorded. Here is the same section of 4 cellos playing the C major scale. There’s one sample for each note:
This is what a sample library recording session looks like (photo courtesy of Spitfire Audio, one of the best sample library makers and my favorite too):
One sample per note? Not enough!
One of the problems that will arise quickly if you try to play a virtual instrument that only has one sample per note is that it is going to sound very mechanical and unnatural. No player playing the same note twice will have both notes sounding exactly the same. That’s why each note usually gets recorded multiple times. You start to get how long a process this could become. When played, the virtual instrument will normally select one of the samples for the desired note randomly. This way, it will make for a more convincing performance. Here is a violin section playing an A multiple times:
Can you hear how each note sounds just a little bit differently? These different samples for the same note are called round robins. This is something that you should check before buying a sample library as vendors will indicate how many round robins are available in a sample library.
In this screenshot of a violas section virtual instrument, you can see which sample is currently being played:
But what about the playing style?
Except for some, musical instruments can also be played in a wide array of manners. You can play short notes (staccato, spiccato, etc) and long ones (sostenuto, sforzando, etc). These different techniques are known as articulations. And you guessed it, they need to be recorded too. For each note. Multiple times because of the round robins that we need. Yes, making a good sample library is a time-consuming endeavor (and it doesn’t stop there!).
Here is an example of a strings ensemble playing different articulations:
All these sample recordings start to take some space on your computer. This is why is it not uncommon to have one virtual instrument occupy dozens of gigabytes of space and sometimes even hundreds.
In this screenshot, you can see all the articulations available for the 2nd violins section of the 8Dio Century Ensemble Strings virtual instrument:
Of course, this is also something that you should consider when buying a new virtual instrument: how many articulations are available in the library and how many of these you will need for the kind of music that you are writing.
Let’s see about dynamics too
Round robins and articulations are only part of the story for a convincing virtual instrument. In a musical piece, not everything will be played at the same volume but will be a combination of different nuances ranging from very soft (pianissimo) to very loud (fortissimo) with everything in between. These nuances are called dynamics. You can’t just simply change the volume of a sample to effectively mimic these nuances: the timbre of an instrument changes depending on how hard you play it. These will need to be sampled too. For each note, for each round robin, for each articulation. We are now in the thousands of samples for one instrument!
How many nuances are available in a sample library is known as the number of dynamic layers. It ranges from 1 to over 10, depending on which sampled instrument you are using.
Here is an example of a violas section going from very soft to very loud, for the same note using a spiccato articulation:
Some virtual instruments will also give you visual feedback about the dynamic currently being played:
Depth and acoustics: miking it up
This is not the end of the story. Not even close! We should now consider how sound is perceived when you are listening to a real instrument. A lot depends on how close to the instrument the listener is. Most professional sample libraries will let you make this choice by offering you different microphone positions. Yes! There will be a sample recording per microphone too!
Virtual instruments will let you adjust the volume of each microphone individually:
In the screenshot above, C-T-A-O means Close, Tree, Ambient and Outriggers. They are different types of microphones each placed at a specific position from the instrument(s).
Processing the samples
Some libraries even have samples for multiple types of vibrato, attack or release of the notes and so on. It can get very, very detailed in order to emulate the real thing. Some vendors strive to make the most realistic sampled libraries, it’s a fierce market and the arms race is on.
Once the recordings are done comes the excruciating job of processing the samples. Cutting them at the correct length, fixing glitches, checking them for unwanted artifacts. As there are thousands of them, this step is not an easy feat. That’s when audio engineers work their magic.
Enter the Geeks
We are now at the point where the instruments have been carefully sampled and each audio file edited accordingly. That’s when the programmer(s) enter the stage. Because all these samples need to be turned into an actual software instrument.
Software instruments are usually played using a MIDI controller. It’s a device that resembles a piano but its purpose is not to produce sound. It will send messages to the virtual instrument such as which key was pressed, for how long and at which velocity in order to trigger the correct sample. The virtual instrument produces the sound, not the keyboard. This is known as the MIDI standard (Musical Instrument Digital Interface). It exists since the early 80s.
It is the job of the programmers to map each sample to each key of the MIDI controller, build an interface that will allow the user to control the instrument such as setting the correct microphone volumes, dynamics, changing articulations, etc. In order to do this, they use a piece of software known as a sampler. There are several samplers on the market but when it comes to sample libraries, Native Instruments Kontakt dominates the landscape. Most vendors are using it. Kontakt uses a dedicated programming language. It’s called KSP (Kontakt Script Processor).
One of the tasks involved is to program the legato transitions between notes. Legato transitions are how two connected notes will morph into each other. Here is an audio example to illustrate what a legato transition is. It is made with a sample instrument using the voice of singer Merethe Soltvedt (sample libraries are not limited to musical instruments! Vocalists and choirs are also frequently used as software instruments). It is part of Audio Imperia’s Jaeger sample library ($599). First, you will hear two notes. They are not connected by a transition. Right after, you will hear the same two notes but this time connected by a legato transition:
Tell me that this doesn’t blow your mind! Come on! Legato transitions use a combination of recorded transitions and programming.
Once the programmers are done, we still need a way to use these instruments to record music. Let’s look into this briefly as well.
As a side note, it is also not uncommon to have famous bands or musicians participate in the making of their signature virtual instruments such as:
Spitfire Audio Hans Zimmer Strings
Native Instruments Alicia’s Keys piano
Embertone Joshua Bell Violin
…
Let’s play!
You can run most software instruments in standalone mode. That means that you can just launch one as any other regular program and play with it. The magic comes from being able to record them and play multiple ones at the same time. There are two distinct ways to do this:
Music notation software
Some composers use a music notation software (think word processor for writing music) and will assign a software instrument to each musical passage. I really like Steinberg Dorico and many others swear by Avid Sibelius.
Digital Audio Workstation (DAW)
My favorite way of composing music. Much like a notation software, your screen will be divided into tracks. Each track represents a virtual instrument or a group of virtual instruments that you can load inside of it. Hit record and play. I’ll write more about DAWs in a later article, this is a (wide!) subject on its own. I use Steinberg Cubase but there are many, many other DAWs that perform equally well out there. It all comes down to personal preferences.
The result
Here is a short cue I’ve made that uses only orchestral software instruments. No synthesis, only sample libraries. Strings, brass, percussion, woodwinds, choirs… the whole shebang:
Where to go from here?
You can have a look at the following sample library developers to get an idea of what is out there. You may also notice the prices of these libraries as you’ve understood by now, they take a very long time to make and require a lot of resources. These are some of my favorite developers:
- Spitfire Audio: https://www.spitfireaudio.com/
- Heavyocity: https://heavyocity.com/
- 8Dio: https://8dio.com/
- Orchestral Tools: http://www.orchestraltools.com/
- Soundiron: https://soundiron.com/
- Audio Imperia: https://www.audioimperia.com/
- Cinesamples: https://cinesamples.com/
I seriously hope that I have piqued your interest about the wonderful world of software music! From here, choose your path:
- If you want to learn how to write music from scratch and go up to a level where you can compose both cinematic cues and music for the movie trailers industry, my friends at Evenant have your back. Check them out.
- If you are already writing music but you want to boost your productivity, check out my upcoming course Building & Balancing a Modern Orchestral Template. Hardcore productivity boost for composers on a deadline.
- Programming virtual instruments is your calling? You will need to learn how to code first. Check out the EPFL Extension School’s Thinking and Creating with Code course that will give you the necessary foundational knowledge to kickstart your coding career (it’s a really good course. I know: I wrote it!)
In a future article, I will talk about what is mixing, the step that comes after composing and recording, and how it is done. Subscribe to know when it gets released.
