Relive 4192 songs with WebGL and WebAudio (in VR if you must)
In 1968, the Swiss radio channel DRS 3 presented the nation’s best-selling music pieces for the first time — an extremely successful broadcasting format was born, the «Hitparade». Every sunday a good part of Switzerland’s people would be listening to the music everybody else around them loves.
Now, 50 years later, D-One Solutions, good friends of Hinderling Volkart, have been approached by the Swiss Radio and Television Company to dive into the many songs and ranks and analyze the hell out of it. But they also wanted people to experience this data in a more explorative way, which is where we entered the game.
We’ve come up with a highly interactive 3D world that will allow visitors to grasp the enormous data and the sound behind it by simply swiping their finger.
«Too Much Heaven»
Apart from logo and lightboxes, the entire user interface was implemented using WebGL.
We’ve certainly been challenged by the considerable number of 3D objects. 4192 songs with 36'215 rankings are included in the 50 years of Hitparade shown. Every star in our 3D scene corresponds to a week’s ranking and has been implemented as part of a particle cloud. Thus the rendering is handled by custom designed shaders that are optimized for performance. Also the lines are very rudimentary WebGL objects, which might be somewhat limited in their application, but show a good speed. We also took some burden from the 3D-Engine by using our specific knowledge of the data behind the objects: we know the period of time shown (i. e. the year 1973) and can decide very efficiently which objects might be visible or not (for example songs from 1984). Thus, the engine does not have to carry out the camera projection for every element(which always means multiple matrix multiplications per object) to calculate the visibility.
A 3D scene with that many dynamically rendered textures is quite a challenge to implement without sacrificing performance or quality.
We applied the same principle to the most complex objects, the text labels. As there are no texts in WebGL, we’re dealing with transparent textures that are created dynamically via canvas operations. Thousands of songs also mean thousands of textures and many thousands of drawing operations. This is not possible with conventional 3D scenery. It would bring almost any computer to its knees. Instead, we create the textures “on the fly” and discard them when the object disappears from the visible area. Instead of creating and deleting the canvas elements required to create the texture each time, we use a pool that recycles the objects. For performance optimization, this pool returns a canvas that already has the correct size if available (the sizes are relatively uniform, since each dimension of a 3D texture must always be power of 2, i. e. 256x128. Only these measures allow smartphones to play a website so complex and rich.
«Hit Me Baby One More Time»
The large number of objects is also responsible for another challenge: in order to find the object under the mouse, very complex calculations are necessary. A ray (Raycast) is placed in the scenery in such a way that it lies right under the mouse from the camera‘s perspective. This straight line is now intersected with all objects in the scene: whenever an intersection occurs,it’s added to the candidates — sorting by distance will eventually return the searched object.
Not only is such a raycaster approach slow, it’s also not applicable to our stars which a are points without dimension in our 3D space. For this reason, we’ve created optimized “hit testers” for both the stars and the labels. Those guys find the matching object (or the referenced song) very efficiently and allow to work around limitations of a conventional raycaster. Like those labels that have a visual size that doesn’t correspond with their object size, because the dimensions had to be rounded up to 2^n as already mentioned. We therefore calculate a sub geometry to carry out the test.
«The Sound of Silence»
While playing some MP3 with HTML Audio element most certainly is no witchcraft, we’d be confronted with some limitations with that approach: on iOS no sound would be played automatically, and we’d have very abrupt switches between the songs.
The WebAudio API provides a Gain node that allows smooth transitions between audio pieces.
The WebAudio API provides powerful tools to circumvent this: via audio nodes we can create complex audio routing graphs. Our graph is rather simple knitted with just a gain node to neatly fade between songs. We’ve also had to make sure that the async loading mechanism (loading and decoding) would not interrupt the user flow. If the browser supports no WebAudio API, but WebGL, we fall back to simple audio elements and do without nice transitions. This only concerns IE11.
Mobile devices not only have a smaller screen — we interact quite differently with a smart phone than we do with a laptop or desktop computer. The obvious game changer is touch versus mouse, but there’s also screen distance and the device position (in your hand versus on a table). Which is why in our 3D charts universe we’ll use the mouse to scroll through time and to look around. We’ll change this behaviour on touch devices though: the user now scrolls with the finger (we really like to keep the native scroll experience) but will look around by moving the device. It’s a little bit of VR and really just makes sense here to use those sensors. Still sometimes a user might get lost because the orientation changes too much (like when you hand over your device to somebody else). We will have a little button appear whenever that happens that allows you to recenter the screen.
VR if you must
We’ve added a small feature for people that are lucky enough to own a cardboard for their mobile phone: you can switch to a VR mode where we split the screen and have two slightly different projections rendered. We also change certain interactions (you can now press anywhere to move forward, often the only screen interaction possible with cardboards). This allow to dive deeper into this rich universe of many songs. Songs that carry so many memories for most of us.
Visit the website: 50-jahre-hitparade.ch
If you also want to be part of a project like that, don’t hesitate to apply for a position with Hinderling Volkart. We would love to hear from you!