Synescape - an immersive real-time music visualizer

I am happy to announce the release of synescape, a music visualization app I have been working on for the past couple months. The process began when I started sitting in on jam session with a some friends playing guitar and drums. My original time with them was spent working on a deep learning project, where I trained a neural network to recognize instrument sounds. After that project wrapped up however, I became interested in visualizing the music that was being played in realtime.

To paraphrase the 20th century phenomenologist Merleau-Ponty: we are all a little bit synesthetic. Even if we don’t explicitly see colors or experience tastes when listening to music, the potentiality for such experiences is always latent within our perceptual systems. With a little bit of help, such experiences can be brought into focus. Synescape takes real time microphone input and creates colorful displays with it.

Synescape also includes support for Cardboard VR, allowing for an immersive and portable visual music experience. Just press the cardboard button at the top of the screen to switch between immersive and regular visuals.

This was a bit of a passion project done in my free time, so I hope there are other people in the world who will be able to enjoy it in addition to the fun my friends and I have had with it. If you happen to try it, let me know what you think! Along with the downloadable versions for Windows, Mac, iOS, and Android, I am also releasing the source code on Github, so anyone who wants to build on this work can be free to do so.