Announcing!.. The Neural Audio Programmer Podcast
It’s NAP time! Announcing The Neural Audio Programmer Podcast, where Joshua Hodge, Christian Steinmetz and I will be hosting a monthly roundtable discussion about the latest breakthroughs, innovations and news around Neural Audio and AI Music tech. We will be getting into the technical depths of the emerging technologies and sparking philosophical debates around their cultural and societal implications as AI Music technology continues to evolve. Furthermore, we will discuss the features and functions that we would like to see emerge as well as our predictions for the future. Will Neural Audio be a possible path to uncovering new music genres? To expand and challenge our views we shall bring guests onto the show that span a range of personalities and opinions. By sharing their story, knowledge and perspectives on various topics relevant to the field we will hopefully identify the challenges, concerns and potential impact of neural audio technologies on the music industry. The Neural Audio Programmer Podcast episodes will be released on the last week of every month through The Audio Programmer’s YouTube channel as well as on Spotify and Apple Podcasts.
In March 2023 we ran the NAP competition, previously titled ‘The Neural Audio Plugin Competition’, where we invited open-source submissions of VST/AU plugins powered by neural networks to compete for prizes supported by major players in industry and academia. The initiative was to help spread awareness of neural audio technologies to the audio software and music tech communities where many had not experienced or heard about this novel field of technology. Not only was the competition a chance for developers, researchers and artists to explore new audio technologies related to machine learning and AI, but it was an opportunity to address the lack of industry tools for music-making that incorporate neural audio technology. By increasing awareness through the weekly livestreams and by providing an opportunity for software developers, researchers and artists to collaborate and engage in this space we successfully broadened the community of neural audio enthusiasts and practitioners.
Qosmo had the pleasure of being a supporter of the competition this year, contributing workshops and talks on the Neutone platform and audio plugin to the participants and viewers that attended. Neutone, being a platform that helps bridge the gap between the neural audio research community and artists, provided an avenue for researchers to submit an entry to the competition through a Neutone compatible model. We were delighted to have received two competition entries from the community that used the Neutone SDK and Neutone plugin to deploy their model: Neurafuzz by Marco Comunità and ddspXsynth by David Südholt. If you are interested in exploring the Neutone platform for music production and/or research check out our resources at the following link:
The total number of submissions massively exceeded our expectations, coming to nineteen submitted projects in total! Initially, we expected a handful of submissions at most and had no idea that the initiative would resonate with so many. To our delight, we smashed this initial target, inspiring us to build on from this initiative and to launch the The Neural Audio Programmer Podcast! The podcast will allow us to communicate and educate neural audio topics as well as hopefully inspire others to explore these technologies.
Our judging panel for the NAP competition (consisting of well-known experts from industry and academia) definitely had a tough time picking from the nineteen exceptional submissions we received. However, in the end there could only be three prize winners where the top three demonstrated excellence across the six main criteria:
Novelty — Does the plugin use an innovative machine learning algorithm or application?
Utility — How much impact will the plugin have on improving creative workflow/ideas?
Character — Is the plugin unique or have character in the idea and/or UI?
Complexity — How much work was done by the team vs. built on existing APIs and frameworks?
Code Quality — Does the code appear organized and would it be possible to use via the README?
Demo — How well does the video and documentation demonstrate the plugin abilities?
1st Place Prize Winner
The 1st place award went to ‘Scyclone’ by Fares Schulz, Christian Scheer, Amirpasha Mobini Tehrani and Valentin Ackva. Scyclone is a neural timbre transfer plugin that converts an input sound into another kind of sound based on the data in which the model was trained on. An example of this neural timbre transfer effect is demonstrated in the following video but in essence it can convert sounds like voice to drums or saxophone to violin. You can also gain familiarity with these neural timbre transfer models through our Neutone VST/AU plugin. We were impressed in particular with the user-interface of this submission due to its sleek and compelling design. Furthermore, the UI design of this plugin attempts to address a critical UX challenge; how do we build meaningful and intuitive user experiences that cater for neural audio technologies.
2nd Place Prize Winner
Following Scyclone, as the 2nd place runner up, was the plugin ‘Neural Resonator’ by Rodrigo Diaz, Lewis Wolstanholme and Adan Benito. The neural audio technology that featured in this plugin models the behaviour of drum membranes and allows the user to deform the membrane in any configuration they want by changing the position of the vertices displayed in the UI. The way in which the vertices modify the sound is a great way to explore the sonic possibilities and provides a strong sense of control and understanding of how to use the technology. The various parameters presented in the UI expose a considerable amount of flexibility to the user and allow the synthesis to be adjusted in a variety of ways.
3rd Place Prize Winner
Our third place prize went to ‘HARD’ by Yiming Wu. A novel and hugely inspiring application of neural audio tech that addresses music transitions for DJs. The plugin provides the ability to decouple the harmonic and rhythmic components of two music tracks, allowing them to be mixed together by combining various amounts of each track’s harmonic and rhythmic content. Several of our contributors that are also DJs saw huge potential in this application and could foresee this function appearing in well-known professional-grade DJ software in the near future.
For more information on the details and results of the NAP 2023 competition check out the official website:
Our mission is to build on from NAP’s success to support adoption of these novel technologies by artists, researchers and audio tech professionals, bridging the gap between the communities and improving accessibility, understanding and inclusivity. Unfortunately, the AI hype and perceived threat has somewhat tainted the impression of “AI” technologies in creative applications, manifesting from ambiguity, fear and misinformation or lack of understanding amongst other reasons. We are hoping to debunk the myths and provide clear insights and transparency into what is driving these technologies under the hood whilst keeping it simple for non-specialists to digest and take forward in their own way. Although, we also hope those with expertise in the field will benefit from our discussions, we aim to make the information we share inclusive to those interested but not necessarily well acquainted with the jargon and low-level technical detail. Overall, we want to prioritise authenticity and identify the challenges that currently exist and that may arise. Please let us know your feedback! Your opinions will help us shape the content of the series and allow us to create more value for our listeners.
The first episode is out now (See below)! Play it back for a taster of what’s to come! This is only scratching the surface. Excited to finally launch this series for those who want to stay at the cutting edge, uncover the cutting edge and deploy the cutting edge. Tune in on the last week of every month for the latest episode of this ongoing Neural Audio Programmer Podcast series.
