Being fully present at a moment and paying attention onto ongoing processes in the surrounding environment always fascinated me. The act of listening is inseparable from the act of sound making (at least, it is the absence of making it). Also, contemplating on a beautiful painting takes part in a process of recreating an internal image on the observed subject. This eternal conversation between perception and cognitive responses is part of our everyday life, if we listen carefully, we can recognise that there are so many changes happening while we are just sitting on the ground and watching birds.
Drawing is a way of representing the world, but it also acts like a direct translator for new ideas and thoughts. I like to refer to the so called Magic Circle a lot, where one draws a circle (defining a set of rules), and everything that falls inside the circle, will become a playground of a game, while artefacts outside the circle will not play any role in the game. We can think of the world as nested networks of magic circles, where each circle defines different rules for different situations based on social, biological or other abstract consensus.
Instead of generating new ideas, the medium of sound or music is much closer to some sort of meditative act that resonates our attention with patterns and rhythms of natural frequencies. It directs us to the understanding of time, sustainable and non-sustainable cycles and repetitive rituals. It is not only presented in the audible domain, it is embedded into very slow movements, celestial transformations, digital algorithms, network topology, music of the spheres and anything that has a relationship with time.
This dialogue between defining new ideas by spatial magic circles (drawings) and observing temporal rituals through sound and time led me to the creation of SoundBow, a visual instrument that helped me understand a lot about play, design, distribution, failure, production, communities, and creative curiosity in general.
Since I've been an “in-between” type of person — I switched my studies in high school from music education to visual arts, then from graphic design to time based video art in the university, just to name a few transitions within my interests — it became natural that my intentions in research and aesthetic practice should combine somehow these two areas, so I made my Phd afterwards on the topic of Visual Sound Instruments. I researched basic interaction principles within sonic environments that have resonating bodies (such as string instruments, wind instruments, membranes, etc) and sonic systems that do not have resonating bodies, only abstract digital descriptions and interfaces (software synths, live coding environments, electroacoustic gadgets). I was an active member of the live coding community at that time (today the scene is mostly called Algorave) and was trying to understand how writing music (thus, code and notation) can be combined with live performance. I was using the visual programming language called Pure Data to build up complex sonic structures from a blank canvas. In fact, this language is still used within SoundBow as the sound engine for recording, manipulating and playing back sounds within the application.
There are some cases, when one becomes the writer (composer) and the performer (interpreter) of a piece at the same time, such as when you take part in a fluxus piece, or you are practicing Deep Listening. There are many cases where you can find opportunities for this state in your environment, based on your interests and experiences. Back then, when I started thinking over these possibilities, a new form of medium was unfolding: the world of touch based devices. These tools became shape shifters of our time: you observe while manipulate a graphical notation in the same interaction cycle, and the visual symbols can turn into anything while you are interacting with them. You become the performer and the audience in a very intimate journey that can be limited or very open, depending on the intentions of the instrument designer. The balance between constraints of a system (where this system means every component and relationship inside the magic circle) and the amount of areas left open for creative experimentation and unintended accidents plays a key role in designing these kind of cognitive computational experiences.
Another important aspect of these recent devices is that they are part of ecosystems where content distribution differs from earlier supply chains completely. An indie artist and developer can make a product grow from the seed of the original idea into the hands of the audience without the need to really deal with logistics in their workflow during the distribution of the piece. Again, this model needs a bit of the already mentioned "in-between" mindset: you become the artist, the developer, the producer, the designer and everything in between, which can raise truly challenging, sometimes frustrating, yet very inspiring moments for the most of us.
Variations, in depth
I made the first incarnation of SoundBow in early 2012, when I was trying out nonconventional user interface ideas within the Processing programming language. At that time, it became possible to combine my favourite languages together (Processing + Pure Data) and play with the results on touch based Android devices. Soon I made a prototype and published it to the Android store (now called Google Play), where it is still available to download.
That year I had a project in which I had to program some sounds and graphics on a Raspberry PI, which is a small, cheap linux computer with a bit lower processing power than laptops and PCs. Since performance, nice graphics and high quality sounds were crucial in the piece, I started using OpenFrameworks for application development, which is a very similar coding platform to Processing, but is built on C++, so low level, native operations and improved graphical performance can be accessed in a handy, flexible way. Since my favourite sound programming language (Pure Data) could be included in the compiled app bundle using libPd, I could continue my workflow: completely separated visual interface development combined with an interchangeable, independent sound engine. This was the time I discovered that OpenFrameworks can be used to build apps for the iPhone and the iPad, too.
Most musicians were using iOS by that time, sound performance, connectivity, digital signal processing and the creative community was insane and completely next level compared to the Android world. So I decided to make SoundBow for iOS, but with a redesign, where the players can do a lot more things with it: record small samples of their own voice, play with those in a more open way by reconfiguring elements around their drawings. Also, a very important option has been added to the experience: people were invited to export their compositions and share them with their loved ones or include them into their professional workflow as high quality sound recordings. The first release was in the mid of January, 2016. I remember well, since it was just after the death of David Bowie. Talking of him, let's pause & rewind for a moment and remember what he said in 1999:
"I don't think we've even seen the tip of the iceberg. I think the potential of what the internet is going to do to society, both good and bad, is unimaginable. [...] It is an alien life form [...] that is going to crush our ideas of what mediums are all about."
This rhymes well with the distribution model of the app stores, as former MIT Media Lab professor John Maeda highlights these analogies in his CX 2020 report. Indeed, SoundBow could reach more than 100.000 installs on Google Play, and reached more than 13.000 people on Apple’s App Store (13.1K unit downloads as of writing this article). These numbers might not be impressive in the world of widely available consumer software products and utilities made by giant companies, but they have very different meaning in a niche, experimental art “market”. It would have been impossible for me to bring an experience at this level of intimacy for the audience without computational (freely scalable, location-agnostic, omni-presented) distributional models such as these stores. Beyond apps, these systems today are applied for most distribution channels from games through music and digital entertainment in general. Content and context are both going over a previously unimaginable transformation by this alien life form that we are interacting with on a daily basis.
Being an "in-between" indie developer, I had many hands-on experience with digital distribution and the nature of a sustainable software. The immediate responses through app reviews and social media took a lot of energy out of me. It took time to understand and deal with the psychological consequences caused by that amount of influence. There were special lessons I learnt while tracking and observing the lifecycle and the effects of my app. One of the most intense experiences was the realisation that there are clones appearing in the app stores that are mimicking the appearance and interaction mechanics of SoundBow. These clones caused quite shocking feelings sometimes for me, in which I did not always know how to react accordingly to these situations.
Another interesting lesson I learnt is when I tried to bring SoundBow to the studios of professional musicians as opposed to regular people who are just playing around with their sounds and drawings and use the app as a toy. By integrating a third party service called AudioCopy and adding support for MIDI output among other music production related solutions, I was trying to fulfil feature requests that led to infinite nested rabbit holes in the development process. Newer versions brought forward more possibilities to errors, and the high-level expectations from musicians were impossible to manage as being only one individual developer and not a dedicated music software company. Realisation of the hardest part of it just arrived later: going into these specialised directions turned my attention away from my original intention, which was to provide a smooth, creative flow for playing with sounds and drawings by people who don't make music or drawings otherwise. It is an inclusive goal at its heart and now I know I have to take it seriously.
Identifying this and understanding the lessons I had (both bad, confusing and good, exciting) during the last almost 10 years, I can now continue taking care of SoundBow and its future variations (I already have some strange ideas to try) and I do it for your convenience and delight. Thank you for being on this journey with me. If you wish, download or update SoundBow from the AppStore, or just go out and practice some cognitive sampling: take your time and listen to the myriad beautiful sounds of the surrounding world.