A Creativity Support Index Analysis of Beat Blender
This work was completed by Jayden Soni and Thomas Young as a part of Professor Bryan Pardo’s course in Computational Creativity at Northwestern University. We used the Creativity Support Index metric to compare Beat Blender (a creativity support tool for rhythm generation built on MusicVAE) to Drumbit (a basic drum sequencer used as a baseline).
Introduction
Developers of creativity support tools, a growing area in human-computer interaction, have a simple, broadly defined goal: to use advances in computer science to assist and augment human creativity. Digital tools that support creative endeavors like music composition production often show up in the form of tools that mirror physical instrumentation, such as GarageBand or ProTools, digital audio workstations designed to aid the creative process for musicians and producers alike. While helpful in some ways, such a design paradigm relies heavily on domain expertise, forcing users of these tools to learn the same complex rules and processes of the original physical instrumentation. Research within the space of creativity support tools instead has focused in some ways on unleashing the new opportunities afforded by software: a medium for dynamic and easily iterable interfaces.
Beyond this, machine learning has presented new opportunities for collaboration of sorts between people and computers, one area of a larger field described as computational creativity. While lots of interesting work is also being done in standalone creative systems, from the development of autonomous painting machines to narrative generators, more and more systems are being designed to support human creative processes. Interesting systems in the musical space in particular include ALYSIA, a lyrics-to-melody generation system to help song-writers, as well as new music production interfaces, such as Social EQ, Audealize, Mixploration (Pardo et. al. 2019).
Latent space tools for music composition are a particularly intriguing area within computational creativity. Machine learning tools like MusicVAE present new opportunities for interpolation and exploration of music in new and inventive ways (Roberts et. al. 2018). At the core of such technology are variational autoencoders, which learn to map familiar concepts like melodies or beats to different dimension spaces. Autoencoders have been applied to a number of visual and audio problems, using lower dimensional latent spaces to force the learning of a compressed representation of inputs such as musical sequences.
Beyond the reconstructed output of these networks, what makes them particularly interesting is the latent spaces themselves, which can be traversed in creative ways by a human musician or composer, presenting an excellent opportunity for the development of new creativity support tools. Along with new algorithmic approaches to music generation like MusicVAE, Google’s Magenta project has spawned a number of these tools through a robust open-source community.
While multiple creativity support tools have been developed using the Magenta suite of algorithms, there has been little testing and analysis of user responses to these tools. Of all of the tools developed in association with MusicVAE and the Magenta project, we found a rhythmic tool called Beat Blender to be the most complete for our purposes. With this in mind, our goal was to evaluate the effectiveness of Beat Blender using the Creativity Support Index (CSI).
Related Work
The Creativity Support Index is a metric designed by E. Cherry and C. Latilupe to measure the creativity support of tools through user sentiment (2014). Users are asked questions relating to the Enjoyment, Exploration, Expressiveness, Immersion, Results Worth Effort, and Collaboration (only in collaborative use cases) of the tools. They answer on a sliding scale related to a specific task at hand (e.g., making a beat). Finally, they are asked to rank the relative importance of attributes to produce a weighted sum of scores making up the CSI.
Since the creation of the Creativity Support Index, several different researchers have used it to measure the creative support that their projects provide. These different projects span a number of different subfields of computational creativity. For example, GEM-NI, a system that creates and manages idea generation through automatic information exploration, published their paper including the tool’s creativity support index (Zaman et. al. 2015). Other tools, like InspirationWall, also use the CSI to demonstrate creativity supportiveness (Andolina et. al. 2015). Some papers, like “Evaluation of Interactive Machine Learning Systems,” have described the need for more research into the creativity support effectiveness of interactive machine learning systems, specifically studies that use CSI as a measurement. In this study, we use CSI to measure the creativity support of Beat Blender, an interactive machine learning system in the music generation domain.
A number of other creativity support tools have been developed for music generation and production. Mixploration is one example that attempts to use a vector space to increase creative possibilities for music producers (Cartwright et. al. 2014). The authors conducted a user study to measure the effectiveness of Mixploration against a baseline tool, a more traditional music editing application. This approach to testing is fairly common, so our study on Beat Blender uses a similar approach, by comparing a more experimental musical tool (Beat Blender) to a more traditional one (drum sequencer). Unlike Mixploration and other user studies in the music generation space, however, we use the Creativity Support Index to determine the difference in creativity support between the two tools.
Beat Blender is a creativity support tool built on MusicVAE as part of the Magenta project. Some notable projects include Melody Mixer, which interpolates between two user provided melodies to generate new possibilities, and Latent Loops, which does not currently have a working online demo but was designed with a similar intent of exploring melodic spaces. Beat Blender is accessible online through a working web application and is comparable to popular existing creativity support tools for drum sequencing, making it an excellent candidate for a creativity support index study.
Beat Blender has 4 corners with preset or user-defined rhythms. The rest of the grid is filled out using the latent space encoding of the interpolation between these beats, and users can draw lines to traverse the space.
Experimental Design
In our experiment, the participants were asked to create a drum beat with both Beat Blender and Drumbit, a basic drum sequencer. After a short introduction to each tool, they were given about 5 minutes to explore different possibilities. We asked them a series of questions to evaluate the creativity support index of each tool after each session, followed by questions to gauge what metrics are most important to the user. These questions were administered using Erin Cherry’s CSI Survey Application, which performs the calculations laid out in her paper directly.
In a post-survey, we also asked users which tool they preferred and why to expand on the comparison between the two tools, along with our own observations of their comments during the study. Survey and observational data were made anonymous, with participants asked to record a unique identifier so that their answers to the pre and post survey questions (Google Forms) could be associated with the csv files output by the CSI Survey Application for verification purposes and analysis.
We tested these tools on users with some musical knowledge, but no extensive experience producing music and/or beats using digital software. Creativity support tools like Beat Blender are intended to present new ways of creating for individuals with a variety of levels of experience in the field; one common application is to make traditionally difficult tasks such as music production easier to learn for a wider population. We wanted to see if users with little experience with drum sequencers preferred a more traditional tool or an interpolative latent space one. This goal also helped to ensure as much as possible that participants did not have an inherent preference towards tools like Beat Blender or Drumbit based on prior experience with similar tools.
Our experimental participants were a mix of 10 individuals with musical backgrounds but no extensive experience producing beats with drum sequencing tools. These numbers were limited by time constraints; in the future, a study with more users with potentially more varied levels of musical experience could produce more robust and broadly applicable results. For this survey, we ensured users met the inclusion criteria of having enough musical knowledge to be able to pick up Beat Blender or Drumbit and produce a beat in a 5 minute time frame. We asked participants if they have played a musical instrument, if they have experience with music production, or if they have experience composing music with a sliding scale of possible answers for each question. Our users were required to have moderate experience or higher with either playing instruments or composition and moderate experience or lower with production tools. We verified these qualities using a series of pre-survey questions:
Results
Beat Blender did not get a significantly higher average creativity support index score than Drumbit. In fact, Drumbit got a significantly higher average CSI score than Beat Blender (p=0.02). The average CSI score for Drumbit was 69.74, while the average for Beat Blender was 55.63. Our results are shown in the table:
“X” marks the mean, line marks the median. The single dot above the Beat Blender plot marks an outlier (CSI=84).
The average “Results Worth Effort” score for Drumbit was also found to be significantly higher than Beat Blender. Each of the other average scores (Enjoyment, Exploration, Expressiveness, and Immersion) were also higher for Drumbit, though not significantly so. The majority of participants agreed that they preferred Drumbit over Beat Blender.
Discussion
Using the Creativity Support Index, we found that Beat Blender is not significantly better at providing creativity support than an ordinary beat sequencer. In fact, Drumbit appears to be the better creativity support tool, according to our study. Our participants shared their why they liked each tool:
Generally, it seems that Drumbit was preferred because if gave users a feeling of having more control. Indeed, Drumbit included more options than Beat Blender’s corners.
Drumbit has options for swing control, different kits and effects, and different volumes and pitches for each track
Additional features are likely not the only reason that a drum sequencer gave participants a greater sense of control. With a beat sequencer, the user has to fill in each beat manually. When the beat is complete, they feel like they truly created the beat. On the other hand, even if the user customizes each corner of Beat Blender, the majority of beats in the vector space are generated, they are not made first hand by the user. One user claimed:
“This one (Beat Blender), I didn’t feel like I created anything. The first one (Drumbit) I felt like I made a creation.”
Users seem to take a sense of control very seriously. Though our users all had some sort of musical experience, most of the users had only occasional/limited experience writing songs or composing music, and most had no experience with music production. One might hypothesize that only expert producers and beat makers would focus on control in a creativity support tool, yet our study shows that even novice beat makers want more control, rather less.
Conclusion
Our participants’ preference for Drumbit as their beat-making tool of choice shows that users prize control over other features in this type of tool. Though Drumbit was preferred by our participants, Beat Blender was shown to have its own purpose. Most users agreed that Beat Blender was fun to use, and qualitative feedback suggests that it helped generate beats that users would not have thought of themselves. Therefore, Beat Blender can be described as a useful exploratory tool, but a less than complete creativity support tool.
The exploratory value of Beat Blender would best be suited in tandem with an interface for fine-tuning. It currently stands as a standalone web-based application, but it would be the most effective as a plug-in in a larger application, like Ableton, which allows users to make adjustments to every aspect of their composition. It could also benefit from a broader space of options, such as changing the speed of traversal of the latent space. By giving more control to users, Beat Blender may become more competitive with baseline tools along metrics like “Results Worth Effort.”
By using latent space tools, creators are able to explore the creative space a process before committing to a single solution. Latent space-based MusicVAE tools are an exciting and novel way to support the creative process. These tools assure that a creator does not favor exploitation over exploration. Beat Blender is a great example of a tool that supports the explorative process. Ultimately, Beat Blender helps music makers avoid local minimums, and allows them to find the globally optimal beat they want. With more user-driven fine-tuning, it could become even better, and we encourage the use of similar experiments for other emerging latent space tools as well.
Works Cited
Cherry, E., & Latulipe, C. (2014). Quantifying the creativity support of digital tools through the creativity support index. ACM Transactions on Computer-Human Interaction (TOCHI), 21(4), 21.
Zaman, L., Stuerzlinger, W., Neugebauer, C., Woodbury, R., Elkhaldi, M., Shireen, N., & Terry, M. (2015, April). Gem-ni: A system for creating and managing alternatives in generative design. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (pp. 1201–1210). ACM.
Andolina, S., Klouche, K., Cabral, D., Ruotsalo, T., & Jacucci, G. (2015, June). InspirationWall: supporting idea generation through automatic information exploration. In Proceedings of the 2015 ACM SIGCHI Conference on Creativity and Cognition (pp. 103–106). ACM.
B. Pardo, M. Cartwright, P. Seetharaman, and B. Kim, “Learning to Build Natural Audio Production Interfaces.” MDPI, Multidisciplinary Digital Publishing Institute, 29 Aug. 2019, https://www.mdpi.com/2076-0752/8/3/110/htm.
M. Cartwright, B. Pardo, and J. Reiss, “Mixploration: Rethinking the audio mixer interface,” in Proceedings of the 19th international conference on Intelligent User Interfaces, 2014, pp. 365–370.
Roberts, A., Engel, J., Oore, S., & Eck, D. (2018). Learning Latent Representations of Music to Generate Interactive Musical Palettes. In IUI Workshops.