Adding Timed Comments and Waveforms to Spotify
Working Prototype Within
For the last week, I’ve been taking a break from client work to build an idea I had following the Future Islands project. That idea was a Spotify powered player where users could leave timed comments on a waveform. No doubt you’ve seen this UX on other streaming services and within other music creation software. A waveform is generated using the track’s audio and users may leave comments along the timeline, giving context to their messages. I experimented with timed comments (and other forms of timed content) while I was at SoundCloud. Heck, I even built a video game once.
From my understanding, Spotify has never had commenting as part of their offering. While this makes their products perfectly simple content consumption devices, it forgoes any opportunity to foster community amongst their users. Spotify users are instead encouraged to share and comment on music externally from the service. There’s pros and cons to integrating comments into Spotify but I do think artists suffer when fans aren’t allowed to congregate.
For this prototype, I sought to only use the Spotify Platform to achieve the solution. So, the waveforms are generated by data which Spotify provides and the users must authenticate with Spotify to playback audio and comment on the track. You can use the solution by visiting www.timedcomments.com. In addition, you may pull up a timed comments interface for any Spotify track by renaming “open.spotify” to “timedcomments” on any Spotify track url.
For example, open.spotify.com/track/123 to timedcomments.com/track/123.
I have already written a few posts about the technology used in this prototype but wanted to go ahead and summarize the approach on this blog. Keep reading to find out how it came together.
Simple Interface
To be perfectly honest, I was only truly encouraged to built this prototype after an inspirational Figma session in which this simple interface came together. Highly inspired by the UI work of SoundCloud, I brought together some of the basic design cues of Spotify (namely, the color green) in an interface I knew I could make responsive later on with the help of Tailwind CSS. I just wanted to keep this simple with only the essential functionality. You may deconstruct this CodePen to learn how the design was them programmed.
Spotify Waveforms
Generating waveforms from Spotify tracks is the first problem I tackled when putting together this prototype. I ended up using the Spotify Platform’s Audio Analysis endpoint to get the loudness levels of the track and then simplify that data into an array of values I could use to dynamically generate the waveform using canvas. Rather than generate this data every time a user visits the app, this prototype caches the waveform data for each visited track once for future visits.
I have also provided a CodePen on how the final waveform was generated.
Spotify Player
Typically when I build custom Spotify player interfaces, I use their Web Playback SDK because it allows you to simply stream full tracks to premium Spotify users who authenticate with your app. The big issue with this SDK is that it doesn’t work on mobile devices currently. In an attempt to make this solution more accessible, I turned to the Spotify Connect Web API. Instead of being able to stream full tracks through your web app like the Web Playback SDK, the Connect Web API allows you to remotely control an open piece of Spotify software on one of your user’s devices. This API can be pretty temperamental but when it works, it’s a decent solution.
Spotify Comments
The comments solution I put together for this web app draws from the Spotify powered chat solution I developed for the Future Islands project. This app also uses Spotify’s Implicit Grant Flow to authenticate users via a popup. Users must also be authenticated in this way to use the Connect Web API.
Once authenticated, users may tap into the text input to begin writing a message. When the input is focused, they may use an avatar slider on the waveform to position their comment on the timeline. Posted comments are sent to an awaiting Serverless endpoint and added to a DynamoDB. These comments are stored with the timestamp of their position on the timeline so they may be visually placed on the waveform again.
What’s Next?
For this particular prototype, I had the thought of making the comments come through in real time like the Future Island’s chat but that is likely something I’ll investigate later. The point of this experiment was to further explore what social features built off the Spotify Platform could look like. Through experiments such as this and in my client work, I hope to continue to investigate this topic. I hope the information shared proves useful to you as well and I’d love to hear what you think and more importantly, what you build. Happy hacking.
