AI DJ Project — A dialog between human and AI through music

Nao Tokui (Qosmo)
Jan 26, 2018 · 9 min read
Image for post
Image for post
Me, with AI DJ Robot — AI DJ Project at Digital Choc 2017 / WWW Shibuya, Tokyo (Photo: Rakutaro Ogiwara)
Video: Yasuhiro Tani, Courtesy of Yamaguchi Center for Arts and Media [YCAM] / Edit: Qosmo, inc.

1. Background and Related Projects

Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
left/center: “2045” — AI DJ Party organized by Daito Manabe and Nao Tokui (2015) right: Bluetooth-based crowd tracking system used at 2045 (Visualization: You Tanaka) (2015)

AI DJ project doesn’t aim to automate the whole DJ process, but rather tries to accomplish a successful collaboration between AI and human DJ. Hence in our DJ session, software and human DJ plays alternately one track at a time (usually referred as Back to Back or B2B)


2. Our method

Custom-made Technics SL-1200 (designed and assembled by YCAM InterLab)
Robot finger for adjusting beat phase (designed and assembled by YCAM InterLab)

2.1 Music Selection

Image for post
Image for post
Recommending music on Spotify with deep learning http://benanne.github.io/2014/08/05/spotify-cnns.html
Image for post
Image for post
t-SNE representation of the extracted auditory features
Image for post
Image for post
Visualization of DJ Playlist analysis. (2015)

2.2 Beatmatching

Reinforcement Learning of DJ Beat Matching (2016)
Image for post
Image for post
Alignment of beats in the beatmatching process https://en.wikipedia.org/wiki/Beatmatching

2.3 Crowd-reading

“A good DJ is always looking at the crowd, seeing what they like, seeing whether it’s working; communicating with them, smiling at them. And a bad DJ is always looking down at the decks and just doing whatever they practiced in their bedroom, regardless of whether the crowd are enjoying it or not.” Norman Cook, aka Fatboy Slim

Image for post
Image for post
Photo by Yasuhiro Tani Courtesy of Yamaguchi Center for Arts and Media [YCAM]
Image for post
Image for post
Camera system for quantifying how much the audience dances to the music

2.4 Visualization

Image for post
Image for post
Each track is mapped into 3D space based on its high dimensional feature vector. (Visualization: Shoya Dozono)
Image for post
Image for post
“BEATMAP”(center) shows how well the beats of two tracks are well aligned. (Visualization: Shoya Dozono)
Image for post
Image for post
Visualization of candidate records in the record box (Visualization: Shoya Dozono)

3. Conclusion and Future Directions

Image for post
Image for post
Image for post
Image for post
Digital Choc 2017 / WWW Shibuya, Tokyo (Photo: Rakutaro Ogiwara)
Image for post
Image for post
Image for post
Image for post
AI DJ Robot — A physical embodiment of the process. GoPro camera provides the first-person view of the robot. (Photo: Rakutaro Ogiwara)

4. Call for collaborators


Past Performances

Image for post
Image for post
Image for post
Image for post
YCAM presents AI DJ / Wildbunch 2017 (Photo: Atsushi Tanabe)
Image for post
Image for post
Image for post
Image for post
Scopitone Festival 2017 (Photo: David Gallard — Scopitone 2017)
Image for post
Image for post
Image for post
Image for post
AI DJ with Licaxxx at sound techtonics #20 / Right: Photo by Yasuhiro Tani Courtesy of Yamaguchi Center for Arts and Media [YCAM]
Image for post
Image for post
Image for post
Image for post
Google I/O 20219 Keynote Preshow

Credit

References

Qosmo Lab

Cultivate the Chaos

Nao Tokui (Qosmo)

Written by

徳井直生 — Qosmo, Inc. / Keio Univ. SFC Computational Creativity Lab — AIと創造性. http://naotokui.net/ http://qosmo.jp/ https://cclab.sfc.keio.ac.jp/

Qosmo Lab

Qosmo Lab

What’s happening at Qosmo, Inc. — Ideas / Experiments

Nao Tokui (Qosmo)

Written by

徳井直生 — Qosmo, Inc. / Keio Univ. SFC Computational Creativity Lab — AIと創造性. http://naotokui.net/ http://qosmo.jp/ https://cclab.sfc.keio.ac.jp/

Qosmo Lab

Qosmo Lab

What’s happening at Qosmo, Inc. — Ideas / Experiments

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store