Machine Learning Meets Art to Reflect a Broader Community

By Ripon DeLeon, Manager, Design & Jason Wittenbach, Senior Software Engineer, Capital One

Last month, we had the chance to immerse ourselves among Charlottesville, VA’s leading civic organizers, entrepreneurs, tech innovators, and futurist visionaries at the 2018 Tom Tom Founders Festival. As a co-sponsor of the Festival’s Applied Machine Learning Summit, our team at Capital One wanted to do more than just talk about machine learning — we sought to create an interactive experience showcasing a real-live example of the technology in an engaging, participatory way, while also highlighting some of the area’s amazing local artists.

A central theme of Tom Tom is “learn[ing] to see your world in a new way.” We took inspiration from that ethos to create a pop-up art gallery that would further explore the idea of viewing the world from a new perspective, powered by a machine learning technique called style transfer.

Enter our Style Transfer Gallery, where festival participants were invited to have their picture taken, which was then superimposed into the style and technique of each featured artist — Laura Wooten, Theodore Taylor, Shannon Wright, and Brandon Robertson — and projected among an array of digital screens. This created a real-time, visual tapestry of participants while also preserving a history of previous festival-goers and providing a unique lens into the community.

Under the hood, we used a machine learning technique called “neural style transfer.” This model takes inspiration from what we know about how the human brain processes visual information in order to teach the algorithm how to create images in the style of a particular work of art.

Under the hood, we used a machine learning technique called “neural style transfer.” This model takes inspiration from what we know about how the human brain processes visual information in order to teach the algorithm how to create images in the style of a particular work of art. Training for a single painting’s style takes hours, but once complete, the model can transform a new image into that style in a fraction of a second. This allowed us to process video collected from a camera in real-time, so participants could see themselves moving and interacting as if they were in the painting, before snapping a self-portrait to be added to the Gallery.

At Capital One, we believe that leveraging the power of machine learning can help us interact with our customers in a more natural, seamless, and accessible way — from how we’re able to look out for our customers and their finances, to our ability to continuously enhance their digital experiences — and the Style Transfer Gallery is a further embodiment of this sentiment.

At Capital One, we believe that leveraging the power of machine learning can help us interact with our customers in a more natural, seamless, and accessible way — from how we’re able to look out for our customers and their finances, to our ability to continuously enhance their digital experiences — and the Style Transfer Gallery is a further embodiment of this sentiment. The gallery celebrated and elevated visitors as both artwork and artist, and it was critical for us to create a space that could present and treat these works accordingly.

Our hope is that through the collection of this artwork, a new window into the Tom Tom community emerged.

These opinions are those of the author. Unless noted otherwise in this post, Capital One is not affiliated with, nor is it endorsed by, any of the companies mentioned. All trademarks and other intellectual property used or displayed are the ownership of their respective owners. This article is © 2018 Capital One.