A Music Adventure Game

MokMok is a music adventure game for PC & Mac. Mok´s are living musical instruments. They move to the beat they play and together create ever-changing interactive songs. Accompany them on a funky journey and experience music like never before. MokMok Episode One is out now!

MokMok is made by 2Beats, lead by @marc_lauper & @samim.


Reimagining Word Processing

Writing is a ancient art form. From Papyrus to MS-Word, the tools we use to write have defined how and what we write. Today, assisted writing tools are using machine learning techniques to understand, manipulate and generate human language. The implications are profound.

This experiment tries to re-imagine word-processing software. It explores new forms of writing, that allow authors to shift their focus from creation to curation, and write more joyfully.

Writing Text

Humans have been writing text, assisted by machines for a very long time. From the first commercial typewriters in the 1860's to today’s laptops, the act of writing has…


Evolutionary Algorithms + Ableton Live

Making music with computer tools is delightful. Musical ideas can be explored quickly and composing songs is easy. Yet for many, these tools are overwhelming: An ocean of settings can be tweaked and it is often unclear, which changes lead to a great song. This experiment investigates how to use evolutionary algorithm and novelty search to help musicians find musical inspiration in Ableton Live.

Making Music With Computers

In the past 30 years, the art of making music has dramatically changed. …


Mining Body Language from Videos

From Gene Kelly’s Step-Dance to Bruce Lee’s Kung-Fu — iconic movement has made history. Communicating through Body Language is an ancient art form, currently evolving in fascinating ways: Computationally detecting human body language is becoming effective and accessible. This experiment explores enabling technologies, applications & implications.

Human Pose Estimation, using OpenPose. Footage by Boston Dynamics.

For over 20 years, Motion Capture has enabled us to record actions of humans and then use that information to animate a digital character or analyse poses. While movie makers and game developers embraced such technologies — it until recently required expensive equipment which captured only few aspects of the overall performance.


A Emotion Enforcing Text Editor

Writing is a magical medium. It allows us to think clearly and communicate quickly. Writing has instigated revolutions, ended empires and started love stories. How and What we write, is heavily influenced by our mental state. Don’t worry be happy is a emotion enforcing text editor, only usable if you are happy*.

State of Mind

Writing is a ancient practice. Recently discovered neolithic writings have been dated to the 6th millennium BC. From Papyrus to MS Word: Tools seek to empower authors to write — and be more creative, more of the time. In the digital age, a tsunami of tools is transforming…


Exploring Augmented Creativity

Seeing drawings that were made over 20'000 years ago has a haunting effect. Human history has been profoundly shaped by our ability to create visual content - assisted by tools. This experiment and report explore a new generation of creative tools, that help us draw.


Fooling A.Is (and turn everyone into a Manga)

Adversarial A.Is are a common sci-fi theme: Robot VS Robot. In recent years, real adversarial examples have emerged. This experiment explores how to generate images to fool A.Is (and turn everyone into manga).

Convolutional Neural Networks

At the heart of many modern computer vision systems are Convolutional Neural Networks. On some vision tasks, CNNs have surpassed human performance. Industries such as Web-Services, Research, Transport, Medical, Manufacturing, Defence and Intelligence rely on them every day.


Music Video for Band “Years & Years”

Shortly after the release of “DeepDream” in 2015, Roelof Pieters and I independently produced the first two videos exploring animated dreaming. The outputs were fascinating. It lead to the release of custom tools and started a collaboration.

The videos & tools were highly popular online, resulting in an avalanche of requests from directors, agencies and artists — all looking to use deepdream. A particularly interesting request came from director Brian Harrison, asking us to dream a music video for the Band “Years & Years” (Interscope).

Years & Years — Desire (DeepDream Music Video)

Note: Production was done mainly with muted audio due to sanity reasons.

Challenges

During the…


Regressing 24 Hours in New Orleans

Regression is a widely applied technique in machine learning. An example of a regression problem would be the prediction of the length of a salmon as a function of its age and weight. Regression analysis is a statistical process for estimating the relationships among variables. Lets have some fun with it ;-)

Experiment

A while back, Andrej Karpathy released convnetjs, a javascript based neural network library. It contains a demo which Andrej describes like this:

The demo treats the pixels of an image as a learning problem: it takes the (x,y) position on a grid and learns to predict the color…


Recurrent neural network for generating stories about images

Stories are a fundamental human tool that we use to communicate thought. Creating a stories about a image is a difficult task that many struggle with. New machine-learning experiments are enabling us to generate stories based on the content of images. This experiment explores how to generate little romantic stories about images (incl. guest star Taylor Swift).

neural-storyteller

neural-storyteller is a recently published experiment by Ryan Kiros (University of Toronto). It combines recurrent neural networks (RNN), skip-thoughts vectors and other techniques to generate little story about images. Neural-storyteller’s outputs are creative and often comedic. It is open-source.

Experiment

This experiment started by…

samim

Designer & Code Magician. Working at the intersection of HCI, Machine Learning & Creativity. Building tools for Enlightenment. Narrative Engineering.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store