Regressing Images

Regressing 24 Hours in New Orleans

Regression is a widely applied technique in machine learning. An example of a regression problem would be the prediction of the length of a salmon as a function of its age and weight. Regression analysis is a statistical process for estimating the relationships among variables. Lets have some fun with it ;-)

Experiment

A while back, Andrej Karpathy released convnetjs, a javascript based neural network library. It contains a demo which Andrej describes like this:

The demo treats the pixels of an image as a learning problem: it takes the (x,y) position on a grid and learns to predict the color at that point using regression to (r,g,b). It’s a bit like compression, since the image information is encoded in the weights of the network, but almost certainly not of practical kind :)

This experiment test a regression based approach for video stylisation. The following video was generated using Stylize by Alec Radford. Alec extends Andrej’s implementation and uses a Random Forest Regressor. This experiment extends it to work on video. The source video is by JacksGap.

Regressing 24h in New Orleans

If the youtube video is blocked, try vimeo

Selection of results

Heavy loading times, sorry!

Selection of results: convnetJS

With bottleneck layer / dropout for extra fun.

Final thoughts

Using machine learning techniques in unusual ways is fun, generating new image styles addictive. Realtime processing might be possible using GPUs (all CPU so far). A VR version could be fascinating ;-) All results in this experiment are reproducible with the linked open-source software. If setups are to cumbersome for you, try Mario Klingemann’s related “Lowpoly Bot”.

Get in touch here: twitter.com/samim |http://samim.io
Sign up for the Newsletter for more experiments like this!

Source Video

Special thanks to JackGap for the awesome source video! :-)