
Computer Vision in Processing
Computer vision is an interdisciplinary field that deals with how computers can be made for gaining high-level understanding from digital images or videos. From the perspective of engineering, it seeks to automate tasks that the human visual system can do.
Referencing Your Webcam
To get started you will need this library to capture video feed from your camera: Sketch > Library > Import Library > Movie Library
Capture— captures live feed from a camera source or webcamVideo— takes in a video sourceCaptureandVideohave the same functionality as aPImage
Some new concepts to understand. These Java functions require self referencing thru the use of this.
import processing.video.*;Capture video;void setup() {
size(500,500);
video = new Capture(this,displayWidth,displayHeight,30);
video.start();
}void captureEvent(Capture video){
video.read();
}void draw(){
background(0);
image(video, 0, 0, mouseX, mouseY);
}
Referencing A Movie
To use the video referencing you’ll need to add your video file into the data folder.
import processing.video.*;Movie video;void setup() {
size(1500,1500);
video = new Movie(this, "boat.mov");
video.loop();
}void movieEvent(Movie video){
video.read();
}void draw(){
background(0);
image(video, 0, 0);
}
jump()- allows you to jump in seconds
time
duration
speed
Computer Vision

If you have a camera and you wanted to detect the movement of an object, how would you track it? Lets take the example of a lightbulb which is a more more straight forward example. First we would need to sample some of the pixels. While most of the pixels would appear dark or “0”, when we hit sampled a pixel with the lightbulb, its value would appear bright at some value close to “255”. Now we don’t want to have to sample every pixel of the screen all of the time when we know the object will likely move a distance relative to its current position. Thus we also need a way to store information on where we found the light bulb — using x and y coordinates.
First we need to be able to determine when colors are similar to one another because if we sample any portion of the light bulb it may not be exactly “255”, it could also be “250”. Or perhaps our lightbulb is a more complex color than white.
How do we determine if two colors are similar to one another? We can test the distance between each one of their red, green, blue values.

Loop through all of the pixels.
GitHub — Daniel Shiffman
import processing.video.*;// Variable for capture device
Capture video;// A variable for the color we are searching for.
color trackColor;void setup() {
size(320, 240);
video = new Capture(this, width, height);
video.start();
// Start off tracking for red
trackColor = color(255, 0, 0);
}void captureEvent(Capture video) {
// Read image from the camera
video.read();
}void draw() {
video.loadPixels();
image(video, 0, 0);// Before we begin searching, the "world record" for closest color is set to a high number that is easy for the first pixel to beat.
float worldRecord = 500;// XY coordinate of closest color
int closestX = 0;
int closestY = 0;// Begin loop to walk through every pixel
for (int x = 0; x < video.width; x ++ ) {
for (int y = 0; y < video.height; y ++ ) {
int loc = x + y*video.width;
// What is current color
color currentColor = video.pixels[loc];
float r1 = red(currentColor);
float g1 = green(currentColor);
float b1 = blue(currentColor);
float r2 = red(trackColor);
float g2 = green(trackColor);
float b2 = blue(trackColor);// Using euclidean distance to compare colors
float d = dist(r1, g1, b1, r2, g2, b2); // We are using the dist( ) function to compare the current color with the color we are tracking.// If current color is more similar to tracked color than
// closest color, save current location and current difference
if (d < worldRecord) {
worldRecord = d;
closestX = x;
closestY = y;
}
}
}// We only consider the color found if its color distance is less than 10.
// This threshold of 10 is arbitrary and you can adjust this number depending on how accurate you require the tracking to be.
if (worldRecord < 10) {
// Draw a circle at the tracked pixel
fill(trackColor);
strokeWeight(4.0);
stroke(0);
ellipse(closestX, closestY, 16, 16);
}
}void mousePressed() {
// Save color where the mouse is clicked in trackColor variable
int loc = mouseX + mouseY*video.width;
trackColor = video.pixels[loc];
}

