Project 2: Prototype
Having refined my motion design, I moved on to prototying the interaction using my keyboard as inputs. First, I had to be able to play each animation on top of one another, as some of the effects needed to be able to play in the background. To do, I exported each video as a PNG sequence using the RGB + Alpha channels (and ensuring the transparency toggle was enabled within AfterEffects). Then, I used the Animated Sprite example from processing to use processing to load my PNG sequences.
I had three sets of sequences for each letter. The first PNG sequence of each letter animates the letter in a bright white pulsing pattern. This first sequence is what appears when the letter has either not been “played” or has been stopped via the keyboard input. The second PNG sequence of each letter animates the letter alone, with no background effects. The second sequence is what appears when the letter has been “played” via the keyboard input, but no other letters are “playing”. The third PNG sequence of each letter animates the letter and, in addition, animates the background with some effect. The third sequence is what appears when the letter has been “played” via the keyboard input, and other letters are playing at the same time. In a nutshell, each letter has a “solo” animation that plays when its playing alone, and a “concert” animation when its playing with other letters.
In order to trigger each sequence at the right time, I first created a keyboard integer (int) and two booleans — one for playing the animations (play), and one for playing the sound files (playSound). I loaded the animations through three different arrays: pulses (the first sequence), animations (the second “solo” sequence), and effects (the third “concert” sequence).
The beginning of each png file starts with the letter, followed by the sequence number and a dash, such as: d1-. I ensured this file name through the AfterEffects rendering queue. I copied and pasted all of the pngs into the data folder of my processing project. The number following the file name prefix corresponds to the number of files per sequence. The pulse sequences, for instance, all have 33 frames, so that’s the number I inserted into my processing project.
An issue I ran into when debugging this program is file size. I found I had to use a 720x480 resolution at 24 frames/second in order to not crash the program as well as increase the maximum available memory to 4086 MB (4 GB) via File > Preferences. Even so, I still am working on minimizing the number of files I have. Some of the files are repetitive (such as when a letter simply stays on screen), so I’m working on figuring out how to reduce repetitive frames while maintaining synchronization with the sound file. Unfortunately, converting my AfterEffects files into 720x480 resulted in some issues, most notably the disappearance of the liquify effect on the “n” second and third sequence animations. For some reason it stopped rendering once I converted the composition to the new size. This is another issue I’m continuing to investigate.
I called on the sound files using another array. An issue I found was the stereo versions of the sound files didn’t work — I had to convert them to mono.
Following the setup of the file, I inserted the background color I wanted (R: 36, G:0, B:54) in the draw section. To trigger the third sequence to play (the “concert” sequence) when more than one letter was played, I created a boolean for one and moreThanOne.
To change one/moreThanOne to true, I created a for loop in which one becomes true if one animation is playing (if play[i] is true). If one is true AND another animation is playing (play[i] && one), then moreThanOne becomes true. In this way, I am able to control whether the second or third sequences play based on whether or not another letter is “playing”.
Another for loop resets the pulsing animation when either the animations or effects plays. If moreThanOne is true, animations resets and effects plays at the 0, 0 coordinates (x, y). If moreThanOne isn’t true, then animations plays at the 0, 0 coordinates. Additionally, if playSound is false, then the sound file will loop and set playSound to true. If an animation is not playing (play[i] is false), then both animations and effects reset, the pulses animation plays, and the sound file stops.
Finally, I mapped each keyboard input (0 through 5) to the corresponding index number. Unfortunately, the current prototype cannot play all six inputs at once — an issue I’ll continue to investigate.
Eventually, these keyboard inputs will be replaced with buttons like that shown below (source: https://www.adafruit.com/products/1479). Instead of directly triggering the animations to play, the buttons will “select” each letter and the user will have to spin an 8-track record (glued to a potentiometer) clockwise to “play” the letter, and counterclockwise to “stop” the selected letter.
Below are all three parts of my prototype code.
/**
* Animated Sprite (Shifty + Teddy)
* by James Paterson.
*
* Press the mouse button to change animations.
* Demonstrates loading, displaying, and animating GIF images.
* It would be easy to write a program to display
* animated GIFs, but would not allow as much control over
* the display sequence and rate of display.
*/
import processing.sound.*;
import java.util.*;int keyboard;
boolean[] play;
boolean[] playSound;
Animation[] pulses, animations, effects;SoundFile[] songs;
void setup() {
size(720, 480);
background(36, 0, 54);
frameRate(24);
play = new boolean[6];
playSound = new boolean[6];
Arrays.fill(play, false);
Arrays.fill(playSound, false);pulses = new Animation[6];
pulses[1] = new Animation(“d1-”, 33);
pulses[0] = new Animation(“e1-”, 33);
pulses[2] = new Animation(“s1-”, 33);
pulses[3] = new Animation(“i1-”, 33);
pulses[4] = new Animation(“g1-”, 33);
pulses[5] = new Animation(“n1-”, 33);
animations = new Animation[6];
animations[1] = new Animation(“d2-”, 106);
animations[0] = new Animation(“e2-”, 106);
animations[2] = new Animation(“s2-”, 53);
animations[3] = new Animation(“i2-”, 106);
animations[4] = new Animation(“g2-”, 106);
animations[5] = new Animation(“n2-”, 49);
effects = new Animation[6];
effects[1] = new Animation(“d3-”, 106);
effects[0] = new Animation(“e3-”, 106);
effects[2] = new Animation(“s3-”, 53);
effects[3] = new Animation(“i3-”, 106);
effects[4] = new Animation(“g3-”, 106);
effects[5] = new Animation(“n3-”, 213);
songs = new SoundFile[6];
songs[1] = new SoundFile(this, “d-mono.wav”);
songs[0] = new SoundFile(this, “e-mono.wav”);
songs[2] = new SoundFile(this, “s-mono.wav”);
songs[3] = new SoundFile(this, “i-mono.wav”);
songs[4] = new SoundFile(this, “g-mono.wav”);
songs[5] = new SoundFile(this, “n-mono.wav”);
}void draw() {
background(36, 0, 54);
boolean one = false;
boolean moreThanOne = false;
for (int i=0; i<6; i++) {
if (play[i] && one) {
moreThanOne = true;
} else if (play[i]) {
one = true;
}
}
for (int i=0; i<6; i++) {
if (play[i]) {
pulses[i].reset();
if (moreThanOne) {
animations[i].reset();
effects[i].display(0,0);
} else {
animations[i].display(0,0);
}
if (!playSound[i]) {
songs[i].loop();
playSound[i] = true;
}
} else {
effects[i].reset();
animations[i].reset();
pulses[i].display(0,0);
songs[i].stop();
}
}}
void keyPressed() {
keyboard = key;
play[getKeyIndex(keyboard)] = true;
}void keyReleased() {
keyboard= key;
int index = getKeyIndex(keyboard);
play[index] = false;
playSound[index] = false;
}int getKeyIndex(int keyboard) {
int index = 0;
if (keyboard == ‘0’) {
index = 0;
}
if (keyboard == ‘1’) {
index = 1;
}
if (keyboard == ‘2’) {
index = 2;
}
if (keyboard == ‘3’) {
index = 3;
}
if (keyboard == ‘4’) {
index = 4;
}
if (keyboard == ‘5’) {
index = 5;
}
return index;
}