Typography meets shaders — making matrix rain shader with Processing and Unreal
When I was showing off a VR project I was working on, I accidentally went outside the map and someone joked it should be showing matrix rain.
The idea started growing on me until I decided to make it happen.
Requirements: Unreal Engine 4.x and Processing 3.x.
If you want to make all the textures by yourself, you also need 3d package : Blender, 3ds Max, Zbrush etc. For some basic 2D operations Gimp or Photoshop will also come in handy.
You don’t need to know anything about Processing — this tutorial will cover the essentials.
We’ll be creating a shader for Unreal, but it could easily be recreated for Unity or OpenGL.
Notice that shader have 3 basic parts:
- Rain of letters
- Random pixel noise
- Breathing veins
If you get lost somewhere, don’t worry. You can download the whole project from here:
https://github.com/SwingDev/matrix-rain-of-letters
Part I: Creating letters texture in Processing
Firstly, we need a font.
You can choose any font as long as it’s mono-spaced — I picked this one.
Ok, let’s get started with our Processing project bootstrap:
PFont MatrixFont; // global font variable
void setup() {
size(1024, 1024); // tiling texture size
background(0); // black background
noLoop(); // We need only one frame
...Now let’s load the font
...
MatrixFont = createFont("matrix code nfi.ttf", 92); // size is 92
textFont(MatrixFont);
textAlign(CENTER, CENTER);
}Center alignment is key for making our result texture tileable.
Remember to store your font file in project’s data folder. Should be in the same folder as your sketch. If it’s not there — create it.
Ok, let’s draw some text:
void draw(){
fill(255); // white color
// number of char 65 in normal font should give 'A'
char letter = char(35);
text(letter, width/2, height/2);
}This will result in a single character positioned in the center of the screen:
Wonderful.
Now, in our setup loop let’s decrease the font size to 20 and then fill our canvas with an array of letters:
void draw(){
fill(255); // white color
// number of char 65 in normal font should give 'A'
char letter = char(35);
int gap = 16; // distance between letters
for (int y = gap; y < height; y += gap) {
for (int x = gap; x < width; x += gap) {
text(letter, x, y);
}
}
}This looks pretty boring with the same characters. Let’s add some variety.
First we need a function that’ll give us a random character:
char randomChar(){
char letter = char(45+(byte)random(105));
return letter;
}Combined into:
void drawMatrixLetter(int x,int y){
char letter = randomChar();
fillLet();
text(letter, x, y);
}Now let’s use this function instead of a fixed character:
void draw(){
int gap = 16; // distance between letters
for (int y = gap; y < height; y += gap) {
for (int x = gap; x < width; x += gap) {
drawMatrixLetter(x, y);
}
}
}Here is the result:
Nice, huh?
Now our letters texture is almost done — but we do need one more thing, and it’s a big one — it needs to be tillable.
This is why we left the edges empty.
To achieve this we’ll need 2 more loops in draw() to add the same characters on the opposite edges:
for (int x = 0; x <= width; x += gap){
char letter = randomChar();
fillLet();
text(letter, x, 0);
text(letter, x, height);
}
for (int y = gap; y < height; y += gap){
char letter = randomChar();
fillLet();
text(letter, 0, y);
text(letter, width, y);
}Finally — we want our texture to be saved on disk, so at the very end of our draw function we add:
saveFrame();So that our texture will be saved to our project folder:
Here is our final result:
Complete Processing sketch:
matrix-rain-of-letters/ProcessingSketches/MatrixLetters/MatrixLetters.pde
Part II: Creating letter tails
Let’s create another sketch in Processing to handle letter appearance and disappearance.
The core of this texture are lanes which mask emission and opacity of each lane using red channel and green channel. Both of them will move from up to down.
I will also use this sketch to generate masks to determine which columns move faster than the others.
The setup is very familiar to our previous project, with a few additions.
void setup() {
size(1024, 1024); // tiling texture size
blendMode(ADD);
rectMode(CENTER); // drawing shapes with center as pivot
background(0);
noLoop(); // We need only one frame
}Black background with an additive blend mode enables us to pack single channel textures separately in one RGB texture.
We will define some global variables as well:
color maskR = color(255,0,0);
color maskG = color(0,255,0);
color maskB = color(0,0,255);
color black = color(0, 0, 0);void setup() {
…
We’ll need 2 helper functions:
// draw a smooth vertical gradient
void setGradient(int x, int y, float w, float h, color c1, color c2 ) {
noFill();
// top to bottom gradient
for (int i = y; i >= y-h; i--) {
float inter = map(i, y, y-h, 0, 1);
color c = lerpColor(c1, c2, inter);
stroke(c);
line(x-w*0.5, i, x+w*0.5, i);
}
}// draw a square
void makeSquare(int xPos, int yPos, int w){
fill(maskG);
rect(xPos, yPos+w, w, w);
}
We also need function which will generate every raining segment and tile it vertically:
void makeRainSegment(int xPos, int yPos, int w, int h) {
setGradient(xPos, yPos+w/2, w, h, maskR, black);
makeSquare(xPos, yPos, w);
setGradient(xPos, yPos+w/2-height, w, h, maskR, black);
makeSquare(xPos, yPos-height, w);
setGradient(xPos, yPos+w/2+height, w, h, maskR, black);
makeSquare(xPos, yPos+height, w);
}And a function to mask fast moving areas:
void makeSpeedBand(int xPos, int gap){
fill(maskB);
rect(xPos, height/2, gap, height);
}Now we can draw all 3 masks:
void draw() {
int gap = 16; // same gap as in previous one
// width needs to be little bit smaller
int segmentWidth = 14;
for (int x = 0; x < width; x += gap) { // horizontal loop
int segRandH = ((int)random(height-30)+30);
int SegYpos = 0+16*(int)random(height/16);
if (x==0) { // repeat first segment to last
makeRainSegment(0, SegYpos, segmentWidth, segRandH);
makeRainSegment(width, SegYpos, segmentWidth, segRandH);
} else {
makeRainSegment(x, SegYpos, segmentWidth, segRandH);
}
if (segRandH>height/2) // for fast tails
{
if (x==0) {
makeSpeedBand(x, gap);
makeSpeedBand(width, gap);
} else {
makeSpeedBand(x, gap);
}
}
}
saveFrame();
}And the end result:
Complete Processing sketch:
matrix-rain-of-letters/ProcessingSketches/MatrixTails/MatrixTails.pde
Part III: additional textures
Ok. Processing part is done but we still need some additional textures.
First we need a pixel noise texture. Something similar to this:
You can create one using Substance Designer, Photoshop, Blender or even your own Processing sketch. Or you can just use the one attached above.
Using the same technique you can make pixel luminescence variations for pixelation effect.
As you remember our shader has this “breathing” effect with veins. We have to create the ‘veiny’ texture.
To understand how it works let’s take a look at this depth texture representing half of the sphere seen from the top:
What would happen if we used it in the clamp of veins part of the shader?
Something like this:
The veins texture must be mapped on the spherical-like surface in order to achieve to achieve a good result. Why? Because we are going to clamp values and result animation will look linear which is gonna look bad, getting shape like this we can achieve ease in and ease out dynamic. For more info you can read 12 basic principles of animation
Corresponding depth map looks like this:
Zbrush is probably the easiest way to create this texture, but any 3D software will do.
After tiling, extracting alpha and getting rims on the edges we arrive at this:
(Red channel: depth map, Green channel: Fresnel effect, Blue channel: alpha)
Part IV: The shader
Finally, we have all the textures we need:
But we can do one more thing.
Let’s pack letters texture, mask for fast lanes and pixel variations into a single texture since we are not going to animate the uvs. Also have to move cell noise to blue channels of tails. This will limit the number of 2D samplers in our shader.
The result being:
We’re ready to create a new Unreal project and import our textures.
Then create a material named MatrixRain and change both the blend mode and the shading model to:
Let’s begin with the rain trails. To achieve this effect we need time which will pan texture with the step of the letter.
Why the fixed value multiplier?
Remember how we got in processing texture of size 1024 x 1024 and the gap between letters is 16?
1024/16 = 64
1/64 = 0.015625
Basically this is distance in uv coords between chars. Why why need ceil of texture at all? It will work with our squares in moving_textures. It will be used to show one char in line at once.
Now we can use this time variables to pan textures
We use the same texture with 4 different animated uvs. 2 of them “jumpy” and 2 linearly panned.
Now using red mask in letters_lanes_pixels texture we can blend 2 speeds. We have to also multiply it by the chars textures located in the green channel.
We are using green chanels from the resulting texture set to simulate emission because that’s where the squares with letters are located.
Ok basic rain is completed but we have to still multiply it by a uniform color. Since we are using additive blend mode, we‘re going to add rain opacity to rain emission.
We have now basic Matrix Rain shader. Let’s test it on a plane
Cool, it looks all right but this is too fun to stop it now. Let’s add some random cell noise.
Firstly we need some random flashing light effect. Something like this:
How do we achieve this? Using pseudo-random function which takes time as input
This is how the function looks inside:
I will not explain how to write random-generator functions but I strongly recommend this article about randomisation.
Using same function we can generate random uv coords.
Now mix it together:
And we have some “random” pixel noise.
Now — time for the breathing effect using veins texture.
As always we start with the general speed of the effect
Using this variable we generate a cyclic value in the range of 0–1 followed by a short break.
We can use this break to calculate new offsetted uv coords using veins time as input.
(Note that I set the uv size up in TexCoords attributes to be 1.5 for the veins to be a little bit smaller)
I also multiplied frac by 1.1 just not to stop clamping too soon.
We also have to determine how much of the gradient we’re going to clamp.
We are taking 20% of the gradient and clamp the rest.
Ok now we can start clamping the red channel of our texture. But there is a catch.
When we clamp our heightmap we have to handle a situation when frac is smaller than the veins step. And we want it to always be stretched to the 0–1 range. So we have to branch it.
Now it’s time to apply fresnel and alpha from the veins texture.
I also added additional blend in blend out using veins speed as input. But this is optional, depending on whether you’re satisfied with the result yet (depends on the height texture) .
Finally let’s add some mask and color
This is how the veins look in a close up
This is almost over. Now we have to mix all the effects together.
End this is how the end result looks like in a close up.
I recommend creating a material instance to play with the variables.
Going Further
- Notice that we multiply speed in rain by 2 and in pixel noise by 4. Try using some non-round numbers to get a more natural feel.
- This sample uses only 2 speeds of rain, it will look cooler with more speed masks.
- Branching gives a better effect, but the material itself works without it (and it’s faster)
- In some cases material looks better when it’s set to be two-sided
- We could add some pseudo-randomness to make the veins more interesting.
- We are using additive blend mode but with just a few adjustments we could use AlphaComposited
- It is always a good idea to add static flags to the material, so we can enable or disable some of the effects.
- Chars texture is now static, but with a little use of compute shaders we could make it dynamic.
