Sometimes I sit down with friends & record the interesting things they say Episode 1 — Patricio Gonzalez Vivo
ELIZA : A few months ago I had the pleasure of sitting down with Patricio Gonzalez Vivo. What was supposed to be a quick 20 minute chat stretched into an hour long discussion that I’m excited to finally be sharing here. The following is an abridged edit of our conversation. I’ve rearranged bits and pieces for clarity and continuity’s sake. Hopefully you will enjoy learning more about Patricio through his own words, and a few of my own too.
PATRICIO: Hi, I’m Patricio Gonzalez Vivo. I’m an artist, I work with code, and that’s pretty much it. I like shaders, I like Raspberry-pi’s, I like to tinker with stuff and yea, that’s pretty much me.
If you’ve ever tried writing shaders, chances are you’ve come across this man’s work. The Book of Shaders and the Pixel Spirit Deck are two of the most thorough and inviting efforts I’ve seen to date at distilling the maths involved in making shader magic happen. There are a few resources out there, certainly more and more as time goes by, but nothing quite stacks up in comparison, despite the book remaining unfinished.
I’ll come back to that.
But first, a nod to Jen Lowe, Patricio’s partner and as it happens, co-author of the Book of Shaders.
“At the time I had a job that allowed me to be very public with my work. And so I was putting time into open source endeavours. I was very stressed out by the prospect of having a child. Full of uncertainty. And I had this idea of doing a book with shaders.
I kind of do this often… I collide a lot of ideas together. The first idea was to make The Book of Transformations — the I’Ching with shaders — and kind of have these one-off projects, grabbing a shader and through the pages, evolving the same code and teaching how to code shaders through that. Kind of like what the cards (The Pixel Spirit Deck) became later. But I had it all clustering into a book, and also an art piece and all that… My wife tells me, ‘you should do only one thing,’ so I started writing the book.
I’m from Argentina so the way that I speak is way more convoluted and kind of Baroque. I tend to go on tangents. So she immediately started helping me edit it and make it into something that is readable and enjoyable. And we collaborate in that way.”
It might surprise you as it did me, especially given the very math heavy nature of shaders, to know that Patricio claims no natural ease with numbers. He just gets by, he says.
We all have the option of learning things any way we like. But the more I speak to people like Patricio, the more I understand that one version of this game, maybe the most illuminating, is based on creating spaces inside which to play. This develops our instincts about the more technical nooks and crannies behind how things work.
“My wife is always impressed with the intuitive feelings I have about maths. I think that what is wonderful about our scene is that you enter into these visual feedback loops, so it allows you right away to start changing things and seeing things react. So I think over the years with math, I kind of developed a more intuitive [relationship with it]. I have my own explanations of how things work and kind of developed visceral feelings about things.
I tried to put some of that into The Book of Shaders with the idea of ‘fuss with things!’ and with time it’s like a kitchen. You start to understand how a sine wave feels or what exponents look like. So you get more an intuitive understanding of things. And of course then there are more objectively hard issues like matrices and quaternions.
But my recommendation is to not be afraid. No one feels comfortable, except mathematicians. And sometimes they aren’t a great crew to hang out with.
If you’re learning a new language and you go to another country and you try to speak their language, there’s this painful moment where you want to say something and you can’t do it.
But on the other side there’s the painfulness of seeing someone trying, struggling in the moment. And you know there’s nothing bad about that because no one is fluent in a language that is new. And everyone wants to help.
Also with practice you develop your own ways of doing things. Most of my math is sometimes overly complicated and simplistic and there’s a lot of honour there. My point is to not feel ashamed of the way things work for you. It’s a good way of breaking the fear.
So try to protect yourself from those mind traps because… we have this phrase in Argentina: circles of saliva. There are these frogs that kill snakes. They have this toxic saliva and what they do is they surround the snake with it. And the snake cannot cross the saliva thing that they made, and eventually the snake dies of starvation.
Yea, it’s not a happy story. But there are a lot of times where we get trapped in the saliva circles, which are mostly about being proud of how smart you are. So it’s important to be very open about how much you know and very comfortable with that, because, especially if you come from another field , you know that other language. In my daily life right now I work with engineers a lot and I notice there’s a lot of respect from the other side for these other things, all this richness that comes from other places.
And actually, my wife is the one who knows about math. She has a math degree. My background is in Psychology.
I started studying Computer Science for about six months before I realized that the only class I enjoyed was Philosophy and I was like, ‘I think I’m making a mistake’ so I dropped Computer Engineering and joined Psychology, which was a huge win.”
Patricio had spent years attending an all boys Catholic school before jumping into Engineering. After being surrounded mainly by boys and men for so long, making the switch to Clinical Psychology, where he was suddenly a minority among women, he says helped him to balance himself out.
“Being the only boy in Clinical Psychology helped me to humanize myself. And when I finished, when I got my Psychology degree, during those last few years I had been thinking that I want to do some art. Having the title of psychologist and having started working in the field kind of gave me space to say, ‘OK, now I can do something for myself.’
So I started going to drawing classes. I started taking very esoteric courses about mythology. And I was in one course about mythology and fairytales all from the perspective of the Jungian school.
Argentina is pretty heavily into psychoanalysis and so where I was studying fairy tales and mythology, there was a class on art therapy. And the approach was very plastic, about drawing emotions and dreams and stuff. That got me moving into another program called Creative Connection founded by the daughter of Carl Rogers, Natalie Rogers. Her approach is called Expressive Art Therapy which is a multi-language kind of process.
If something starts with a movement, then it’s very easy to translate that movement onto paper. And from that it can become a drawing and from the drawing it can become a story. So you always look for your path towards words. I kind of did a three year post-degree on that and when I finished I needed to start selling myself, to have my own practice and to promote some groups that I was organizing.
I found that Flash was the best thing to do this, so I taught myself Flash. And from there I was led to discover Processing. That’s where I got into the book of Daniel Shiffman, and it totally blew my mind.
It blew my mind that there was a scene, doing these kinds of things. And it was working with computers in an expressive way. For me it was like, oh I’m helping others to do this, and this seems like a language that is not native to me. So maybe this is my expressive language.”
This is when Patricio started getting into creating installations. And thanks to an introduction, he was able to get his foot in the door with a number of institutions around Buenos Aires that were promoting digital art and supporting artist residencies.
“I applied to one, Interactivos 2010, and there I worked a whole year on an interactive table to do collective touch drawing. It was a circular table where you could draw with others in a very ‘mandala’ kind of way.
To get that interactive piece to work I realized I needed to use something that had more computational power so I moved to Open Frameworks [and programming in C]. And there I pretty much felt at home.
Obviously there was a learning curve. I was at the beginning, learning, and as I was a user of Linux, it had always been my dream to learn C because I knew that Linux was in C.
In 1999, I had a magazine that talked about computers and it mentioned Linux and I decided Windows sucks because it was like the capitalist model and all that. I decided to use Linux only so I started with my first Red Hat 5.1. And at that moment it was kind of hard to do, so I did it with a friend and I have been a Linux user for a long time since.
So it was this idea to learn C, but I never had time because I was always studying something totally unrelated. When I learned about Open Frameworks I was very curious about this approach to C and C++.
And I was lucky. I presented a second project, Efecto Mariposa, that I made in 2011, which was a sandbox. The Kinect had just appeared and nobody was doing that yet and I wanted to do something more natural, in the sense of like, with less technology.
One of the problems I had with the drawing table is that it was visible technology, like, this projection. Everything was very glowy and I was seeing that it was triggering all the anxiety that technology produces in us. So it was, you know, all that stress.
And also as part of the Jungian technique called the Sandbox, which is you come and you kind of like, shape these kind of landscapes that you dream of and you populate it with little creatures. So I was like, ‘I want to make a sandbox, but interactive.’
I put a Kinect on top and projected it with a screen. I presented it to Centro de Cultura Espagnol, and they decided to fund me. So I became their resident to keep pushing the project forward.”
At this point Patricio hit a bit of a snag. The CPU on the machine he was using was chugging. There just wasn’t enough processing power to handle the project. He had a hunch though, that shaders might solve the issue. The cultural centre offered to either hire someone to fix it for him, or to find someone who could teach him how to fix it himself. You can probably guess what he chose.
“I had been telling them that I had the intuition that it was with shaders that this processing power problem can be solved. Because I was simulating the geosphere, so like the tectonic plates, the water, also the growing patterns…
I made a reaction diffusion algorithm to make nature kind of like, grow naturally. I also made a simulation of falling water. So there were rivers, and also on top there were clouds. So you move your hands on top and if the conditions are right, it starts raining in a place and the water comes down and that produces life. And if the conditions were right again, a little flocking particle system would appear.
And this was technically very sophisticated. So I decided obviously to go with the guy who could teach me shaders.
And that’s how I got into shaders.”
This was still pretty much the dark ages in terms of freely available shader resources. IQ, Inigo Quilez, hadn’t yet burst forth onto the scene with the online shader sharing platform Shadertoy, and all of his crazy custom functions. Documentation and code examples were sparse, if available at all. So in a way, Patricio was really quite lucky that he hit the wall that he did with Efecto Mariposa. At that time you pretty much needed a PHD to access this stuff, and here he was learning how to hand roll his own shaders.
On a personal note, I’ll always advocate learning to do the thing over paying someone to do it for you, especially when it comes to your creative work. It’s fun to challenge ourselves and learn new things. It also quite often lands us in places we didn’t expect. Especially if you share what you know.
“So after that work, I was kind of happy with it and I got to show it in different places. In Brazil, in Argentina… and through that I kind of like, broke up all the shaders I had made for that and created an add-on which was super welcome in the Open Frameworks community. Thanks to that I met Zach [Lieberman], and Kyle [McDonald] and I went to New York to visit as a tourist.
I met up with Zach who had just married Momo, and he was talking about suddenly being a parent and a husband. I started talking about being a psychologist and wanting to change careers. So it was kind of like an open and beautiful talk. And he said ‘You should apply to Parsons, you have a lot of good work.’ I went as a student and they gave me a full scholarship and that’s how I moved to the US.
I was very lucky that I got a job after Parson at a digital map company called Mapzen. I was working close to the Flat Iron [district] but I was living in the south, in Sunset Park, and I had to take the R, which is like an incredibly slow line and so the commute was like an hour twenty… an hour and a half… Luckily it goes very slow — no, it’s very empty. So I had a lot of time to spend with my laptop. I’d sit down and do iterations on cards and do weird patterns and kind of develop this weird workflow.
The whole Pixel Spirit [Deck] was born and developed on the New York subway.”
At the time of this recording, Patricio is in the process of creating an extension pack for the Pixel Spirit Deck. He’s using the Codevember challenge to iterate on new patterns and add-ons for the library. If you’re not familiar with the Pixel Spirit deck, it’s both a fully fledged Tarot deck and a physical shader library of sorts. The cards follow an order, and every one of the cards builds on the ones that came before it. Each card features an image rendered in shader code on one side, and the code used to create it on the other, which is a great way to hone your intuition about what math makes which type of shape, pattern or the like.
People have done a lot of fun things with this deck, from sculptures and even tattoos. Check out the hashtag #thepixelspiritdeck if you’re curious.
Ok, as promised, let’s jump back now to the Book of Shaders. Of course I had to ask… Are there are any plans to complete it?
[laughs] “I really want to finish the Book of Shaders. And also I feel this thing of … I don’t feel great that it’s half-baked. But it requires a lot of time and because it’s a collaboration it’s not just my time. So yes, and right now we’re focused on family, on making humans from scratch. It takes up kind of.. Yea. I can see the moment when I don’t have to have a ‘job’ job and I would have more time to dedicate to it. And I certainly want to do that.
There’s one chapter, the chapter on Fractals, that chapter freaks me out. Because it’s the most math heavy. It’s so much work to make it in the way that I want to explain imaginary numbers and fractals and all that. It kind of paralyzed me before starting because it’s like… This is a lot of work.
And this divided into chunks of one hour of commutes will take me like, months and months… Also when I’m very motivated with a project I get quite obsessed and sometimes that can diminish my skills as a parent. So it’s not just my energy but my presence at home that will be reduced. And yea, kids require presence and time.
But yea, I feel it. I do.
I prefer to do it well, especially because then you enter into these legacy issues. People very generously started translating the book, but now I have this problem that if I change something in the english version I have to tell everyone. I have to signal it somehow and it’s very hard for me to keep track of whether that phrase was changed in Chinese or Japanese or this other language that I’m too ignorant to know how to even approach.
One of the other things that’s happening is the tools are getting old and there is new technology and it’s WebGL 2.0 instead of 1… So every time that I jump into it, it’s not just the text. There’s a bunch of upgrading that it needs. So right now I’m very thankful to all the people that donate money. Most of the money goes to pay for the servers. And recently Google has been pushing more for Https, so I had to pay somebody to fix all that because I didn’t have the time and energy and knowledge.
So that’s that.”
Admittedly, there are a lot of factors to account for. Patricio and Jen also welcomed their second child this past year, so the book is understandably on hold for now. In the meantime, we’ll content ourselves with the Pixel Spirit extension pack that’s on the way.
Interestingly, long before The Book of Shaders even existed, it was while playing with his young son that Patricio had a flash about Signed Distance Fields. This moment turned out to be the catalyst of sorts for he and Jen to finalize the idea for the book.
“I was playing with dough with my kid and I was saying ‘oh this is just like SDF’s!’ It comes with these basic shapes and this could be how you… and I tried this a couple of times when I was teaching at ITP and SFPC. In both classes I took clay. I’d bring clay to class, to explain Signed Distance Fields. And it didn’t work completely, at all. Like it was… in my mind it was very clear the relationship. Like how you combine SDF’s and all that. But it wasn’t clicking for people.
I was explaining the same thing of like, imagining SDF’s as height maps. So I was asking everyone to imagine it like a shape and in clay doing that shape but kind of like a height map. And I was like, ‘this is going to blow everyone’s minds!’ and you cut the top off and you have the shape and you go down [to the bottom] and you have the opposite.
So we started constructing 2d images and subtracting on another… I was super excited but I was seeing that I was… you know when you’re in class there are some times where you bring in a metaphor and it immediately makes everybody keep going with you? And then there are other moments that you see everybody’s like ‘ehhh … I see that you’re very excited but it’s not clicking’ .
So I started working on the original idea of The Book of Transformations. That was the original name of The Book of Shaders and my wife was like, ‘That’s too obtuse. Makes sense in your head, I understand that you like mystical things but you should name it something that’s more representative of what it is.’ And it was also kind of related to this idea of The Book of Shadows, which is like this mystic arts thing. So I was like ‘ok, Book of Shaders’ it will be.
And with Pixel Spirit [deck] I had the same idea of having a… I was clear that I wanted to make a progression, so I started making this like progression with SDF’s and it wasn’t clear the shape that it would take.
But then I found a website — a Chinese website that prints cards. And it’s very hard to find places that print the back of the cards differently. They usually do the same pattern for all the backs and you can put whatever you want in front. And I started messaging with them. Part of my Jungian training … I was into astrology and Tarot and I was like, ‘ok, I want to make a Tarot deck.’
I like Tarot so I started shaping it and I got obsessed with this idea that all the Arcana have to be represented. And then I struggled with keeping the educational progression there. In the process I also realized that I can put the code in with functions and eventually it can be a library. And I was slightly seduced by this idea of all my code being open source and like, I don’t know what happens when I put it in a git repository and it would be excellent to have something that is open source in a way, but you still get paid. So it was kind of like this idea that people will be buying the code, technically. They’ll be buying the thing.
So I was kind of playing with this idea in my mind.”
We’d chatted online months prior about how when you put together the parts of your interests that feel separate, that that’s part of becoming authentic. Patricio seems to do this over and over throughout his work. He’s also created and open sourced a host of tools for himself, like the GLSL viewer, a console-base OpenGL Sandbox. As we wrapped up our time together, I wanted to ask Patricio about the importance, or not, of style, process and creating your own tools.
“Ah yes!! Yes, well. I think… I think it’s very hard for an artist to be happy with his own work. And I think that’s ok. I think the eternal question for any artist is kind of like ‘am I an artist? Is my work worth my being an artist?’ And I think without that question things can crumble.
Unless you’re like Picasso. He seemed to have the ego of being like, very confident. But that didn’t play in his favour. I think having that question and being uncertain in your work is actually what keeps the movement alive. You keep wanting to be better than what you are. You want to be more true to yourself, so it allows you to be checking constantly on what you’re doing.
I guess I have the privilege to work with engineers who are at a moment of their lives where they’re very open and generalizations are bad in general, but I think there is a lot of respect for design and for art. Especially in niches like the gaming industry or VR where obviously art sells and developing an eye really makes a difference.
Working with artists and engineers is this privileged space of like… an engineer will make a perfect PBR model, totally photorealistic all the way down to the atom where things get rendered super realistically. The illumination is gorgeous. But then if you see what is in the industry, it’s totally stylized, not at all realistic.
And there are all these things that artists know. Like that shadows aren’t black, they’re full of light. And actually if you put warm colours in shadows you get more punch in your images. So there is other like, more analogue knowledge that when you put one result into another one, kind of becomes obvious that there is the knowledge, but you also need the eye.
That comes from contemplation, from practice.
Now that I have a kid I have less time to waste. And I have more clear limitations on time, on what I can do and even what I can put in my house. So I’m more comfortable with doing what I can do. I can be less frustrated… a little. Last Thursday, I had a long conversation with my wife, ‘I’m going to quit everything, we’re going to move to nowhere, we’re going to eat like, berries from the trees… and I’m going to do art!’ So that’s still there.
As for my framework, it’s kind of like the tools I was talking about yesterday and yea, I try to develop tools that are flexible, that keep me iterating and allow me to jump into something and lets me get excited as fast as possible.
I see people sometimes get stuck in trying to get things to be perfect. To get things to be perfect takes a lot of time. And I really admire that and actually I think, (also having kids kind of gives you the view of the long game,) eventually with time you kind of have your own pre-prepped stuff, and your tools. And that becomes part of your process.
Then you learn that you are the resonant box of this equation. So your own motivation is equally important and if you’re not motivated or like, enjoying what you’re doing, it’s very easy to make something that looks like it bores you or burns you [out].
So I try to do something that excites me. But I think I struggle like the rest in other places.
If making tools is your cup of tea you can learn a lot about tools and about the science behind them, which is fascinating and I like it. But I feel that at the beginning the fastest that you can feel the canvas… you know this thing about the white canvas and the paralyzing — one of the best ways is to cover everything up.
You cover, you get rid of that moment and you start putting colour there. So whatever gets you moving is the best way. If you’re using somebody else’s tools, go for it. That will get you moving and that will get you feedback on what you’re doing. And then there’s always time for becoming more obsessed with your tools and I think that’s more of like, an after, a consequence of getting frustrated with someone else’s tools.”
Just as we were rounding off our discussion, I happened to ask Patricio about Raspberry-pis.
“I love the Raspberry-pi. It’s a whole linux situation. Do you use Mac? So the Terminal, once you make it your friend, I think you gain so much because you gain a lot of autonomy in a lot of senses. You can make your own website, you can admin your own website.
So Linux is the free unix right? And the unix system is developed so well. I think it’s from the 60’s? And the architecture behind it is brilliant. That’s the reason that it’s still used today and is in every phone.
The architecture behind it is such a brilliant piece of engineering and it teaches you so much about how to make your own tools and pipeline. The whole concept is to not have one program that has 100 features, but instead make one program that does one thing ok and is able to communicate with others.
Your computer becomes your IDE so that you don’t have a computer that has programs, you have computers that have commands and those commands get extended the more programs you make.
So each program is more like a command. It changes your conception of computation and if you connect two computers and like Linux, knowing more about Linux and Unix allows you to think more distributedly. Like, ‘Oh I can have a process that enters into this machine and triggers this process…’ If you want to do server side, client side, distributed smart homes… there are so many beautiful things. The beauty of Raspberry-pi is that it’s so cheap that you can have a whole army of them.
I do silly projects like buying broken cameras at antique shops and making like, an instagram camera that [runs on a Raspberry-pi]. You know, you can do crazy things. It’s like a toy that is super under-budget and accessible. So that’s why I love them.
As a programmer, it also keeps you sharp with the code. And they’re doing great work helping kids to learn too. My school used to have… I used to code with LOGO at my school and I remember the year that they switched suddenly. Computer classes weren’t about programming LOGO anymore. It was learning how to use the word processor.
I actually had the same experience in Canada in the 90’s. One year of LOGO and suddenly in the 2nd grade, I was instead learning how to type without looking at the keyboard.
Do you wonder what happened there? Microsoft. Well, at least in Argentina it was a political move.
They said that the future of computers would be knowing how to type on a word processor. So schools and education were sold that the future would be Office. They were justifying, ‘your kid will get a job because he will use Microsoft Office, and use Word and Excel’. And it’s so lame. It’s such a lame thing and I think they pushed back the evolution of things so much.
I think the Raspberry-pi really has the spirit of the beginnings, where the computer is a tool. I have this complicated relationship with Apple too where I think in a way Apple does the same thing [as Microsoft]. They want us to be users and that’s why it’s so hard to ‘misuse’ those computers. They dictate what we can do and can’t do with their computers.
But the Raspberry-pi is putting up a good fight of like, this is a computer and a computer is a tool, and a tool that can do anything that you want. And there are all these wonderful projects for kids and for grownups. They’re explaining to us that they’re tools and they’re proposing a different space very clearly, through hardware, which is not the typical user space, which is so passive. But having these kinds of movements that put you in the front seat and gives you like, a tool in your hand. I think it’s great. So yea, I support every kickstarter related thing that I can.”
So, that wraps up my conversation with Patricio, to whom I extend many thanks once again. Sometimes I sit down with friends who do fun creative, digital things, generally, and I record the interesting things they have to say. If you’re into it, keep your eyes and ears peeled for more.