A Computer Scientist Hangs Out with The Artists for a Decade
I have been interested in the intersection of mathematics and the arts since I can remember. Computer Science is applied discrete mathematics. In my early academic career, this intersection was emphasized in grad school, followed by the Digital Arts & Sciences (DAS) programs at the University of Florida (UF), which I helped to form. Then came ATEC (Arts, Technology, and Emerging Communication) in Dallas (actually, Richardson). ATEC and AHT are part of the University of Texas at Dallas (UTD). I have been in ATEC (now AHT) for a decade. What new things have I learned about the connections between computer science and the arts?
Artists already Code
I remember when I first learned about Processing it was from an artist, so it is not as if we computer scientists are there to teach the standard entry point in computer science curricula (programming, or coding as it is more colloquially known). Here is the kicker. When a computer scientist talks about code, it is usually code that is based on writing. You write a program.
But in the arts world, the “node-based” approach has gained a foothold for at least a decade. The difference is control flow (i.e. the flow of control so common in written languages Python and Javascript) versus data flow (i.e. the flow of data through a network of functions). The artists and computer scientists are encoding process differently. All analog computers (mechanisms in existence long before their digital progeny in the 1940s) are data flow. For example, an astronomical clock from the middle ages is a data flow machine. The computer scientists would clarify data flow in terms of functional programming, which still lies on the outer fringes.
Data flow approaches are common in media art from 3D design and fabrication (such as Rhino Grasshopper), video compositing, and modeling package interfaces (such as Maya).
Aesthetic Computing
This is a class I taught for 12 years at UF. The idea behind the class was to bring a broad understanding-of and appreciation-for aesthetics (of the sort familiar to artists) to Computing. Several of us met in southwestern Germany (Dagstuhl) in 2002 and went at it. There are many fascinating people involved, and a book was produced. From my own perspective, aesthetic computing was about creatively expressing and representing mathematical constructs at the core of computing. For example, imagine a mathematical tree data structure but created out of clay, textiles, or quilling.
Even though these efforts may seem non-utilitarian, the task of making creates attention. The person doing the making learns about the constructs because making enables attention, something at the heart of memory and how we process the world. So, the artists are creative and know how to make. This means that this art-driven approach can be used when teaching computing. If you enable attention, that is all you need. The idea that the maker was doing the learning, and not that our goal is to create wider communication and public consumption, took the better part of the decade. See the Dear Data project for ideas on creatively representing data, or browse through the designs of Martin Krzywinski. It doesn’t always matter if most folks cannot immediately grasp the data using these methods. The goal is attention, and not only communication.
Observing
I don’t know why I missed this while doing aesthetic computing at UF. The other thing that art method emphasizes is observation/perception. You are sitting in front of a fountain or a work of art. Can you translate these observations into the language of computing? In this case, computing becomes a way of seeing, almost like the green rain present in The Matrix. Computing does not always have to be about utility — solving a problem. It becomes a sort of interpretation not unlike observing something and then committing that observation to a written result. There is a wooden table in front of you. It supports bowls, glasses, silverware, cups, and other odd objects. You see it but how to interpret this structure as JSON? What better way to learn about data structures than by seeing them in front of you? I had some nascent ideas in 2014 on a TedX stage, but I admit that this was a bit discombobulated and without sufficient elaboration.
Challenges
Even though these two strategies (observing computer science, and creatively representing computer science) have worked wonders in my AHT classes, there are some significant challenges for wider adoption.
Take observation. Most would say that computational thinking is centered on problem solving, whereas I and my students are out and about looking at the world through a filtered computational lens. Especially, within engineering, the goal is to forge new devices and interactions, not use the theory to explain what can be seen or touched. There is a completely different mindset spanning perceiving and utility. Mathematics educators have long promoted math walks or math trails. This needs to be extended to computer science, information science, and statistics.
Now, let’s take creative representation. Computer scientists grow up with the idea that mathematical notation is sufficient for representation. Maybe a diagram or two. But why have students create artwork? The answer, as indicated earlier, is that both observation and representation enhance attention, which lays at the core of cognition. That is the point. Utility is not the only outcome for computer science.
Summary
Where do we go from here? The artists and computer scientists have a lot to talk about. This essay suggests that this conversation is a two-way street. It is not just about technology serving the artists. It is about art methodology (observation and representation) being useful in understanding mathematical disciplines such as computer science. Both observation and representation foster the essential psychology of attention. The more you attend to something, the more you learn.
