Processing for Android

Andrés recently asked me to write a preface for a book that he’s writing about Processing for Android. It was a good opportunity to write down what I find interesting about Android, how it fits with Processing, plus a little bit of the history of the project. The book is available for pre-order, and the preface follows…

The Android version of Processing got its start when I received an email from Andy Rubin back in February of 2009. He was interested in having Processing be more tightly integrated with Android, and the discussions led to initial funding that helped us work on building an Android version of the project during the year or two that followed. As one of the test applications, I used some code developed by Casey Reas (the co-founder of the Processing project) and we were elated to see the first version of it up and running on the G1, the very first widely available Android device.

One of Casey’s artworks running on the G1 (circa 2009)

I was especially excited then, as I am now, about the Android platform as an incredible canvas to work from: you have mobile devices with a range of ways to communicate with the outside world (mobile network, wi-fi, bluetooth, etc), higher performance CPUs than even the desktop machines we first used to develop Processing in 2001 (not to mention GPU performance better than the SGI workstations we were using around the same time), and a broad range of sensors — accelerometer, magnetometer, GPS, and gyroscope — that opened an amazing number of possibilities for applications and ideas. It’s the platform I wish we had when we first started the project, with its focus on all the interesting use cases that come from openness and having the right set of tools and APIs to go along with it.

Technically this is the model 2500, but you get the idea (Source)

I’m especially excited, for instance, about possibilities like completely reinventing the interface for mobile phones, because all of those pieces are available to be customized, and you’re just a few Activities or Fragments away from an entirely different end-user experience.

Since the Handspring (later Palm) Treo, mobile phone interfaces really haven’t changed much — a grid of applications as a home screen, and a phone “app” that essentially emulates the interface of a touch tone phone. The Treo’s interface dates to 2008, which built on the PalmPilot interface from 1997, which itself referenced other organizer tools and too many icon-driven interfaces to mention from decades prior. The touch tone phone, meanwhile, dates to at least 1968, with the Model 1500 from Westinghouse. Suffice to say we’re still working with a set of forms, interactions, and metaphors that have been heavily repurposed over the years, and I’m drawn to the idea of people experimenting with alternative interfaces and having the ability to rethink how those elements might work. Innovation comes from getting people the right tools to play with ideas, and while a new phone interface may not necessarily be around the corner, there’s so much to be learned during the process itself.

Wait a second—those look like apps! In the 90s? The PalmPilot Pro, part of the second generation of Palm PDAs (Source)

This idea of experimentation and exploration is at the core of the Processing project. Projects are called “sketches,” to keep people in the mindset of writing a small amount of code to get started, and not overthinking it too much until you understand your problem better. This is not unlike being able to scribble something in a sketchbook and flip to a new page — perhaps even tearing out the previous page and throwing it out. This approach is a little backwards from typical software engineering practice, which encourages figuring out structure first, and filling in the details over time.

But we found that this iterative approach was helpful not just for our professional work, but also with teaching beginners how to get started. So we built Processing in a way that allowed people to write a few lines of code that would make something happen — no knowledge of classes, threads, animation loops, or double-buffering necessary. Over time, as the user’s experience grew, and their ideas more ambitious, they could learn the details of these more complex concepts when they were ready.

Interestingly, this also mirrored things that we, as more seasoned developers also wanted: why write a lot of the same boilerplate to do the same things? Why get RSI retyping the same handler and utility functions? Could we give people a toolbox that was useful enough on its own that starting a project didn’t mean collecting the initial set of libraries that were required just to get things done?

The Processing Development Environment, circa 2010. Type a little code, hit Run, and it automatically becomes a fully packaged application that’s working on the Android Emulator (or on a phone).

A related idea was simply how one gets started. Most Integrated Development Environments (IDEs) require a lot of background — even training — to use. Even with 20 years of coding experience, a friend had to sit me down to explain how to use Eclipse to set up a project. With Processing, it’s a matter of downloading the software, writing a single line of code, and hitting the Run button. This gets even more important with a platform like Android, and we set out to make the Android version of Processing be just as simple. We want users to be able to download the software, write a little code (or open an example), plug in their phone (or tablet, or who-knows-what), and hit a button and see something show up on the device. Once that works, it’s all downhill from there, and hopefully you’re having enough fun that you’re on your way to losing the rest of the afternoon hacking away on your idea.

This book covers a wide landscape of how the Android platform can be used, and how Processing can be a helpful bridge to enabling everything from quick experiments to professionally developed applications. It’s exciting to have Andrés writing it because we want to see even more people building and playing with the platform, but also because you couldn’t have a better expert in how Processing itself works. Andrés first started working as a core committer to the Processing project when we started to fold his popular OpenGL and Video libraries into the main project, which over the years led him to being in charge of the 3D and Video portions of Processing. His experience with 3D got him deeply involved in the internals of how OpenGL works on Android, and his work there was the basis from how he rebuilt the desktop version to move from the old style fixed function pipeline used in GL to the brave new world of shaders and highly multicore GPUs. It was through this experience, combined with being an avid Android user, that Andrés became a core maintainer for the Android portion of the Processing project, and has helped shepherd it ever since, with everything from ongoing development, to mentoring Google Summer of Code projects, to finally, this book. Suffice to say, you’re in good hands.

I hope you’ll enjoy the book, and we can’t wait to see what you build with Processing!