Untold AI — A conversation with Chris Noessel of SciFi Interfaces

The Senior Design Team Lead in the Watson Customer Experience at IBM and man behind Sci-Fi Interfaces gives a behind-the-scenes breakdown of his incredible Artificial Intelligence Infographic

Sci-Fi Interfaces is one of those websites the Internet was (originally) built for. It’s a passion project focusing on an incredibly niche subject, created and maintained by someone with real subject matter expertise.

Much of Chris Noessel’s professional work, academic study and enthusiasm for science fiction has come together in the creation of the Untold AI infographic. It’s breathtaking in both its scope, and as a graphic design achievement. When I saw it a few weeks ago I knew I had to reach out to Chris and find out more. He was generous enough to answer my questions and provide insights into not only the creation of the piece, but also on how we talk about Artificial Intelligence within science fiction.

The Adjacent Possible: First, can you give some quick background on scifi interfaces, and then, what inspired you to take on the Untold AI project?

Chris Noessel: In 2005 my grad school thesis advisor, friend, and author Nathan Shedroff approached me with an offer. He had noticed the connection between the Motorola flip phone and the Star Trek communicator design, and wanted to know how far this connection between sci-fi design and real-world design went — and what it means for designers. He wanted to write a book on it and liked my writing. So we agreed to coauthor it, and embarked upon a 7-year journey studying as many sci-fi interfaces as we could and putting our findings into a book, with Rosenfeld Media: Make It So: Interaction Design Lessons in Science Fiction. That book was a big-picture overview of massive trends and topics in sci-fi, but I was also interested in the movie-by-movie analyses as well. So just before publication of the book, I began the scifiinterfaces.com blog, based initially on the research I’d done for the book, but since then it’s gone in many new directions. Its mission is to analyze and critique interfaces in sci-fi movies and television shows to increase our literacy of that medium (and build up an informed skepticism about it), and to use these fun examples to keep the readership’s real-world design skills sharp and up to date.

The Untold AI idea came about when I was invited to participate in the Nordic AI Retreat organized by Andy Budd and ClearLeft. It’s run like an unconference, with attendees suggesting topics at the start of the day and seeing who is interested. I had recently completed a series of posts analyzing the movie Forbidden Planet in light of the Fermi Paradox which looked at a synthesized list of answers to the Fermi question, and then noted that Forbidden Planet is the one film that portrays the most likely answer. It’s closest to the science. So when it came time to make a suggestion for an unconference session about AI at Juvet, I wondered how the sci-fi squared up with the science, and suggested it. The topic attracted seven of the participants, and we started to gather candidates for the survey. We even made some headway identifying takeaways, but ran out of time. So I picked up the project when I got home, and shifted from pen and post-it notes to digital tools.

Plus I think it’s an important virtue to look at things from a big picture, systemic perspective. Finding the extents of a possibility space is often a major insight in and of itself.

I should also note that my job at IBM entails lots of AI work, and I published a book last year on a new mode of interaction enabled by advances in AI, and I’m just starting to write fiction about narrow AI, so I’m interested in the topic from many perspectives.

Ok, just a quick break here to say, if you like Q&As like this one with authors, academics and other creators, or original essays on science fiction topics, please consider giving The Adjacent Possible a follow, or better yet, please sign up for the Adjacent Possible newsletter, which in addition to giving you a heads up on new content like that mentioned above, is jammed pack with links to the most interesting sci-fi related stories about A.I. space, tech and cool innovations.

The AP: This was a pretty massive project, what was the scope and parameters of what you were going to look at?

CN: Oh my yes it was. It took nights and weekends for several months after Juvet. (And it’s still going!)

The scope was big by design. I wanted it to be pretty comprehensive. My authority and expertise in this domain is built on screen sci-fi, since you need to see a thing in use before you can evaluate it. That means movies and TV shows. There’s a lot of sci-fi literature that investigates AI (for instance, I’m reading the Culture series right now by Iain Banks, and it’s got AI baked deep into its worldbuilding), but I just don’t have authority in that domain. But still, screen sci-fi is massive, and I like to be as comprehensive as possible. So I looked back through all the movies and TV shows I could find that had an AI component: A robot, a spaceship AI, a disembodied one. I looked at lists people had collected online. I looked for keywords on IMDb. I don’t think I got them all, but I think I got most.

I’d have liked to include more narrow AI, but that would have expanded the survey of shows by an order of magnitude, and ultimately, the purpose of the project is to help guide sci-fi to be in-line with the science so we hedge our bets in the right way. Narrow AI is already here in the world around us (and less of an existential threat) so there’s not as much need to address it at scope.

The AP: Based on your study and research, how has the depiction of AI in movies and shows changed over the last 50–60 years?

CN:If we’re just looking at screen sci-fi from the 1970s on, the biggest change is in the literacy of the audience. Back when disco was king, computers were something that existed in giant organizations. Neither writers nor audiences had a good grasp of what AI was and could do. Kubrick blew everyone’s mind with 2001: A Space Odyssey and Star Trek (the original series) started people thinking about A.I. in pop sci-fi. In that time and through to the late 1980s, AI was either helpful robots (think R2-D2) or evil, disembodied AI (think HAL) with occasional evil corporate AI thrown in (Robocop’s ED-209 Enforcement Droids).

The personal computer revolution of the 1990s and the smartphone revolution of the 2000s increased the literacy of writers and audiences, and sci-fi had to get more nuanced. More people had daily interactions with technology and were thinking about it, so more people were realizing the systemic effects that mass penetration of technologies had, even those that were less than full-blown general AI. In the late 2000s and 2010s, the Marvel Cinematic Universe has done much to bring views of AI back to the positive, in very smart ways, Ultron notwithstanding. Most every kid (and a few blog authors) want to be Iron Man with a JARVIS/FRIDAY, or a Doctor Strange with a Cloak of Levitation.

I think we’re close to a point where sci-fi and audiences are informed enough to begin to influence their thinking about speculative AI in productive ways.

The AP: What were some of the more surprising findings from your research?

CN: In no particular order…

• The US produces the most sci-fi AI shows, but not the most movies. That honor goes to India, and they outproduce us by a lot.
• The UK is your best bet for shows that think about AI (though the #1 show, Person of Interest, is by the “home team”).
• I’d read the short story “Roller Ball Murder” as a teen and liked it. But because of this project I finally got around to watching the original 1975 movie Rollerball, and it’s one of the worst films to be made in the last 100 years.
• AI shows have been trending more positively, but are still slightly negative.
• Cinemaphiles (like readers of my blog) probably think more negatively about robots than the general population.
• A single visualization will resonate more strongly than 19,600 blog post words that precede it. I’ve gotten much more attention for this graphic than the blog posts, but they essentially say the same thing.
• It was surprising to find Prometheus in the top 10. I really didn’t like this film. I would not have guessed it would wind up in the top 10 of anything.
• I was surprised that so few manifestos spoke out against autonomous weapons, which terrify the shit out of me. (After doing my research I was compelled to venture into publishing my first speculative fiction as a result.)
• There are so few sci-fi shows that address either goal fixity or economic control of humans, which are some of the biggest concerns of scientists.
• There are very few sci-fi shows that address AI helping humans fix problems that we cannot solve alone, which seems to me to be the big promise of AI. The Hitchhiker’s Guide to the Galaxy is one of the notable exceptions.
• And, of course, I was surprised that sci-fi misses about half the science.

The AP: You’ve turned your research into an absolutely stunning visual representation. How did you decide on that for an output, and tell me about some of the details in the execution?

CN: My undergrad degree is in graphic design from the University of Houston, with its Cranbrook connections, so I think they would be very proud for that praise. :) Thank you for saying so.

One of the failed representations.

The analysis

The big challenge is that it’s a lot of data, and it’s a multistep process. But I felt it was important to show that there was a great deal of analysis that happened, i.e. that I’m not just posting a hot take, so kept working at trying to simplify understanding without simplifying the content.

There were clearly two sets of base documents: sci-fi shows about A.I. and A.I. manifestos. So I needed to cluster those. But comparing these was apples-to-oranges. So there were two sets of abstracted suggestions from each, takeaways and imperatives, respectively, which needed to be clustered. Those clusters could have been organically shaped, but would have made scanning them difficult and text placement next to impossible, so columns made the most sense.

Then there was a mapping between them, showing where the connections were made, and of course, where they weren’t. That pointed to a graph drawing of some sort. I wanted the reader to be able to easily scan the different lines between nodes, so it could have been a “slope graph” but I just find those things ugly. A Sankey diagram is much more lovely, but is also optimized to show volume of flow between nodes, which didn’t make sense here, since all graphs should have the same thinness and let the number of graphs help tell the story of which was most represented. So it’s more of a modified Sankey.

Color was another challenge. At first I just went with unique colors for nodes, but it was just a cacophony. So I started to unify the color palette, trying to encode other takeaways, but not hit people over the head with it. So, for instance, the takeaways are color-coded for their tone (red for negative, gray for neutral, and green for positive) and I kept the lines that connected to the shows this same color — that way you can look at a show and see whether its takeaways are more positive or more negative overall. Similarly, I let the lines between the manifestos and the imperatives keep the color of the imperatives, allowing you to look at a manifesto and see what it is most concerned with, say, designing the right A.I. vs. encouraging an accurate cultural narrative.

The mappings between takeaways and imperatives was trickier. Neither seemed to take precedence over the other, so I colored those lines as a blend between the two. Those blends are really lovely to just look at, especially when translucent, so they add a visual interest to the center of those columns.

The rest is management of attention and visual hierarchy, born of my formal design education and 25+ years studio experience.

The conclusions

One of the most eye-opening criticisms I ever got of my writing was that I had described something true, but I had not answered “So what?” or “Why should anyone care?” or “How can I use this?” So I wanted to include a lot of what comes logically after the analysis as well. These didn’t really fit a single diagram, so I chose to make lots of small ones.

I realized I needed categories for the takeaways the first time I realized how many there were, how hard they were to parse, and that the reasons they were or were not represented in sci-fi were different. So I used my information architecture skills and found five broad categories that seemed to fit, and color coded the imperatives for that. Then I needed to explain those categories, since they were my own invention, so I dropped in a paragraph of text. The main question of the analysis is what aren’t we telling, which begs for a proportion answer, so pie charts help summarize the data that could be calculated by counting things up above.

I had actually stopped there, until I found out the standard sizes for posters gave me an additional row at the bottom. So I was able to add some of the more interesting and fun takeaways that aren’t exactly germane to the core question.

Including the top 10 shows rewards those makers for their good work, and perhaps introduces some sci-fi fans to new titles. The three bits at the bottom are just interesting. The geo bit affords a lovely graphic as well (I’m a sucker for world plots) and especially Fuller’s underutilized Dymaxion Projection.

I should also note that it didn’t fall into place like transcribing inspiration. It went through many, many iterations as I tried to figure out how to tell this complicated story in a way that felt simple.

A colophon

Finding the Sankey/Alluvial diagram mapper was a major problem. There are lots out there, but most don’t handle multi-step diagrams well. Few let you specify order, and fewer still let you move nodes manually. The visual options for most are very unprofessional looking. I shopped around at a lot of places, but finally found Flourish, which produce the most user-friendly tools, controls, and download options available. The raw output from Flourish is below, so as you can see, I still had a lot of work in Adobe Illustrator to massage it from the raw output to the poster version. Fortunately, Illustrator can open SVG like it was a native file, so that transition was seamless.

I want to give a shout out to Hoefler & Co.’s lovely typeface Sentinel. Knowing I was aiming for rich levels of detail at 10 feet and 10 inches means I really needed something that would work not just at multiple sizes, but work when tiny text was next to giant text, and Sentinel fits the bill. Plus it’s a lovely sci-fi pun.

Ultimately I know that most people will experience this first in digital channels, and that there is more exploration that can be done in an interactive medium. That’s an order of magnitude more work and involves some production skills I’m very slow at. So maybe someday if I find a patron or some very talented volunteers I’ll make it more suited to the web. It would be even cooler to make it easily updateable and even so we can crowdsource the addition of literature sci-fi!

For more on this incredible project, check out Chris’s website.

If you’re as blown away by this infographic as I am, and are as big a sci-fi nerd as I am, consider buying the poster.