Lessons from The Expanse: In Space, No One Complains About the Documentation
A futuristic look at information technology.
Note: An updated version of this piece can be found here.
One reason fictional television shows are interesting is that they cause us to reexamine on what we think we know. Game of Thrones can cause us to re-examine our political systems, Looney Tunes cartoons can cause us to re-examine our intuitive theories of physics, and The Office can cause us to re-examine various management theories.
Having done the hard work of watching the critically acclaimed SyFy series The Expanse, I thought it’d be interesting to use the show as a lens to look at future of coding, data analysis, and data visualization.
If you haven’t seen The Expanse don’t worry — I’ve included relevant clips where appropriate.
Much of what makes The Expanse seem so real and plausible is that it consists of reasonably extrapolated advances in medicine, space-travel, and information technology. These include a) intelligent devices that respond to intention, b) mature Virtual / Augmented Reality technology, c) gesture based interaction, d) data-literate characters, and e) undercut hairstyles.
Before we begin, to make sure we’re all on the same page, let’s cover one thing that The Expanse misses: if people are programming in space, they are also likely to complain about the documentation.
0. Coding (and documentation) in space
In The Expanse, and other sci-fi shows, nobody programs and codes software. And if you don’t code, you don’t have cause to complain about the documentation of the programming language or open source software tool you are attempting to use.¹
Despite the lack of TV-made, dramatic moments that can occur while arranging symbols on a computer screen, there are a few reasons why future space-faring people will write software while hurtling through vast amounts of nothingness.
The traditional reason for why people might code in space is something like “astronauts may need information-manipulation tools that haven’t been prefabricated to make sure things go smoothly.”
While that answer is technically correct, I submit that there is a better reason: working on a programming project is a nearly resource-free way to cure boredom. If you take reasonably bright minds and send them to Mars for a multi-month voyage, giving them enough to do while they hurtle through the vast nothingness engulfing the universe will be important. Sure, the databases of the spaceship might be loaded with television, movies, music, books and a simulation of what Reddit might be like over the duration of the journey. But having people make measurable progress on a common goal that requires no raw materials and specialized skills is a much better plan. And at least on a spaceship you won’t have to debate about the benefits of an open-office plan.²
If individuals on spaceships will be coding, it is no stretch to say they will also be complaining about the documentation. I have yet to meet a programmer or data scientist who has been entirely satisfied with the quality of the documentation of the tools they use to make their programs.³ (In fact, there are entire companies — like StackOverflow — that are predicated on a general failure to sufficiently document programming languages.)
Why is poor documentation so common? Cue Warfel’s Carpentry Theory of Software Development —
Creating software with a programming language of your choice is like walking into a wood shop that is fanatical about the efficient use of raw materials. Once in the shop, you then use the available tools to build the thing you want. Except that every tool in the wood shop (the bandsaw, the joiner, the drum-sander, the hand-planes, the chisels, the tables, the clamps and the sandpaper) is also made out of wood. By trial and error, it has been determined that while trying to whittle a canoe with a bandsaw is possible, it’s not worth the effort — you end up generating sawdust and taking a lot of time. Instead, it’s more efficient to first make a wooden “canoe-maker” tool, and then use that to make your canoe. Your final creation will be made out of wood, made with tools that you’ve also made out of wood, and so on.⁴
To make this whole thing more complicated, you aren’t the only person in the shop; people who bought a membership are busy trying to make their own tools that might be more efficient at carving the front end of a canoe, or the top of a piano, or whatever else they are interested in. Those who are willing to give their creations away for free call themselves “Open Source Contributors.” And at their workbenches, each person has a complex, multi-step process they use to create their final devices. It’s easy for developers to explain how they made the leap from the second to last tool to the final one. It is much harder to step back and explain what their thing does to someone who isn’t in their corner of the wood-shop, as everyone else is missing the context of all of the intermediate steps.
Still, even with poor documentation, frustrating programmer experiences, and software metaphorically made of out wood, future information technology will accomplish several of the things showcased in The Expanse.⁵
1. Devices that respond to intention
In The Expanse, electronic devices (like futuristic phones) respond to the intention implicit in users’ gestures and speech. Consider this short scene, in which the chief of security transfers information and assignments to her subordinates:
In essence, with one swiping motion, the security chief is telling her device “transfer the files that I think are relevant to the people I am thinking of by vaguely pointing in their direction and not anyone else.” This is surprisingly complicated to get right, as for it to work, the device has to infer both which files and which people are relevant.
The Expanse is also replete with devices that respond to vocal commands, as if Siri were hooked up to a drip-line of performance enhancing drugs.
Roughly speaking, speech recognition consists of two broad steps: first, a machine must figure out what words have been said from the audio data it captures. Second, it must infer what was meant. Not only are the sounds of speech hard for a computer to parse, the second step of figuring out what the user meant is just as difficult. Just like pointing, imprecise language leaves plenty for the machine to infer. Both methods of interacting with computers imply that the devices are responding to the intention of the user instead of what they explicitly state.
For instance, take the phrase “Extrapolate the ship’s trajectories.” Three out of the four words are reasonably unambiguous: “extrapolate” means to predict, given past trends, “ship’s” in this context, are things that have a history of positions, and “trajectories” are position-traces. These words all relate to one another, and in the context of the right program or processing device, these words could be strung together to figure out what should be done.
But unfortunately for us, the trickiest word is “the.” What does “the” mean? “The” is one of the easiest words to use and one of the hardest for non-linguists to define. “The” is technically a determiner, which means it isn’t defined like a noun, and is more defined like a function that takes input and gives output. “The” more or less selects the things that are implied by context to be relevant — “the house” is the one you are talking about. But a computer has no idea what is relevant in conversation or natural speech. “The ships” could refer to many different subsets of ships. Is it all ships that the device knows about? All ships that the device has ever tracked? All the ships selected in the last five minutes? Now imagine if the phrase was “Extrapolate the possible destinations” — the word “the” in this phrase carries more meaning(s) than in the first phrase we considered.
The point is that one of greatest tricks software ever pulled was to convince you that it can figure out what you intend.⁶ If you’ve ever been frustrated when using a computer, it’s because you’ve run into the invisible wall of computer stupidity — software is only as “smart” as the quality of the communication between the people who make it. To achieve the illusion, an incredible number of man-hours have been poured into giving you the impression that the computer can read your mind as long as you press the right keys or click the right buttons.⁷
If we think about the steps needed to translate a desire to a real-world action, both voice and pointing are “further upstream” when compared with using a mouse or television remote. If we extrapolate from these ideas even further, it is possible that future programmers and people who interact with machines will utilize Electroencephalography, or EEG caps to detect small differences in brain waves and make machines respond to their thoughts. This, however, is stupendously difficult. Like a reverse blood-brain barrier, we aren’t always so great at getting information out of our brains, even after years of practice. The brain is so complex that for EEG-based programming to have even a shot a working, future programmers will likely have to be experts at explicitly articulating themselves.⁸
2. Mature VR / AR
The Expanse utilizes mature Augmented and Virtual Reality to great effect.
See for yourself:
VR promises to revolutionize not only entertainment, but it also holds a similar promise for the future of data manipulation and visualization.
Future data analysts may utilize VR because it allows for the use of kinesthetic and vestibular senses when exploring complex, high-dimensional data. VR also allows for analysts to work in a collaborative environment, and if done right, it can also make data less abstract. Imagine that a large, complex dataset was mapped to a natural environment, like a forest. Instead of certain data being “correlated,” for instance, analysts could observe that virtual trees grow every time it gets windy.
Having worked closely around and near VR for about half a year, one of the most interesting things about VR is how hard it is to communicate what VR feels like for the first time. I can tell you that once you experience a well-done VR demo you won’t just be excited about the future; you’ll be emotionally moved, as if you had just discovered music for the first time.⁹
No matter how hard I try, words can’t capture what it’s like to put on a headset, and be 30 feet beneath the surface of the ocean while watching sea creatures swim by.
What I’m trying to say is that if you haven’t experienced it yet, you’ll have to take it on faith that VR is amazing when done well.
3. No-touch, gesture-based control
Watch these characters interact with their data display:
As a device for television, this gestural control is brilliant — it frequently allows the mood of the character to be communicated as well. The characters on The Expanse often control devices without touching them or speaking to them. While on one level, this represents the inference of intention; on another level, it is also an interesting lens with which to consider hands.
Hands are interesting because of their extreme versatility and the orders of magnitude of sensation they can feel. By many definitions, hands are some of the most versatile mechanical devices.¹⁰ However, the way we currently use our hands to interface with our devices is pretty crude.
Don’t be alarmed, but no-touch, fine-grained gesture-based control of an electronic device has been around since 1928. Specifically, natural gestures are used to control a theremin, an instrument once described as sounding like “a cello lost in a dense fog and crying because it does not know how to get home.” Broadly, theremins work by generating and detecting fluctuations in electromagnetic fields, which can be modified by one’s hands.
Take a look for yourself:
Currently, one can navigate and interact with VR worlds by utilizing the following methods: proprietary controllers, directed-gazing, or walking around. However, none of these methods can transmit too much information back to the machine.
Instead, I submit to you that natural hand gestures are the missing piece of virtual reality that few are working on, and they are perhaps the best way to transmit a good deal of information to the device controlling the virtual world.
One company working on gestures in VR is Leap Motion. Their approach utilizes cameras to infer where one’s hands are and transmit that back to VR headset.
In addition, Google’s Project Soli is also tackling gestural control with a radar-based device that will scan your hands 3000 times a second.
In comparison, our computer keyboards are limited — each button corresponds to a pre-fixed symbol, just like a piano. With project Soli, there is a chance to make a device analogous to a pipe organ for working with information.¹¹
If you ask me, we waste a lot of time typing individual letters. Court stenographers have gotten away from letters and more or less type syllables. If you had the appropriate gesture technology you could “type” by doing something akin to conducting the air above your keyboard. This may sound like a stretch, but to read some considered concrete details about how this could work, see Hotkeys for Normal People (my short exploration of this topic).¹²
Lastly, combining that data with a quirky set of gestures unique to you is one way to ensure that even if someone were to gain access to your private device, they wouldn’t be able to navigate it effectively. Possibly you could even use fine-grained hand tracking to form gestural passwords that only respond to your exact hands completing a specific shape. (If your hands are scanned 3000 times a second, you could get a reasonably strong password in a short amount of time.)
4. Navigation and Future User Interfaces
According to The Expanse, Futuristic User Interfaces will have lots of excess lines and occasionally utilize dark blue:
But more interesting than how the data is presented is what data is presented at all.
On spaceships, there will be plenty of data internal to the spaceship to display — the status of the components of the ship, the state of the engine, health data on each person inside the ship. Also, it will be useful to visualize what might be close to the spaceship, for docking procedures and last minute avoidance.
More interesting, though, is the navigation data. Clearly, some of the data displayed in space will concern a space-traveling vehicle’s position in the solar system, along with relevant solar objects. Or in other words, space maps.
In our current approach to space travel, however, space maps are largely for psychological reassurance. This is because, by and large, spaceships aren’t taken for major detours along their routes. Unlike science fiction, we have not gone far enough beyond the moon to explore the possibility of a ship making an emergency rendezvous or side trip to a Martian moon. Currently, all major space maneuvers are pre-planned.¹³
It turns out, though, that autonomously navigating through space is pretty hard. Yes, it’s mostly empty. But gravitational and centripetal forces conspire to make space travel a bit more interesting. In the same way that tides, trade winds, and doldrums make traveling over the oceans more complicated, space is full of more and less optimal routes to take.
What this means is that naively traveling between solar objects, like aiming for where a planet will be after the months you expect your journey to take, is not efficient.
Instead, there is a spot between the Earth and the Sun where the gravity of both objects cancel each other out. It takes little fuel to navigate away from this spot — no need to fight gravity, because the gravity at this point, for all intents and purposes, doesn’t exist. For various mathematical reasons, there are actually five of these locations, called Lagrangian points, for every pair of objects in space, where the centripetal force of orbiting the sun is canceled out by gravity, and so on. These points are always in flux, as the planets orbit the sun. Hopping from one of these areas to another can save an incredible amount of fuel. In fact, researchers have determined that there is a network of possible routes that maximizes how much time a rocket might spend in the the low-force requiring areas.
Determining where your are relative to these low-energy-to-navigate paths will be important when traveling through space. Few things could suck more than running out of fuel in space or getting stuck in a place that is hard for other ships to get to. Without visualizing the invisible contours of force, it will be impossible to figure out how much fuel one will need to, metaphorically speaking, perambulate through space.¹⁴
5. Data-informed characters
Most characters in The Expanse are data literate. While some are engineers and machine specialists, all characters demonstrate some aptitude for working with structured information. (Before I lose you, rest assured that the thesis for this section is emphatically not “All people will be data analysts in space” nor its related extension “teaching people how to code will solve all of our problems until the chimpanzees learn how to do it.”)
These days, astronauts are managed by a team of people looking over their shoulders, working through checklists, and each space-action is highly choreographed. Space is no place to be imprecise. Therefore, it seems no small stretch to say that people who will fly through space must be able to reason probabilistically about the chances that any one of their actions will lead to unfortunate outcomes. On some level, our brains use probabilistic reasoning to get us through the day. Unfortunately, we aren’t currently trained to use precise probabilistic reasoning when we use conscious deliberation to come to conclusions.
Space travelers and space workers (like asteroid miners, if we get there), will likely be trained to accurately reason with probabilities. Decisions made in space will be nothing if not data-driven. This implies that on some level, future space travelers will be data literate, even if the data is their own first-hand experience. The only way to become data-literate is via training, which also suggests that there should be fewer disagreements about the consequences of actions in space: given the same data, and a trained reasoning process, people should come to the same conclusions. If someone has personal experience with a particular problem, they’ll be able to convince everyone else, because no one will be stretching their inferences towards the conclusions they are biased towards. In other words, decision-making training is one way to make space travel safer, and will likely be mandatory for future space travelers.
Conclusion
When thinking about the evolution of programming, code and information manipulation, it is helpful to consider other types of tools. Hammers, for instance, have undergone an interesting evolution. When hammers were first used, there was no standardization. Anyone who could find a rock they could grab with their hand, could, if they wanted, start hitting their thumbs. Over millennia, hammers have been standardized, and the pace of hammer development has slowed considerably. Here, for instance, is an engraving from the early 16th century (I’ve highlighted the hammer):
In our modern times, there are a variety of standardized hammer types, each with a specific use — blacksmiths, electricians, stone masons, and carpenters all use different hammers. I think it’s safe to generalize that the same thing will happen with coding: currently, programming tools are rapidly evolving, and they will likely settle and cool into more stable forms in the future.
With gesture-based control, mature AR/VR, and devices that respond to intention, users in the future will have it much easier than we do now. Whether the quality of documentation will improve with time, however, is something I won’t try to predict, and only fate can know.
Endnotes
[1] Though feel free to start loudly complaining about documentation in general after you finish reading this essay. ^
[2] In an attempt to figure out if any astronauts have actually programmed in space, and thus had cause to complain about the documentation of the programming languages they used, I reached out to NASA and celebrated programmer Peter Norvig. So far, I haven’t heard back from them. Also, there is plenty written about the documentation of how to use the space ship or what to do on the space missions — see here. ^
[3] From personal experience, the Python package SciKit-Learn has good documentation. And according to my friend Vynce, Perl 5 had “amazing docs.” ^
[4] This is the story that is told, at any rate — at some point, the tools used to make the tools available in the wood shop will require other trappings like a power source, or, heaven forbid, C++ code. ^
[5] While programming isn’t showcased on The Expanse, one of the better- known sci-fi books to feature programming in space is Vernor Vinge’s award-winning book A Deepness in The Sky. In this book, programs have accumulated over millennia and ships employ Linguistic Archeologists to dig through layers of old code and modify them as needed. Still, no character complains about the documentation of said code. ^
[6] Yet if you kick a computer, it won’t get the message. Clearly, we programmers have been slacking. ^
[7] This trick is so ubiquitous that it makes learning to program a computer frustrating. At some point, everybody who learns how to program confronts one of their subconscious assumptions: that words transmit all of the information we communicate when we talk with someone. It turns out that we use words to communicate recipes for inferring what someone else means, rather than the full meaning itself. And so one of the first hurdles beginner programmers must surmount is figuring out how to communicate the entirety of what they mean with the actual words they use and nothing else (like presuppositions). ^
[8] The brain can really only “output” two things: signals to contract or relax muscles, and minuscule fluctuations in electromagnetic fields. That’s it. Moving away from motor-control based input to EEG-based input would be both a major paradigm shift and quite difficult — using the best technology available now, it’d be (very) roughly equivalent to deploying one hundred air-pressure sensors equally around the globe to try and infer specific bicycle traffic patterns in Golden Gate Park, organized by bike type.* ^
*This analogy suffers from the fact that air pressure is indirectly related to bike traffic while EM fields are more directly related to what neurons do. So the issue with the air-pressure/bike analogy is one of how the sensors relate to the thing being sensed. That, and the scale of the problem is probably too small. So the two issues are the relationship between the entities, the scale of the problem, and the fact that neurons aren’t always arranged along a gradient. Okay, so there are three issues, and these don’t include the fact that EEG devices sum over both large areas as well as all six layers of neurons beneath every spot on your skull you can poke your finger at. And so on. The moral of this paragraph: brain analogies are hard.
[9] This doesn’t happen with every VR experience, but it can happen with some. ^
[10] I don’t expect people under the age of consent to, legally speaking, fully understand this point. ^
[11] When playing a pipe organ, all you have to do is press a button to make the instrument sound like flutes, or organ pipes, or trumpets. (However, this metaphor also ignores the idea of using “chords,” or multiple buttons at once, to encode different words — stenographers and closed-caption transcribers utilized chorded keyboards because it is faster and more efficent.) ^
[12] Dear Project Soli: please send over a dev kit as soon as possible. ^
[13] When settling on the route of the Voyager probes, the researchers at the Jet Propulsion Laboratory printed out 10,000 possible trajectories, each involving only four major course adjustments. ^
[14] It will likely be computationally infeasible to simulate all of the possible optimal routes in full. There are analogs to the Lagrangian points for more than pairs of objects. Just as chess computers only calculate what is relevant instead of computing the entire tree of possibilities, space maps will likely be dynamic, only showing the calculations and projections from the local “branch” of computational exploration. ^
Note: Thanks for reading! All clips from The Expanse utilized under Fair Use. This piece was enabled by Kineviz, a creative studio focusing on data visualization.
