Things of Interest Relating To Interfaces
I’m interested in interfaces. They tie together several areas I find fascinating — from technology (what can you build?) to consciousness and the human mind (what can you understand?).
Here is an interface:
Here is another:
Here’s another, a slightly less obvious example:
That’s Facebook’s Graph API, an interface for third party apps to connect to and access Facebook’s data. It’s not a user interface, but it’s still an interface — an interface of a different kind, an “application programming interface”. The Apple Maps UI enables users to interact with Apple Maps; a cockpit enables pilots to interact with the plane; the Graph API is a means for systems outside of Facebook to interact with Facebook.
Programmers use interfaces at every level. Platforms like Facebook provide APIs for third-party developers to build apps on top of. Operating systems also provide APIs for software in user-land to interact with the underlying hardware. Every class and module provides an interface for other pieces of code to connect to.
A more profound example: programming languages themselves are interfaces between the programmer and the computer. The programmer might be thinking on the level of “how do I transform a set of user data into pixels on the screen?” The CPU operates on the level of electrons shuttling around in memory. A whole stack of programming languages provide an indirect interface for the programmer to interact with the CPU and achieve their high-level goal.
This post is mostly a brain dump of concepts, and other related links, that relate to interfaces.
Philosophy
The deepest philosophical topic that is relevant to thinking about interfaces is consciousness: what is human awareness and how does it function? How do humans gain knowledge of reality?
This breaks down into two relevant sub-topics: perception and cognition. Perception is the level of consciousness that takes raw sensory inputs and synthesises them into awareness of objects in the world — it is vision, hearing, touch, and so on. Cognition is the level of consciousness that takes perceptual observations and synthesises them into abstract knowledge — into concepts.
It’s a debatable philosophical point, but I side with the view that perception is the basis of all human knowledge. The exact workings of human perception are a subject of study for several areas of science, from neuroscience (which studies the underlying mechanics) to psychology (which studies the subjective internal experience). The workings of cognition are an even broader subject of study. Cognitive science studies some of the mechanisms of human thought, but, arguably, all of the humanities study different aspects of human cognition.
These fields provide knowledge that can be applied to build better interfaces — interfaces that better enable humans to understand the underlying system they are interacting with. (For example, a well-designed cockpit should enable the pilots to understand the overall state of the aeroplane, and all of its subsystems).
However, applying abstract scientific knowledge to the design of a concrete product is a very difficult problem. This is because the amount of potentially relevant knowledge is huge, and therefore difficult to hold in mind when dealing with a problem like “how should this screen be laid out?”. To be practically useful, the scientific knowledge of perception and cognition needs to be synthesised. The end product of such synthesis should be a set of principles that can be applied to concrete problems of interface design.
A specific sub-field of philosophy that is of special relevance is epistemology. Epistemology is the study of knowledge: how do you gain knowledge? Is certain knowledge possible? What is rationality? What is objectivity? One area of epistemology I think will be particularly useful is the knowledge of the nature of concepts and abstractions. This is basically the question of how you group observations of particular things in the world into categories.
Data Visualisation
Data visualisation is the field that studies how to effectively communicate data via the human visual system. In the new age of big data and data science, it has become an increasingly hot space.
The big name in data viz is Edward Tufte, whose Visual Display of Quantitative Information [Amazon] has deservedly become a classic. Tufte studied different information graphics, good and bad, and identified the qualities that differentiated the good from the bad.
One of his big ideas was that graphic designers typically consider data to be boring, and so often jazz up information graphics in an attempt to be interesting, yet end up obscuring (or outright misrepresenting) the underlying data.
Tufte points out three strengths of the visual system: it is capable of vast “parallel processing”, it can quickly grasp the hierarchical structure of information, and it can perceive complex interrelationships between data points. The graph above (of weather in New York) is one of his examples of information design done well.
Tufte holds that good information graphic design requires the intersection of three areas: substance (the meaning of the underlying data), statistics (the mathematical analysis of the data), and design (the visual representation of the data).
Tufte began his career a few decades ago and he focuses on print media. One of his students, Stephen Few, has applied and extending his ideas to software user interfaces. Stephen Few’s book on dashboard design [Amazon] breaks down the different elements of visual processing, and explains how to effectively organise communicate data via a computer interface.
What I especially like about his book is that he integrates related concepts to develop his own ideas. For example, he makes use of the concepts of system 1 and system 2, the idea that there are two levels of cognitive processing; and situational awareness, a military concept denoting the mindset required to handle complex and rapidly-changing situations.
Since he has an active career as a UI consultant, he also provides many illustrations from projects he’s worked on — in other words, he’s able to combine both theoretical knowledge with knowledge gleaned from practical experience. It’s a powerful combination and an example of what I mentioned above — the synthesis of abstract knowledge into usable principles.
However, his work (or that which I’ve read) is focused mainly on passive data consumption. There’s a whole other question around the most effective way to enable user interactions with a data interface (for example, what options you present the user to query, filter and rearrange data).
Programming Language Design
It’s something I mentioned above. I don’t want to go into too much depth, because it’s a huge topic in it’s own right, but programming languages interest me as interfaces designed for smart people to achieve something very complicated.
When programmers build user interfaces, they often seem to approach the problem with the implicit mindset that “make the interface simple” means “assume the user is stupid”, and end up building something oversimplified that frustrates smarter users, or users that have a clear idea of what they want to achieve. The real trick is in building an interface that gives a huge amount of power and capability to the end user, without giving them something overcomplicated and confusing. Fortunately, a good illustration of “the right kind of simplicity” is provided by the top-tier of programming languages, such as Python, Ruby, OCaml, Erlang, Haskell, and Lisp (and its many incarnations, such as Clojure).
These obviously fall on a spectrum — Haskell leans toward a mathematics PhD’s idea of simplicity (that is, theoretical purity), whereas Ruby, inspired by natural language, leans towards everyday understandability.
I wonder if any ideas from these languages can be applied to building interfaces for non-programmers? For example, an everyday spreadsheet, if you squint, looks kind of like a functional/dataflow programming language. And the accessible flexibility provided by spreadsheets has proved enormously valuable — it would be impossible to compute, but Microsoft Excel has probably helped produce hundreds of billions of dollars of economic value worldwide. (Think of all the countless organisations using Excel internally to plan, manage and organise projects).
Some additional interesting things
Brett Victor is an interaction designer who previously worked at Apple and has written a ton of interesting articles. I’m not sure how to categorise his work, but his interests seem to be tied around pushing the frontiers of user interfaces, and designing systems that more effectively augment human cognition. For example, his article on the Ladder of Abstraction proposes a new concept for thinking about complex, dynamic systems.
Nicholas Carr is a technology journalist who wrote a book called The Shallows [Amazon], a study of how the internet is altering the way we think. It’s been a few years since I read the book but one idea I remember is the observation that different forms of technology augment human capabilities in different ways, and computers are part of a category of technology that augments our awareness and cognition. For example, the invention of maps enabled us to gain awareness of wide geographical areas. The invention of clocks enabled us to perceive time much more accurately.
The uber cognition-augmenter is writing, which changed the whole way we communicate complex concepts. (For example, it is much easier to develop a long and complicated argument in written form than via speech). He makes a strong case that long-form pieces of writing (books and essays) are the best way to communicate understanding of complicated topics, and I completely agree. Writing can synthesise vast amounts of information, and though summarisation is always valuable, some topics cannot be properly understood via brief summaries.
This article is something of an unstructured brain-dump of a range of topics I’m interested in. I’m currently investigating product ideas that lie in the intersection of the above ideas, such as tools that bridge the gap between data analysis and decision-making. I’m not sure yet what these will look like, but I suspect there are many big opportunities in the general space of communicating insights gleaned from data to end users.
