In the last year, I’ve slowly learned more about web accessibility. Once I started learning, I was actually horrified that I’d gone so long in life online without thinking to make things I design accessible by wide audiences. When I started learning web design principles in 2015, just making something work on both a phone and a computer was a cool, new thing.
But interestingly, while we brainstorm about how technology can evolve in the not so far future, we’re barely moving in those directions. I understand why; figuring out new paradigms of interactive design isn’t easy. Plus, if someone does have a good idea for a new product, it can take years to prototype, refine, and develop something new.
Regardless, the pace seems extra slow when I stumbled across this video from 2011 of Microsoft’s Productivity Future Vision.
That brought me to reading Bret Victor’s A Brief Rant on the Future of Interaction Design. I’m quite new to thinking about design in this capacity, but now that I am, I’m honestly surprised that this rant was published all the way back in 2011. We don’t have a fraction of the technology Microsoft’s motion graphics animators dreamed up for us 9 whole years ago.
Why does the way we interact with technology matter so much?
There is actually a ton of research and philosophy out there on how the way we interact with tools and technology influences how we think and what we create. Technology is, essentially, a modern tool. Victor has a handy, brief definition of tools.
“A tool addresses human needs by amplifying human capabilities. That is, a tool converts what we can do into what we want to do. A great tool is designed to fit both sides.” — Bret Victor
Learning a little more about the history of tools and the ways we use them is interesting. Martin Heidegger, a German philosopher, has some particularly interesting theories for designers. A big part of his ideas involve how we unconsciously use tools, which can sometimes be limiting to how we utilize them. Some of the key takeaways from his approach is that we’re “engrossed” with the world and see tools as “invisible.”
Here’s a little quote from the philosopher himself.
“We use things not in an analytical, detached way, but by absorption in the practice of using them — our focus is not on the tools or the practice of using them but on the what we’re trying to do (ready-to-handness or zuhandenheit).” — Martin Heidegger
Most of his work seems to imply that this absorption is a bad thing. He points out that we only pay attention to our tools when they break. Giving this a modern parallel, just think about when PhotoShop freezes on you. You probably aren’t actively thinking about PhotoShop as a program or a tool until it gives you trouble and you need to tinker with it to get it working again.
We’re always processing stimuli from different sources.
Heidegger isn’t necessarily wrong, but from what I understand of this, it’s just a byproduct of dual processing theory in psychology. This theory strives to explain the differences between intuitive and deliberate thinking. Intuitive processes are often executed a lot more quickly than deliberate ones.
It also is laid into the fabric of how we multitask; you can drive a car and still think about all the things you need to do that day, that you forgot to water your plants on Animal Crossing, or about anything else. You aren’t solely focused on the act of driving. This isn’t necessarily a bad thing, it’s just how we process different things at once. I think a great deal of how we use tools creatively falls into this same type of dual processing.
With this in mind, let’s get back to Victor and his rant.
Victor isn’t crazy about how most of our interactions with technology are boiled down to swiping flat screens.
Shifting back to Victor, I was torn between agreeing with Victor on some things and shaking my head about some of his other opinions. Frankly, it also says a lot that this was published back in 2011 and we haven’t even reached the technology shown in Microsoft’s Productivity Future vision yet. It makes me question just how much interaction design has really evolved or even been widely discussed in the last 9 years.
Here are a few quick quotes from Victor that summarize his stance.
“Notice how you know where you are in the book by the distribution of weight in each hand, and the thickness of the page stacks between your fingers. Turn a page, and notice how you would know if you grabbed two pages together, by how they would slip apart when you rub them against each other.”
Later on, Victor compares this to our touchscreen-based technologies of today.
“Now, take out your favorite Magical And Revolutionary Technology Device. Use it for a bit.
What did you feel? Did it feel glassy? Did it have no connection whatsoever with the task you were performing?”
Screen-based technology isn’t pure evil.
It has so much room for improvement, both in terms of accessibility and efficiency, but attacking the use of phones and iPads isn’t entirely productive.
On one hand, Victor is right about some things. When he comments on how he’s actually tried a lot of the very high tech simulations of potential human-computer interactions and the animations making pretty mockups of what the future looks like, he’s quite right.
However, as he argues against what he calls “Pictures Under Glass” technology, I can’t help but feel like he’s dramatizing the situation. It’s true that even advanced haptic feedback from a phone doesn’t match the type of feedback and sensation you get from grabbing a book or lifting up a glass of water.
It’s interesting to think about all the different ways that we grip things and use our hands with ordinary objects, but it’s very hard to replicate those things with small, portable devices. Think about the linear, tactile, and audible click that comes with using a keyboard. Larger, stationary devices offer more options for physical feedback than a cell phone does.
We can create more advanced forms of interaction, but that raises other questions and concerns.
I’m very interested in the idea of interactions responding to more than just hands. It would be interesting if the technology that tracks eye movements or the movements of other limbs became more common. There’s a lot of potential and it’s worth thinking about how we could really revolutionize the ways we interact with technology rather than just swiping on a screen, but right now, screens are primarily what we have to work with.
Victor does shed light on an issue, but he doesn’t provide any solutions. That is essentially the typical purpose of a rant though, so who are we to say that he didn’t succeed at what he set out to do?
While reading through his response to a lot of the common comments he got on his original rant, it did feel like he was back peddling a little bit on a lot of the things he initially was very assertive about. He’s calling for different types of interaction, which is valid and needed in the world, but he also expresses concerns about things like brain interfaces. I almost chuckled at that part of his response. We’re now in a time where Elon Musk’s Neuralink technology is actually raising a lot of moral and ethical questions about this exact type of technology.
I’m excited and curious to see how interaction design evolves because it is a complicated and nuanced field of study. It’s vital that we think about accessibility and that we strive to design systems that are easy to use, functional, and efficient.
Things are changing rapidly and the need to think about interaction design. While we might be skipping over things companies once theorized would become mainstream, like smart glasses, there are lots of other new technologies emerging every year. But if we want to use those tools efficiently, we need to come up with interaction designs that work.