Being There to Experience the Beginning of a Subjective Computing Era
When I think about literature, cinema, video games, or any form of art involving the representation of a world, I usually refrain from describing my experience of work as being immersive. Over time, I’ve come to find that the concept of immersion is often used so vaguely that it actually loses its meaningfulness. More specifically, in interactive media and entertainment contexts, it’s often related to how efficiently an immersive work succeeds at imposing its own world by bombarding the senses, rather than making the experience singularly and subjectively engaging. With that said, if we take a slightly oblique view, the concept of immersion might lead to a lot of interesting ideas. For instance, the immersive computing field allows us to experiment and drives us to question how computers can process subjectivity.
From that perspective, by simply observing the advancements in current AI and immersive computing research fields, to name a few, the paradigm appears to be shifting from a more technical one to an existential one. As such, we’re left to question what it means to perceive the world through a machine, and in turn, how that same world is perceived by a machine.
We also speak of computing as being immersive, as if it weren’t already. In reality, computing is immersive by design. I don’t say this because I believe it can be used to create “virtual” worlds, but rather because of its systemic nature. One of its primary purposes and functions has always been to integrate into our living spaces in such a sophisticated and seamless way; naturally fading into the fabric of everyday life and becoming as transparent as a window’s glass, reflecting and mirroring its surroundings.
In the same way, we speak of virtual reality as if reality wasn’t already virtual. But we know that the notion of virtual concerns potential, as in an object or being’s potentiality to become or to evolve into something different, more specifically, the opposite of its actual state; the physicality of that same thing. Gilles Deleuze suggest that “the tree is given in the seed, but as a function of a plan(e) that is not given”¹ as if the tree was actually absent, but virtually present in the seed.
A Subject Oriented Paradigm for Computational Spaces
D. Fox Harell describes subjective computing as “an approach to designing and understanding computational systems that serve improvisational, cultural, and critical aims typically exhibited in the arts.”² However, as the notion of immersion is my starting point, I adopt a different approach by exploring subjectivity as a way to express the ambiguous and singular nature of human inhabited and uninhabited computational spaces³.
If immersive computing revolves around the notions of space, sensorial experience, embodiment, and physical interactions, I’m tempted to say that we might be progressively moving into a multidimensional subject oriented approach based on a subject, percept and context relationship model where everything revolves around the idea that the subject perception of a thing is also a means of interaction. In other words, the environment alters perception, and perception alters environment.
In my opinion, such a model is a way to understand human-computer interaction in a more subjective manner. To me, being immersed is being subjectively engaged in such a way that I’m no longer simply consuming someone else’s content, but also altering it by my presence.
Since the experience becomes all the more subjective, it naturally becomes less predictable. Although we’re not quite there yet, computational environments will involve ever-changing spaces that reflects the singularity of one’s ambiguous human essence.
As for now, these spaces break from the traditional touchscreen and filtered interfaces we’ve grown accustomed to using in our day-to-day. Relying on haptics, artificial intelligence, voice control, computer vision, and more, these immersive realms pretend to enhance our interactions in the most natural and seamless of ways.
What’s the point to all of this? As these systems respond to natural human speech, physical movement, psychological behavior, and context, they adapt to our environments to become inhabitable spaces and spaces that inhabit us.
Paradoxically, as with any system, the more seamlessly integrated they are, the less control we have over them. As such, an invisible technology is not necessary a transparent one. Generally speaking, such technologies tend to be rather opaque, closed, and invasive.
Now, by simply observing how computing devices are literally getting closer to the mind (desktop => handheld => headset => ???), it’s important that we question the level of invasiveness involved in these technologies and their effect on us. Not only am I referring to issues related to a person’s intimate space, but also those regarding how computing systems can actually filter perception, thus alter subjectivity.
Reality is Conflictual
When it comes to creating immersive environments, the design and development tasks are approached quite differently. More specifically, the linearity and predictability offered by traditional interface design doesn’t appear to be relevant any more. Designers no longer rely on the abstract metaphorical elements they’ve been using as shortcuts all these years. In other words, those abstract real-world concepts we’ve extracted meaning from and represented visually via calendar icons or color-coded categorization labels for example, are no longer reflective or adapted to cater to the needs of users in such a context. Immersive spaces require tangible, real-world interactions that evoke meaningful and authentic emotional responses. As a result, those working in the virtual or extended reality field must become immersively engaged as they manipulate the environments they work in.
However, this sensorial and particularly physical approach to design for immersive spaces goes against the Seiryoku Zenyo maxim, or as Bruce Lee once said, “maximum effect from minimum effort.”
Let’s put it this way: we’ve always interacted with computers using languages, as we do amongst ourselves in “real” life, with words, pictures, numbers, and metaphors. As our brains are “hardwired for language”⁴ metaphors aren’t just “shortcuts” that you can let go of because you think they’re outdated. Metaphors are also a very powerful and efficient way to communicate an idea.
As I type this text, every key I hit has the potential to trigger or evoke a powerful emotional response. A single finger tap can prompt any task that a computer can perform. However, there’s no direct relationship between the key I’ve hit and the computer calculations being performed in the background. When it comes to immersive design, we have to come up with new ways to build an imaginary continuity between the virtual (simulated) and the actual (physical) world to bring relatable meaning to a person in an immersive context. Usually, this is done by allowing the experiencer to perform whole-body actions to interact with the computational space. However, this comes at the cost of consuming one’s physical energy, which wouldn’t typically happen in the traditional human-computer interaction approach.
As we know, immersive computing technologies, in a way, allow us to physically experience an “hallucination” by tricking the mind into thinking that it’s experiencing an “actual reality”. However, beyond questioning the believability of the illusion, what this really means is that everyday reality is a conflictual experience in which mind and matter collide with one another. From that perspective, subjective computing is a field that allows us to research and experiment ancient and visceral questions, such as what it means to live with a mind contained inside a body.⁵ A question that is also explored through those cutting edge AI-driven robots trying to compute what reality “feels” like for the first time as the complex piece of software is simply trying to inhabit its “new flesh”.
Affordances
To deal with the human-computer interactions experienced with immersive computing systems, designers and developers have to keep affordances top of mind.
In the design field, the term is often used to describe “a situation where an object’s sensory characteristics intuitively imply its functionality and use”⁶. In a virtual or extended reality context, sensory displays are used to highlight affordances, which ultimately serve to indicate actionable interactions and clarify when these interactions should occur.
If affordances can be used for functionality purposes, the affordance concept, in its original form, becomes less relevant to usability. James J. Gibson, the psychologist who coined the term in 1966, explains:
“The affordances of the environment are what it offers the animal, what it provides or furnishes, either for good or ill. The verb to afford is found in the dictionary, the noun affordance is not. I have made it up. I mean by it something that refers to both the environment and the animal in a way that no existing term does. It implies the complementarity of the animal and the environment.” ⁷
In other words, affordances describe the potential interactions between a subject and its environment according to both its needs and intentions. However needs, intentions, and the environment are always changing. Therefore, if subject and context continuously transform one another, the relationship between both can be unpredictable and ambiguous. As a result, all potential interactions can’t be predicted, and can vary from one person to another.
With the latter in mind, as we progressively move from a technical mindset to an existential one, we might also rethink our use of the common “user” concept in the computing field. Let’s face it, we aren’t just “using” computers; we live with them and experience through them. We’re immersed in a vast interconnected technological network. And if that’s the case, human-computer interactions should no longer be about a designer’s or an engineer’s ability to hack one’s mind to force a perspective or view. Now, it’s more about finding ethical, non-invasive approaches to viewing computer software as a computational space that can be subjectively, responsibly inhabited and built, rather than existing as a simple media consumption tool.
Understanding Meaning
Although, subjective computing might be about us experiencing reality through computers, it’s also about computers adapting to the way every single one of us understands and experiences our environment in our day to day, which comes with significant risks. It might also be about discovering how valuable the intimate experience of space really is and what it means to build it, inhabit it, perceive it, compute it, as well as invade it. At the end of the day, this might also open up to some deeper philosophical and ethical questions in regards to the mind as a computational space itself…
Nous sommes une entreprise fièrement québécoise. Pour lire cet article en français, c’est par ici!
_____________
- Deleuze, Gilles, Guattari, Félix. (2004). A Thousand Plateaus, Capitalism and Schizophrenia, A&C Black, p.293.
- Harrell, D.(2016–09–29). Subjective Computing and Improvisation. In The Oxford Handbook of Critical Improvisation Studies, Volume 2. : Oxford University Press. Retrieved 21 Jan. 2018, from http://www.oxfordhandbooks.com/view/10.1093/oxfordhb/9780199892921.001.0001/oxfordhb-9780199892921-e-003
- A computational space being an imaginary environment or an actual place built, inhabited and/or perceived through computers.
- Sassone Kara ( 2014–04–17). Our Brains are Hardwired for Language. Retrieved 20 Dec. 2017, from http://www.northeastern.edu/cos/2014/04/iris-berent/
- A very interesting project exploring this subject is The Machine to Be Another by BeAnotherLab: http://www.themachinetobeanother.org/
- Retrieved 21 Jan. 2018, from http://www.usabilityfirst.com/glossary/affordance/
- Gibson, J. J. (1979). The Ecological Approach to Visual Perception. Houghton Mifflin Harcourt (HMH), Boston.
