Sensual Computing
Mark Pesce
373

Hi Mark, these are really interesting ideas, and I like the way you’ve outlined them . I feel that my personal philosophies on VR/AR user interface, interaction, and experience align nicely with what you’re describing. I may be in a position to implement/explore some these ideas, and am interested in continuing the discussion.

sensual computing defines a methodology to ‘humanise’ data, making data ‘subject’ to the human. […] delivering [data sets] to the individual in a way that allows the individual to explore, correlate and develop

Perhaps we could step into a concrete example here. I’ve been working on a force-directed graph VR demo recently, and have mostly focused on the “visualization” and “interaction” aspects that you’ve described. I’m curious about your perspective on this, and the paths you might take.

For my graph project, I envision the ability to “plot” data axes in a variety of non-standard ways. Any particular axis could define how a graph node reacts to proximity (hands, body, head, other nodes), to its angle away from your gaze, to being tapped or struck, to being pulled away, and so on. These reactions could be visual, physics-based, haptic, audible — there’s really no limit to the effects and feedback that could be applied.

It seems to me that these new dimensions of data “sensualization” (as it’s not just “visualization” anymore… “data-senz”) and interactivity — combined with being immersed within the data — opens up entirely new ways of discovering patterns/relationships/etc. within the data. You can feel it, experience it. However, I haven’t thought too deeply yet about how to bring in the “decision making” and “data selection” aspects... I likely need to first connect the tool to real data, with a real problem in mind.

working to craft a solution that uniquely meets their needs

Certainly, there would be bespoke design/planning required to translate an organization’s particular goals/data into something sense-based and experiential. I wonder, however, do you see room for general-purpose tools here? A visualization/interaction tool, for example, where the customer-specific work primarily involves “mapping” between data values, data axes, visualization styles, sensory feedback types, desired outcomes, and so on.