UI Framework for HUUUUGE monitors?
Before the article begins I have to dedicate this post to Dr. Ginger Campbell. I discovered Dr. Campbell’s “Brain Science Podcast” on a whim several years back and have been obsessed with Neuroscience ever since. There are well over 100 podcasts that review and recommend incredible resources. Please check out the “Brain Science Podcast”. Listen to it on your way to work or as you walk your dog and you will be amazed about how much power you gain by knowing yourself better in the context of Neuroscience.
From Small to Huuuuge!
I have been developing a new Content Management System (CMS, yes another one), that from the ground up is focused on encouraging the creation & efficient recall of connections between content.
I felt obligated to explore backend technologies like Neo4J and Kenny Bastani’s Mazerunner for Neo4J, as well as front-end technologies like CSS3 transforms, WebGL, and Mike Bostock’s D3. It has been a priority to leverage backend technologies that are tuned to traverse connected data as well as UI technologies that take advantage of our brains’ abilities to process space and motion.
I have learned to be patient and not generate a product based on existing technologies but to build an idea based on research and seek or build the technology required for the vision.
Developing the vision of this CMS consisted of constantly bombarding the following areas of study together while developing this CMS:
- Information Theory
- Chaos Theory
- Graph/Network Theory
- UX Development
What do you get after you mix these up with a pinch of caffeine?
In my case, two things:
- A feeling. This feeling is that there is some pretty low level commonalities that seem to have some interesting implications to UX development and education. Do you share this feeling? Let’s talk ;-)
- A slew of new personal UX perspectives, some of which already exist and that I am discovering for myself for the first time and a few others that do not seem to be documented or explored in depth.
I hope to share a lot more about all these areas as I catch up with the current research but for now, I wanted to share some thoughts on one area of exploration that keeps tapping on my shoulder, UX design for ultra-wide screen real-estate.
The Tree or the Forest? Why not Both?
What led to an interest in exploring UI patterns for large/wide interfaces?
It started by discovering a few common themes I come across while learning about Neuroscience, Information Theory, Graph/Network Theory, and Chaos Theory. The concepts of self organization, self similarity, and emergent behavior are pervasive in these fields and the interplay between the concepts and the fore-mentioned fields of study are a lot less coincidental than I could ever imagine (more on this in a future post).
Emergent Behavior & Self-Similarity
In Neuroscience, reductionism only takes you so far. The complex phenomenon of consciousness requires acknowledgment of emergent behaviors within a complex system.
The architecture of the brain allows for self-organization of self-similar module patterns across all scales in the brain which gives way to higher levels of self-organization and emergent behaviors in the form of cognitive processes and consciousness.
Because of this complex emergent behavior, we can consciously organize relationships between established ontologies. Very often we leverage a common and most helpful organizational structure: Hierarchies.
Hierarchies are very common in the way we organize information. Here are just a few examples:
- Books : book > chapters > sections > contents
- Mind Maps : root node > children > children … > leaves
- Curriculum : area of study > grades > sections > standards
Perhaps, you recall making outlines while taking notes in high school. You might also recall your instructors in college showing slides organized in groups. What does this do for us? It gives us the ability to modify and leverage Context and Focus.
Context and Focus
Context and focus is best described as a dynamic state of perception. In this dynamic state, you are free to modify your context and/or focus. When you are actively aware of your context and your focus there is more information available for you to make new connections.
The ability to zoom in and out of hierarchically organized information is an incredible ability. It is no coincidence that our evolved methods of organizing of information compliment our brains self-similarity in structure and emergent self-organization. It seems that applying Conway’s Law here is one way to think about how the communication structure within the brain ultimately determines possible methods of information creation and organization.
The ability to switch context and focus allows us to understand the micro and the macro. It allows us to encapsulate, abstract, and relate across different scales. Whether teaching or learning, its an incredible tool for integrating new information or recalling encoded information from memory.
It is my experience that encouraging focus while maintaining context helps flush out new relationships and deeper understandings of relationships within and across scales. Here are a few examples:
Context and Focus while studying
While diving into literature or developing software for my side project, I end up creating a physical space where I can focus on different items at different contexts.
I can write on my whiteboard to catalogue high level goals and structures. I can keep smaller more transient sketches on paper. I can reference well established information from books on my desk or on my monitor.
Changing focus and context has been key in organizing information efficiently across many disciplines all the while trying to integrate new ideas into software. This physical environment gives me flexibility to focus or broaden my attention and to relate details to big picture ideas.
Context and Focus in Data Visualization
- Changing the Type of Scale / Context
- Switching Scales (circle packing aka “Zooming”)
- Viewing Multiple Scales (tree)
- Viewing Multiple Scales ( aka “Overview and Detail”)
The image above shows my productivity setup and it seems to be a common setup for students in general. Students can have multiple notebooks/books, a computer, and possibly a white board to help them plan, organize and make new connections. I am of the opinion that this helps promote focus and context and that focus and context is a helpful tool that compliments our brain’s natural structures and functions. Therefore, shouldn’t I encourage focus and context while developing my new CMS? How would this be achieved?
Many web applications that I work on professionally follow common patters in mobile-first development because they come with requirements that demand it. Therefore, the UX and the functionality must be optimized across large/medium to small/x-small devices (to leverage Twitter Bootstrap’s terminology).
On a small device, context and focus becomes more challenging.
Modern mobile operating systems like iOS do their best (and they do a pretty amazing job) of conserving context by taking advantage of skeuomorphism, animation, and spacial cues.
All of these techniques try to restore context by allowing our brains to re-prime existing patterns common to many people (skeuomorphism), enhancing relationships between objects (animations), and organization of objects on the screen (spacial cues like Google’s Material Design and iOS).
While one gets a pretty good sense of context, ultimately, the cognitive load is higher and context and focus is not as optimal as it could be with such a small sized screen.
Imagine if you could only have one book open at a time while trying to research a topic across multiple books. Likewise, imagine that you could not keep your notes in front of you while reading a text book. If you wanted to take notes, you would have to shut the book, then pull your notes out from behind you. The transition of focus and changing of contexts would seem to be more difficult and an increase of cognitive load seems inevitable (Still looking for research on this topic).
If you are stoked on Hebbian learning, then relating your book to your notes might be better done by encouraging less obstacles for changing context and focus between them.
Apple has acknowledged this important need by allowing two apps at one time with the larger screen real estate on iPads. They also emphasize context and focus through one of OS X’s most popular features, “Mission Control” which allows the user to spatially navigate between multiple applications and “desktops”.
So does that mean bigger is always better for encouraging context and focus in a user interface?
Is a Bigger UI a Better UI?
For promoting context and focus, it seems that indeed a bigger screen that allows continuous access to efficient and meaningful navigation between different foci and context is ideal. I certainly work best with two large monitors while doing web development.
Desiring a larger screen is also task dependent. Not all tasks or applications require context and focus. I certainly don’t need a ton of context when scrolling through twitter, but then again it would be nice to see it’s relationships to other content already in my head…
Considerations for Large UI’s
There are some interesting considerations that could come into play while designing for large monitors.
For instance, the evolution of our vision has pushed higher concentrations of blue-sensitive receptors to our periphery perhaps selected as a useful trait to identify water. How might that effect your color scheme selections?
“The Eureka Factor,” highlights the work of cognitive neuroscientist Christine Chiarello who showed that presenting visuals to the right hemisphere of the brain (by covering your left eye) can increase remote associations. Do we start to put stimuli on the very left edge of a large monitor to activate the right hemisphere to promote creativity?
There is so much more research applicable to large UI’s like leveraging the incredible and well evolved spacial orientation powers of our brains and the intricacies of our visual system.
I am experimenting with these ideas in my new CMS and developing a framework conducive to responsive designs on large to medium monitors. I am still gathering research and will certainly have plenty of anecdotal accounts to share in the future.
I hope to go beyond the simple table/panel based UI systems present in iMovie or Photoshop and provide some new tools to those also interested in developing web applications specifically designed to optimize context and focus. I see these tools particularly relevant for analytics dashboards, content management systems, and learning management systems.
Additionally, I hope to see companies like Apple invest time into large monitors, API’s, and new design patterns to support information rich applications. Perhaps new product categories are on the way? Someone has to supply the Minority Report with interfaces right?
So in retrospect, maybe I am not actually building another CMS. Perhaps, with an emphasis on context and focus, what is emerging is a new type of system that can be more appropriately referred to as a “Content Integration System”.
Perhaps, the future will put more emphasis in the way information is integrated into someone’s personal repository. To site Peter Thiel, it’s possible we will go from “0 to 1” and approach a power law emergence of technology within education. Perhaps, optimizing large displays for context and focus is just one step towards a new level of meta-cognition?
What do you think?
Do you feel different when you can “spread out” and take advantage of context and focus when trying to learn or be productive?
Do you think there is a market for large monitors and UI frameworks built to support context and focus on these large monitors?
Let me know your thoughts and please share any related research or projects you know about!
Follow me: John David Martin
“MUST SEE” Resources
- Campbell : “Brain Science Podcast”
- Cockburn, Karlson, Bederson : A Review of Overview + Detail, Zooming, and Focus+Context Interfaces
- Kounious, Beeman : “The Eureka Factor”
- Gleick : “Chaos : Making a New Science”
- Strogatz : “Sync — How Order Emerges from Chaos in the Universe, Nature, and Daily Life”
- Barabasi : “Linked”
- Sporns : “Networks of the Brain”
- Thiel : “Zero to One”