Ever become frustrated by users who ignore instruction and click around with reckless abandon? These are my favorite test subjects — the ones who engage with wonder and curiosity, like a child with a new toy. Their unexpected behavior offers an instant reality check, shattering assumptions formed around structured personas and ideal workflows.

We rely on patterns to guide design and our users. Yet, one rule has proven true time and time again; predicting user behavior in the wild is impossible.

Warning: the patterns we explore here reach far beyond user interaction. They can be applied to all human decisions. So, prepare for some realm-hopping, while we sit in judgement from our design studios, deciding which outcomes are the result of user error or design flaw.

So how do we extend empathy to those who skip past the carefully curated content we’ve put in front of them? By recognizing that every decision can be mapped back to priorities, principles and perspective. The three P’s, a decision triangle, if you will. The flexible framework for understanding user action in any situation, accommodating for the uncontrollable and range of user capacity and ability.

Priorities — when to act
Ideally, priorities are the things we organize to plan our day. The checklist of tasks we want to accomplish starting with the most important. When using an app or tool, it’s the reason we picked up the device. They prompt action. When priorities are dictated or shaped by external forces, users can become overwhelmed, resentful or disinterested.

Humans require water, food, shelter, and connection. When those core needs are not readily accessible, survival becomes the priority. Understanding the source of a user’s priorities is just as important as knowing their priorities. It’s so easy to assume they come into our experiences with the same priorities every time; that they’re self-motivated, clear-minded and focused on the task at hand.

Principles — how to act
Principles can be a compass or a storm. On one hand, leading to success through honesty, hard-work, and putting good into the world. We can expect acceptable outcomes when weighing the pros and cons of our choices across a collection of good principles, because it’s rare that any decision can satisfy all our ideals.

On the other hand, allowing a single principle to become all consuming, can disengage the governor of our emotions and drive us off course. Take justice, honor, respect, or fairness. All wonderful tenets, but individually, they can be weaponized, justifying rage or recklessness. Ever hear someone shout, “But it’s the principle of the matter!” right before making a terrible choice? Misaligned principles can sabotage our priorities and result in broken hardware, personal conflict, emotional trauma, or even physical harm.

As designers, we should evaluate the principles our users hold. One example: it’s become expected by our users these days, that any investment, time and/or money, will return desired outcomes through the smooth execution of tasks. This has been repeatedly reinforced by the continued improvement of technology. So much so, that they become shocked and appalled when something malfunctions or feels more laborious than expected. They may feel cheated, betrayed, or ignored; grunting phrases like: “This should not be so difficult!” or “I paid for this — it should work!” Hopefully that’s not the case, but in the wild, where anything can happen, make sure we send praises and apologies to our professional and tirelessly patient call center support team when they do.

Design principles or pillars are critical for measuring success and keeping us honest. When compromises must be made, we’ll use those principles to articulate and justify those compromises. Then we fall back to our supporting principles, like providing as much consistency as possible, with predictable feedback, and clear remedies for potential mistakes.

Perspective — understanding of available actions
Perspective is the comprehension of choices. The observed possibilities whittled down by all the principles, priorities, emotions, experience, skill, education, capacity, and anxieties at any given moment.

It’s easy to criticize the quarterback, who makes millions of dollars throwing a football, when they miss an open receiver during a crucial play, especially when the choice was so obvious to those viewing the game from the stadium or TV screen.

During training, priorities for each play are defined. Offensive principles are drilled repeatedly. Yet fear, frustration, and exhaustion can cloud judgement, no matter how clearly content is presented. From the quarterback’s perspective, a split-second decision must be made while dodging pressure, ignoring pain, and constructing a view of the field from quick glances.

This is obviously an extreme example, unless we’re designing experiences for quarterbacks. It does, however, demonstrate that there can be two truths. In the same moment, there was an open receiver from the perspective of those in the stands; and from another perspective, obscured by chaos, helmets and hands, there was not an open receiver.

For us, no matter how hard we try, we can never view an interface we designed with unbiased eyes. And see what the users sees.

Conclusion
In the wild, sometimes survival takes over, principles fade and our perspective narrows. Anxiety can paralyze — or trigger fight or flight. From the safety of our design studios, it’s easy to forget how unpredictable the world can be. Our past experience may offer insight, but it’s not the same as someone’s present struggle. So, beware of the, “all you have to do” mindset. Or, wishing that they would just read the prompts.

empathy (noun)
The ability to understand and share the feelings of another.
Oxford Languages

Even if we can’t truly feel all that our users experience, we can be more understanding of their decisions by considering the triangle of priorities, principles, and perspectives that led to their choices. Using this framework, we can identify the deltas between expectation and behavior; and develop a foundation for measuring what was once unpredictable. In doing so, we not only become better designers, but wiser and more empathetic humans.

Priorities, Principles, and Perspective: A framework for empathy was originally published in Bootcamp on Medium, where people are continuing the conversation by highlighting and responding to this story.

User Experience (UX) professionals work with end-users to iterate and refine workflows to make accomplishing tasks easy, interaction intuitive, and information as digestible as possible. This is the standard process for creating successful experiences in a competitive marketplace. Those who act with integrity will earn trust and establish loyal customers. Regrettably, we’ve also learned ways to take advantage of common user mistakes and tendencies. Dark UX patterns are used to make money, drive traffic, and capture data quickly at the expense of the user. These patterns are for short term gains and erode trust in similar services and the platforms they run on.

In the political realm, advertisers and politicians commission polls and seat focus groups to gain insight into the trends of public opinion. Think tanks and lobbyists use this information to craft talking points in support of policies and candidates they hope will provide advantages for their causes.

The purpose of this essay is to ask, if politics were a UI, is it working for the mutual benefit of politicians and constituents? I’m also including some observations (opinions) from the last couple decades that have helped me become less anxious and frustrated with politics.

Limited choices

This pattern pressures users to make decisions that are only best for the app or service. Similar patterns are the escape room (hidden navigation), the "dead end", and "roach motel"; screens without a skip or back button. It doesn’t matter if you dislike both apples and oranges, you must choose one to take part.

Beware of Division

George Washington did not belong to a party and was concerned about the influence of regional and political factions in our new country. He dedicated most of his farewell address, published on Sept 19, 1796, to this subject, as a “matter of serious concern.”

"One of the expedients of party to acquire influence, within particular districts, is to misrepresent the opinions and aims of other districts. You cannot shield yourselves too much against the jealousies and heart-burnings, which spring from these misrepresentations; they tend to render alien to each other those, who ought to be bound together by fraternal affection."

"In this sense it is, that your Union ought to be considered as a main prop of your liberty, and that the love of the one ought to endear to you the preservation of the other."

George Washington further cautioned that even though political party associations may be helpful in discourse, they are more likely, "to become potent engines, by which cunning, ambitious, and unprincipled men will be enabled to subvert the power of the people, and to usurp for themselves the reins of government; destroying afterwards the very engines, which have lifted them to unjust dominion."

Unfortunately, no one listened to George Washington and the election of 1796, between Thomas Jefferson and John Adams, was the first of many two-party dominated contests rife with negative and divisive campaigning.

Our best has always come when we are unified. Sadly, the times we were the most united were in the aftermath of catastrophe. In those times we were forced to look past our differences to the collective greater good that George Washington and other great people recommended that we never lose sight of.

Beware of the “us” and “them” trap. Things are simple when we can point to someone else who embodies the cause of all our troubles. But it is just us; we’re all in the same boat. The notion that we can let part of the boat sink and be OK is madness.

Two Parties

Some good advice would tell us that we should never make important decisions without reviewing a pros and cons list. In a two-party system, a political party is only half of that list, it represents the pros or the cons of an issue. That’s all a party can ever be because they are either for or against something.

Our two modern parties are often divided on rural and urban lines, disagreeing about the balance of individual vs. collective freedoms. These are especially important debates because neither individual nor collective freedom can be absolute. Neither can rural areas thrive without urban cities and vice versa; Yet short-sighted individuals with notions of quick gains would have us believe otherwise. If one part fails, we all lose.

The Cost of Party

Imagine what it would take to squeeze the variety of people across our country, all those pineapple, strawberry, melon, and peach lovers, with vastly different life experiences into just two categories; those that prefer apples or oranges.

When we join our voice to a party, it is cancelled out by the voices of the opposing party. This is by design, it silences most of our country to elevate a few. We spend so much time defending our party’s positions and their candidates instead of finding tangible solutions. With all the money these campaigns rake in, do they really need us to defend them? All the while, the politicians in our party pander to the smaller yet louder undecided voices, otherwise known as swing states and swing voters.

Confusion with overwhelming information

Some User Agreements have dark patterns because they primarily serve to protect the app and service sponsors while the language used in them is not accessible to the average user. Confirm shaming and trick questions are similar patterns that can pressure users into making decisions they don’t understand.

In our politics, confusion is generated by a barrage of over simplified solutions to complex problems, half-truths, red herrings, hyperbole, conflating correlation with causation, what-about-isms, and fear mongering; creating an overload of information that is incoherent at best.

Debate

The foundation of politics is debate, an ancient pastime where participants spar using opposing viewpoints. An intellectual exercise, a mental and verbal chess match. In some forms, rules are monitored by judges and the winners declared by jury. The professionalism of debaters is evident by their ability to put personal feelings and bias aside while they argue their positions, as the assigned prompt may not be in line with their personally held beliefs. In competition, debate is scored on factual accuracy, logical consistency, and emotional appeal. It’s both sad and comical typing these out while thinking of our current political dialogue, which offers a super-sized portion of emotional appeal while going exceptionally light on the logical consistency and facts.

Our debate on economics is a good example of one with little logical consistency or facts. There’s real economics and then there’s political economics; the two versions have little in common and the principles of the latter change constantly at the whim of the debater. Yet, the hidden element behind all political debate is real economics.

Debate used to be isolated in the senate chambers to test the soundness of policy, but today it is performed for the masses, across all forms of media as sound bites and take downs.

Negotiation

Another component of politics is negotiation. An art form where two parties dance on edge of consensus until there’s an understanding of priorities. Both play coy with the price they are willing to pay to obtain the best deal.

In our political climate, shame and intimidation are used to shortcut negotiation and stifle debate. We take offense when offered $100 for something we feel is worth $500. This false outrage has always been a tactic to gain leverage in a negotiation or debate.

When we remember that we’re negotiating and debating, which is what politics is, we can part ways civilly. So, when one side says they want something outrageous, don’t get offended and make the mistake of thinking that they do not know what they are doing. That’s what the opening of a negotiation is supposed to be.

Example (made up) political issue and debate

To illustrate how this works, a hypothetical political position on water could be, water is us. Because our bodies are made up of at least 60% water, we have a symbiotic relationship with it, and it should never be regulated or restricted in any way. Otherwise, it’s like our bodies are being restricted and restrained. Anyone who wants to capture water or build systems to divert water is anti-water and anti-body. This is the free water party. Free Water for Free People! Some inspirational imagery might include waterfalls and running rivers.

A counter position, from the water peace party, could be maintaining consistent water levels and creating places where water animals can come together and live. Images of tranquil lakes, ducks, and otters would represent their motto, Peaceful Water, Peaceful World. Anyone that does not want to capture water for wildlife wants loud and destructive water. They would show mud slides and tidal waves as emotionally triggering representations of their opposition.

In the real world, water is important for life of all kinds, within certain volume thresholds. Floods and droughts can cause devastation. So, the issue is not whether we should have a water policy or not, though the farm and hydro power lobbies would prefer that debate, it is figuring out what our water policy should be. How much water should be captured for times of drought, how much infrastructure should be built to protect against flooding, how do we balance the needs of farms, recreation, natural habitats, and energy production? How do these priorities change in various locations as seasons and weather patterns change? All of this is extremely complicated, there are no quick fixes, every option is expensive, so to prepare people to vote for one side or the other, we’ll just boil it down to who loves or hates ducks.

Bait and switch

This pattern is used to capture unintended clicks from users who aren’t paying attention. Similar patterns are crowded controls, sneak in the basket, and preselected options.

The Reality Show

Politics is the original reality show. From their press offices, they highlight their best side in a scripted narrative of curated drama.

To play along, our trusted news sources have replaced answers with anxiety in the never-ending coverage of the reality show instead of reality. They fill airtime with disingenuous debate, that seeks the extremes instead of the agreeable. Even legitimate news programs will pull from the fringes for the entertainment value over substance. If the intension is to provide their audience with equal exposure to both sides, it’s failing, because the result entrenches their viewers even deeper into their existing viewpoints. When both extremists are clearly absurd, we tend to ignore the side that tracks closest to our perspective, because we know people who are not crazy, that believe and live the way we do. But we assume the person representing the other opinion must be a reliable sample of everyone on that side. This is how bias is reinforced, by making comparisons of unequally populated datasets sourced from intentional or unintentional selective associations.

News vs. Media

Newspapers clearly mark each section so that we can get the information we want quickly, and we do not confuse opinion with the reporting of current events. In the opinion section, there’s no requirement for facts, often no fact checking, it’s just an opinion of the author.

Cable TV news channels have opinion and editorial programs as well as shows dedicated to news. But because actual news requires time to produce, the allocation of editorial to news is drastically different. Often 80% editorial, or non-fact-based content, to 20% (these figures may be too kind) actual news. So, when a cable “news” network gets blasted for being biased, that is because most of their programming is biased, and not actual news. This is still considered bait and switch because of the assumption that It’s all news instead of mostly people talking about the news.

No News is perfect

Some may say, “I only read the news,” assuming this is enough to avoid the sensationalism of the TV variety. But the printed and digital news is not without dark patterns.

One misconception is that newspapers cannot keep up with our fast-paced modern world. They do quite well because most events play out slowly. What they are not great at is keeping up with our curiosity, which has been trained by a constant flow of 24-hour-a-day headlines. Most articles are filler side stories and speculation to bide time for further details to become known. So-called experts and pundits are called upon to read the tea leaves and tell us what the future outcome will be. Social Media discussions with “sick burns” become news. Headlines that contain words like “will”, “would”, “could”, and “should”, pose hypotheses instead of providing answers which leave us with even more questions, sucking us into a never-fulfilling loop to fill the void. I have been stuck in this loop, all my free time spent checking for the news that would prove I was right and everyone else was wrong. That story never broke, it was like being addicted to anxiety. The healthiest serving of news per day may be the amount a newspaper can provide, then we can carry on with our day with less stress.

Conclusion

When I started thinking about this subject a couple of years ago, I was under the assumption that we lived at a unique time in political discourse. That modern technology and misinformation made us more divided than an idealized past of honorable people acting for our nation’s mutual benefit over self-interest. That there was consensus for those, obvious to us now, history altering decisions. The third estate, known for journalistic integrity, sought to hold power to account with rigger and without bias. But this is not the case, we have always been easily divided. Part of the American ideal is the notion of good vs evil, black and white, and the correct side (our side) will always triumph overing the wrong side. There are some fascinating studies about how our brains process information and lead us to choosing sides; it is part of who we are. We love a good rivalry.

Our legacy is one of fierce debates. Agreement taking time, too long in some cases, after many concessions and lots of frustration. Threats of civil war were used as negotiation leverage during George Washington’s presidency, about 70 years before the actual Civil War. Misinformation has been rampant throughout our history, even from recognized news outlets. Though the scale and speed of that information has increased exponentially, the cause of misinformation is the same. A rush to get news and information out as quickly as possible, not waiting for all the facts, and neglecting to verify sources. Excluding those who intentionally push misinformation of course, this group ranges from adversaries wanting to harm our national security, to those who want to go viral for ad revenue, to jokes and pranksters.

All About Economics

Politics in the U.S. became a $14–20 billion industry during the 2020 presidential election, with about 70% of the revenue going to advertising. Politicians want our attention for votes and donations, advertisers need our eyes and ears to justify the cost of ad space. They are competing with entertainment, sports, social media, exercise, work obligations, and family time. To make us take notice, this once transient industry uses the information they have gathered about us to exploit our tendencies into seeing the world through a set of sensationalized dire circumstances. It’s difficult to decipher who is crying wolf and what are actual threats when everything is presented with the same energy and at the same rate. This is a well-funded effort to put us in one of two boxes. A positive aspect, the 2020 election exceeded the 60% participation mark the potential voting population, a level not seen since the 60's.

Common Ground

Outside of the sensational, at the ground level, in face-to-face conversations, we do find consensus on most issues that affect us personally. It’s when the conversation becomes hypothetical that we become defensive and barricade ourselves in with our tribe.

Wouldn’t it be nice to vote on resume instead of rhetoric? There’s no other profession that we hire people this way. Yet, this is our system. Our division may be a safety mechanism that slows down legislative progress, which challenges the belief that we voters want change. Every election is about some form of change, in theory, but what we really want is normal, a balancing of the boat, whatever that means. Anytime there is change, we vote the other party into power as the pendulum swings. Not us per say, the swing voters do.

References

• George Washington Farewell Address 1796
• Political Industry Size in Money
• Political Advertising Distributions
• Voter Turnout 2020 Election
• News Sources and political leanings — TV
• News Sources and political leanings — Radio/Podcasts
• Regulated TV and radio sources and equal time
• Legislative Process
• Regulations Process
• The Federal Reserve

Note: As great and progressive as George Washington was in many areas of his life and influence, there was at least one glaring hypocrisy in his crusade for liberty. He was a slave owner, supported legislation on slavery, and ordered his own escaped slaves to be hunted down. If there was a more authoritative reference for this article, both in contributions to the forming of our nation and the warnings about political division, I would replace it.

This is not a doom and gloom discussion or a prediction of Skynet and evil robots. I’ll mainly focus on the user interface aspects of AI, in what’s intended to be more of a brain storm than a dissertation. Though I’m an experienced designer, I’m novice in the AI space so there are many people much smarter than I discussing these types of things; but since AI and machine learning are new and may not feel that accessible to designers yet, I’d like to share a few observations so far.

With that said, with great data comes great responsibility to be transparent, secure, and helpful.

Bias and Self-fulfilled Prophecies

I’m glad we’re looking at the privacy, bias, and other ethical dangers when we consider how or if AI should be integrated into certain technologies. But, with the potential of AI to become intertwined into every aspect of our lives, we need to go much deeper.

A year or more into learning this technology has probably uncovered more questions than answers. I’m still enthusiastic, though it’s sobering to understand how difficult machine learning really is, how iterative it can be and how important it is to break down tasks into their simplest actions for training. Bias, in the context of a person is tragic, unfortunately it’s so easy for bias to be reinforced into a machine learning model. In the case of image classification, where images are categorized based on identified details, the process of training starts with a supervised learning algorithm and a target classification. At first the algorithm thinks that nothing fits into a classification, but as it begins to pick out some characteristics it can be very strict about what the grouping criteria are, excluding everything it has not experienced yet. Each new image introduces a new set of details to include or exclude from the classification while the system is also taught to ignore unrelated details like background elements or lens artifacts. That’s why It’s very important to gather a lot of input material from a wide variety sources, which has its own set of challenges.

Reinforcement learning, which is based on a pattern of providing rewards when random actions produce desired results, can also produce bias. The process is incredibly fascinating, especially when the AI does something unexpected to achieve those preferred outcomes. Sometimes the system will exploit rules that we never thought needed to be defined or we impose unnecessary rules because we assume the outcome can only be achieved the way a human would do it. In the case of a recommendation system where the machine is rewarded when a user chooses one of the recommendations, we must ask, “was this reward based on the recommendation being most correct or was the user just ordering off the menu?” Another option is that the recommendation was wrong, but the user became distracted by the interesting suggestion and away from their originally intended task. Think about going to a restaurant, one chosen with a particular meal in mind, but after sitting at the table and reviewing the menu you have three options: leave if the menu doesn’t contain the desired meal, try to order something that’s not listed on the menu, or adapt and find a second-best selection from the menu. If we adapt because the other two options feel like too much work, then the AI is rewarded for a self-fulfilled prophecy. In reality the user chose from what they were provided even though they may have preferred a different set of choices all together.

Training Models or Training Users

How would the experience with a recommendation system, like the restaurant example, change our behavior and the decisions we make in the future? One of the promises of AI is the ability to make sense of tons of complex data. The reality is training computers to think for themselves is really difficult and when those computers are introduced to the unpredictability of people, things becomes exponentially more challenging. Complicated by users compensating for poorly trained AI or mistakenly providing rewards that reinforce bad behavior.

We can understand that some restaurants specialize in specific cuisines, some recommendation engines specialize in specific contexts, or those intents that are most apparent from our past choices. When viewing a set of recommendations about potential movies to watch, a service demonstrates knowledge that a user normally prefers martial arts movies but what about when a movie is selected based on intentions other than personal preference? Situations like family time, a date or entertaining guests could be the context where a user would most require assistance picking the right movie, unfortunately the system may look at those instances as anomalies in the data and discard them or include them as a new preference.

Contextual understanding is an area where AI will continue to improve, I look forward to the day when an AI has the ability to capture user intent equally across devices and situations. To do so effectively, the AI will have to know a lot more about each individual. In the interest of transparency and expectation setting, users would benefit from a system that exposes the level of contextual understanding the service can achieve. For instance, would I be comfortable with an AI that can tell the difference between personal, work and family time? Other users may not mind providing data for the promised benefits, but others might, so it would be interesting to consider controls that can adjust the level of contextual awareness an AI can detect or instruct the AI to ignore certain contexts that are not relevant for an acceptable level of assistance, like everything that is not work related.

Some deep learning systems use a Human in the Middle approach which provides a way for people to have a practical understanding of how well an AI model is being trained. Controls are provided to help shape the neural network’s perception of the training data to improve accuracy and reduce harmful mistakes. As we improve training processes to reduce bias, learn to present comprehension in a user friendly way, and create the standards that ensure safe outcomes when testing AI with actual users, more accessible machine learning tools will be abundant for the next generation of designers and creators.

Along these same lines, services must provide better tools for capturing user feedback. If we, the users, are the guinea pigs, especially for something that could have damaging effects, we should have a more meaningful contribution to how an AI is trained and implemented. There must checks and balances in place from the beginning, with extreme transparency. AI has the power to super charge our current technology which reminds me of the quote from Dr. Abraham Erskine, a fictional character in Captain America: the First Avenger, “The serum amplifies everything that is inside. So, good becomes great. Bad becomes worse.” Think of AI as the serum used to create Captain America, where every previously unnoticed imperfection of a technology is now magnified. In some cases, the results from AI will be great, but we need to also consider and be prepared for the worse.

It’s a good idea to calm our excitement for AI and to take a measured approach. We can’t even imagine what the impacts of AI enhanced services will have on us long term, the more we can test and discover edge-cases and potential bias early, the better. Unfortunately, there’s just no way to truly know everything until it’s working in a real environment. One way to counter some of those negative effects is to give our user’s more controls and properly set expectations.

To challenge my initial statement, if technology is so great, why does it feel so annoying, like a great betrayal has occurred when we have to do something the old manual way because a technical error has occurred? We need to ask ourselves, “what’s the technical and human toll of success?” Could a successful product actually condition us to become over reliant on this technology? We have become dependent on many technologies already, now imagine living without them, think about your last power outage. Many of us have given up land-lines and are completely reliant on cell towers and battery capacity to communicate. This new dependence has worked out so far, though there are still benefits of corded devices with consistent connection, so we must challenge ourselves to really dive into these tradeoffs. With more and more cyber-attacks occurring, we must also consider the impacts of a compromised AI, beyond the errors or bias we accidentally build into them.

What does Success Look Like?

Remember when I said this is more of a brainstorm? Feel free to hack, modify or even disregard these thoughts as silly to create something better.

Thinking about a set of principles that could be used to justify the use of AI and measure success, a Ten Commandments of AI if you will, that consumers of our technology can use to anticipate how helpful an AI should be, as well as what they are giving up and any potential side effects.

Thou shalt:

1. Acknowledge when personal data is being collected, what was collected, how it was collected, and how it will be used.
2. Acknowledge when an AI is communicating with a human, or in any situation where an AI could be confused with a human.
3. Acknowledge the level of human contextual awareness and potential bias within an AI system. An AI system should not make assumptions based on perceived race or gender.
4. Always build for inclusion, where possible, solving for permanent and temporary disabilities in the areas of communication, vision, hearing, mobility and cognition.
5. Provide a clear list of values an AI is intended provide, as well as all potential risks from using the technology, especially after prolonged use. These risks should cover hardware, software, battery, security, privacy, physical, social and psychological impacts.
6. Display compliance scores from periodic tests run against a set of AI best practices, including bias tendencies, system health, and report any instances of miss-use.
7. Incorporate human in the middle and user in the middle feedback systems, that allow meaningful contributions within context of an issue or success. Build in systems to report bad behavior and that can potentially take down a system that’s causing harm as quickly as possible.
8. Provide clear safeguards and procedures for dealing with system failures, power outages, and/or user errors. Make it easy to reverse destructive behaviors and errors where possible.
9. Build in fallback systems that allow the AI portion of the technology to be disabled or bypassed without bringing down the whole system, so basic tasks can still be performed.
10. Take all steps, processes and policies necessary to maintain data security and protect the user from cyber threats.

Ten may not be enough to capture everything, share your thoughts.

Traditionally we’ve built experiences that are seamless and magical, where the user doesn’t need to know how they work or what happens under the hood. Some of this is for security reasons or to increase user productivity, but as designers we must design for a more secure transparency and ways for the user to engage a technology on their terms.

Below is a list of some favorite AI and ML resources for those that want to learn more.

Design Guidelines

• aimeets.design
• Guidelines overview — Microsoft HAX Toolkit

Bot Framework Composer

• Create a Chat Bot

Creating Bots requires detecting and confirming user intent using a conversation. These conversations occur as text chats, but could be vocal and/or visual. Think about designing traditional UI in a more conversational way.

Classification Engines

These are great tools to build apps that respond to object, audio and pose detection.

• Lobe
• Teachable Machine

Fun Art and creation experiments

• AutoDraw
• Quick-Draw
• Cartoonify
• Semantris
• Handwriting
• SoundMaker
• BillTJonesAI

Computers can create art, music and poetry, with a community of creators

Recommendation engines

• Peronsalizer

ML and Simulation tools

For a deeper dive into the world of AI and machine learning using Unity and reinforcement learning to build custom ML solutions

• ML-Agents

We hear the word intuitive a lot in our industry, from our clients, “the UI should be intuitive;” or perhaps it shows up as a pillar in the creative brief, “the target audience will find the UI intuitive.”

Merriam-Webster’s dictionary consistently refers to its root word, intuition, when defining intuitive; with only one variation specifically related to software, “an intuitive interface.” This outlier which is clearly our client’s intended meaning continues with, “readily learned or understood,” but seems to contradict the broader definition of Intuition which is more like, “quick and ready insight” or my favorite, “the power or faculty of attaining to direct knowledge or cognition without evident rational thought and inference.” Or, in other words, by some gut instinct or mystical power.

It would be easy to accuse me of getting caught up in the terminology and ignoring the essence of what is meant by the word intuitive in the context of UX design, but that would be like focusing on just the creative and ignoring the scientific side of user experience. Creatively, we devise solutions that are not readily apparent, which is where many of us live on the design side; but to satisfy the scientific, where a design does not become a solution until it’s been tested and validated, we must have something tangible to evaluate. So if intuitiveness is the benchmark, what exactly are we measuring and how do we measure it?

Patterns of the Past

The word intuitive implies that we could quantify the user’s abilities, maybe their clairvoyance or aptitude for deciphering what is presented to them. I must admit, there were times that I’ve blamed the user when they were unable to understand an arguably vague design that made perfect sense to me. To evaluate the design, we are actually analyzing its familiarity to the user or the legibility of previously established patterns and affordances. Many of the conventional patterns we use today have direct links to the physical world, but their affordances were refined until they made sense digitally. However, in the emerging technology space, traditional UI frameworks are often not available or do not fully accommodate new interaction scenarios.

It’s an exciting time for technology and it looks like the pace of innovation is only accelerating, which will continually change the way our users interact with the digital world. We’re already starting to see where some of these newer technologies are pushing us into realms that do not have a direct correlation to the physical or even a cognitive connection to an easy to understand phenomenon, much less a parallel to a familiar design language. A simplified example of this is trying to create an icon for a specific feature within a virtual construct, like AI or Holographic, without just rendering existing iconography using dotted lines. Icons are effective for taking tangible patterns into the digital world, but now that we are starting to communicate a virtualization of this digital space, it’s worth evaluating how these patterns will hold up to the layers of abstraction inherent in implementing new technology on modern infrastructure.

Another trend that will force us to reevaluate our standard practices is a new generation of designers joining our industry who are less emotionally connected to what we consider hard established rules. Since each generation will have less experience with the physical world equivalencies upon which many of these standards are based, we are about to reach a time where, for some, a design feels symbolic because of its links to the past and for others, the same design is completely abstract. By welcoming their seemingly controversial ideas, we’ll be forced to question the value of perpetuating what we have come to know as standards to those that do not fully understand their references.

Looking toward the future

Unfortunately, we cannot rely upon mystical forces to guide our users through the innovative experiences we create. The idea that the right layout and combination of design elements could produce a clean and efficient presentation of information that anyone can understand is an intriguing challenge, but it always felt like an all or nothing thing. As soon as the user experiences a little friction we compensate by adding “helper” content which causes the once elegant design to become cluttered. At this point we are not designing to the user anymore, we are designing solutions for our design, and as we iterate in frustration, we must admit that our design has literally failed to meet the definition of intuitive. Though it’s a great word for communicating our intentions, should intuitiveness be the ultimate goal?

The barrier to entry for any user is past experience and their ability to translate interaction patterns based on the affordances presented. It can be said, if it’s new or unfamiliar, than it’s not intuitive, but where does that leave us that are working in new technologies?

The reality is, It’s always been OK to teach the user something new. Unfortunately the typical solutions for user on-boarding have been dreadful. It’s like we’ve given up at this point and a tutorial was put in as a last resort or with very little budget left. Even worse, we hold fast to the minimal design direction and send frustrated users to a help menu where we outline all the workarounds for the beautiful interface.

Sometimes we just get so caught up in getting the user into an experience, that we take for granted the magic of discovering new technology or becoming impressed with something common being used in a new and more powerful way. Intuitiveness of design requires there to be an understandable conversation between user and designer; the emergence of inclusive design is evidence enough that our language, in these conversations, was too shallow and narrowly focused on a specific persona.

A design should start by listing out all the expectations we have for the user. There’s often a mismatch in what we assume our users will bring to the table, based on the persona we expect a level of knowledge, past experience, pattern fluency, or workflow understanding. There was a time when we relied on a specific tool or device to accomplish a task, now we can choose from a plethora of apps. Consider how many ways we can capture, enhance, and share photos on our mobile phones. Each of these applications contain unique and innovative interaction patterns based on their specific workflows. Not only will these patterns challenge tried and true standards of the past, but they also increase our bias of what’s intuitive, which is more reliant upon the experiences we have in common.

The Myth Exposed

Now, having exposed this myth, I hope we can begin the design process focused on informing users on how they can use our technology to accomplish their goals. Not through the traditional tutorial that we force everyone through, but by leveraging our years of experience, acquired skills and collective talents to create streamlined and responsive interfaces that go beyond teaching or reminding our users how to interact. Beyond the interface and the persona. Designing systems to detect the experience level of the user, understand the pace that they want to work, and conform to the different modes that are appropriate given the context of the experience and the types of tasks they wish to complete. Instead of relying on the intuition of our users, we should have a deeper conversation with them. Listen not only to the buttons they press, but things like how fast or often they interact, which can provide clues to intent.

As we move into a new era of UX design and technology, focusing on empowering users to have a common understanding instead of assuming they can decipher our design dialect is the safest strategy. Devise ways to measure a user’s comprehension at each step in the user journey, with the understanding that not everyone will want to follow the same work stream. A flexible system for providing information and allowing the user to customize the experience should be the backbone of the design. Though, we should never attempt to outsmart the user, we can use patterns discovered in our user testing to figure out the proper context to unobtrusively offer assistance; producing a more user-centered design and one that might even appear intuitive.

The myth of intuitive design was originally published in UX Collective on Medium, where people are continuing the conversation by highlighting and responding to this story.

Holographic computing is a powerful thing. The amount of data we can absorb in three dimensions is astounding. Experiencing things in an immersive context speeds up our understanding, helps us make decisions faster, and hard-codes our memories in spatial awareness.

The design implications that stem from such a rich environment are equally astounding. Think about your favorite map app. On a 2D screen, we spend a lot of time panning, zooming, and switching views to understand things like elevation, depth, and proximity. Now imagine that same map in HoloLens 3D. Suddenly, everything that our minds previously had to stitch together — different views and angles arranged like a Lego map — is now instantly understood. It’s alive, it’s complete, it’s right there in front of you. Experiencing something digital in the same way we understand the physical world is pretty magical.

As designers, this real-world understanding is a constant and relatively new challenge. How do we translate our knowledge of pixels and resolutions to the physicality of 3D realms? How can UI designers confidently iterate across different “worlds”? How does design thinking evolve from here?

Get into the space

Designing for an immersive mixed reality platform means imagining and exploring the literal space around the user. Think about the walls, floor, ceiling, windows, and furniture that may exist. Anticipate everything about the user’s environment. Walk a mile in their shoes, as it were. Contextualize.

Now consider how building on the existing elements in the room might enhance the experience and uncover opportunities to create solutions that feel more grounded and natural. From the opposite perspective, consider the negative impacts of existing furniture and such. Uncover ways to provide guidance to the user in setting up their space, to curb potential distractions and disarray.

Keep your design explorations as physical as possible. Use real objects as stand-ins for holograms. You won’t get a sense of the physical space if you don’t create it, stand in it, move around it. Prototyping for holographic applications is a little goofy — embrace it. Get your team together and craft the experience using anything you’ve got handy. Blocks, foam, exercise balls, stuffed bears, each other, whatever. The early, rudimentary prototypes will inform your design direction and interaction model.

The distance challenge: legibility and accuracy

Depth perception is a huge challenge in 3D design. It affects every piece of information and object the user interacts with. The mind perceives things as smaller at a distance, which means details at a distance are more difficult to decipher, especially text. The amount of muscle control required for the eye to focus on objects at a distance is much greater than focusing on elements closer to the user. The slightest movement of the head affects perception. Reading small text from far away requires the muscles in the
neck to hold the head as still as possible. The eyes need more time to focus, so they must remain motionless while blinking intervals are increased. Basically — human eyesight is a cool but complex thing. All these processes are happening automatically, but prolonged reading at a distance can cause fatigue. This is why the humble road sign is so great. It presents the exact right information at various distances, quickly and effectively. Referencing real-world patterns like this creates powerful virtual world solutions.

In line with legibility, virtual spaces require precision. We’re imperfect — we misjudge distances, we move awkwardly, we can’t focus on too much stuff at once. Being precise in your design considerations means overcoming the natural movement of the body, especially at a distance. Imagine a tiny button ten meters in front of you. You walk forward, maybe you start talking to someone next to you, you miss the button. You have to recalibrate for a moment, try again.

As a designer, you can change that interaction. In the virtual space, you’re free to dictate the size of fonts, buttons, and any other bits of information that need to be observed. Create designs based on how far the content will be from the user. Keep in mind that to maximize effectiveness, the bulk of primary content should stay between one and two meters from the user. Elements closer than one meter can cause nausea, especially if they’re connected to the user’s gaze.

The user feels confident when they can easily absorb the information required to make choices. Swiftness and accuracy replace fatigue and frustration. The goal is to be responsive and always provide the right amount of feedback in reply to the user’s input. An experience that has a flexible workflow, makes the user feel confident, understands intent, and responds with the desired result is rewarding and engaging.

Think like a game developer

Which is not to say, “develop a game for HoloLens.” The point here is to design the scenes all around you, and to keep in mind that all that immersion and scenery will undoubtedly affect performance. Game development practices are commonly measured by frame rate, and it’s the same with HoloLens. To produce quality graphics and more immersive experiences, we’re always balancing what we do visually against a GPU budget. For designers, these are the major performance consumers: polygons, object count, texture size, transparency, complex shaders, dynamic lighting and shadows.

As you design the space around the user, certainly don’t shy away from playing with all the design elements available to you. The trick is to balance left brain and right brain, developer and designer. Be both, be conscientious of everything going on in the space. The magic of HoloLens lies in that 360°-ness. That cinematic, fluid experience. What’s behind the user, below the user, in their periphery? Where’s the action? What should they hear and see, and where should they hear and see it? Immerse the user. Be playful. Remember that with the HoloLens we’re no longer restricted to designing within a physical frame. The world is our canvas.

Start building

Thinking about 3D space only scratches the surface. Once you’ve oriented yourself, done the prototyping, the explorations, the iterations; you’ll need a solid toolbox to start building. HoloLens has a ton of guidance for developers and designers alike. Start exploring these resources, and have fun!

Adobe Illustrator Texture Template

As designers, we work with physical objects or on a whiteboard to understand the size and relationship of the UI system. Using this template, it’s easy to plot the UI system out in Illustrator to finalize the design and aesthetic. We can even compare how it fits within the user’s field of view.

All about typography in HoloLens

My colleague Dong Yoon Park wrote this fantastic piece about text in 3D space. There’s also a nice overview of a recent Meetup talk from Yoon on the same topic.

HoloToolkit-Unity

After installing Unity, this collection of scripts and components is the quickest way to get something interactive running on the HoloLens. There are plenty of examples included with the toolkit and a community of active contributors.

On the developer side, here is a list of all the tools you may need to develop applications for HoloLens.

Holographic Academy

If you have some C# scripting abilities, here’s a set of tutorials that cover the fundamentals of holographic experiences and input methods like Gaze, Gesture, and Voice. The Documentation section has more in-depth study of designing for holograms.

Microsoft HoloLens Dev Center

The central hub for all things HoloLens has links to everything listed above, plus things like the Galaxy Explorer source code, video gallery and so much more.

Microsoft Store

To see what other developers are building on HoloLens, I suggest games like RoboRaid for its spatial understanding, and Young Conker for their use of the environment including things like sofas and tables. Fragments is another must see, it’s probably the best example of the visual quality bar a HoloLens experience can achieve, it’s so immersive and amazingly performant. There are so many great apps on the Store and new ones being added all the time. There’s always something new to check out.

If there’s anything I missed, let me know in the comments below. I can’t wait to see what you build!

To stay in-the-know with Microsoft Design, follow us on Dribbble, Twitter and Facebook, or join our Windows Insider program. And if you are interested in joining our team, head over to aka.ms/DesignCareers.

Designing for Mixed Reality was originally published in Microsoft Design on Medium, where people are continuing the conversation by highlighting and responding to this story.