AI’s biggest challenge is human, not technological.

People often ask me whether AI really works yet — is it as smart as AI companies paint it to be, how much is automation versus AI? The questions point to the technological hurdles gating AI being effective.

However, we often overlook a far tougher challenge in giving life to AI: understanding human behavior.

AI’s biggest challenge is going to be human, not technological.

Software optimizes on the human values we give it

In my time as VP of Product at Mezi, we constantly made our AI smarter, less breakable, and able to recognize more inputs. But at the core of the user’s AI experience was a set of human values we defined manually upfront. Only then did the AI know what to optimize for in its interactions.

That’s a travel assistant, but what about when AI takes on the role of doctor, therapist, friend, teacher, artist or soldier? Does it want you to follow your dreams or make a steady income? Does it want you to hear happy or sad music after your breakup? Does it let you enjoy the holidays before getting back on your diet?

We don’t have a firm grasp on our humanity

Technology has advanced far more quickly than the human self-awareness has.

A few historical milestones in technology:

  • In 1879, electricity found its first practical use in the light bulb.
  • By 1903, the first airplane took flight with an electric engine.
  • By 1957, the first satellite launched into space.
  • In 2016, the first low-cost prefab housing was launched into space to create habitats outside of Earth.

Let’s compare this to the evolution of human nature in the United States:

  • In 1865, Jim Crow laws enforced racial segregation upon the abolition of slavery.
  • In 1942, Japanese American internment upon Pearl Harbor attack.
  • In 2017, immigration ban for citizens of 6 Muslim-majority countries.

Unlike the exponential growth of technology, human nature follows a Groundhog Day pattern. In the 3 examples above, the same need for control, dominance and “othering” repeats itself. And that’s reminiscent of the human nature shown on the school playground where one child bullies another or at a baseball game where one side boos the other. We intellectually understand equality, love and compassion yet are run by underlying and competing desires. This plays out in attempts to reform our circumstances while not reforming our deeper thought process — until this happens, we’re bound to relive our mistakes on different audiences under different circumstances. I argue that human nature itself is ripe for disruption.

We don’t know ourselves well enough to endow values onto infinitely powerful machines.

A new focus on reflection and self-awareness

Self-awareness: Conscious knowledge of one’s own character, feelings, motives and desires.

Our best intentions often butt up against a competing and undeclared set of intentions. We intend to get along with our parents — but when they do that thing they always do, we lose our temper. We are overcome by even stronger intentions to assert our independence, show them they’re wrong, or conceal our vulnerability. We intend to find a job that we’re passionate about— but are confronted by stronger intentions to relax and treat ourselves to Game of Thrones, spend time with friends and family, or avoid failure.

Most people can’t name, much less prioritize, the intentions that truly guide their lives. The AI world will face the same issue when attempting to seed artificial consciousness.

A plethora of self-development experiences and methodologies exist, and it’s time that we get past apathy and explore them. From my own journey, the following have been the most profound and transformative:

  • Mindfulness meditation. Amazing for developing the ability to be calm, self-aware and intentional on demand.
  • Landmark Forum. 3 and 1/2 days of intense self-inquiry in a room of 150 people shedding decades of limiting beliefs and walking out connected to and moved by myself, others and what’s possible in my life.
  • Eckhart Tolle. The book “A New Earth” turned my world upside-down and helped me discover spirituality in a modern and intellectually grounded way.

A new kind leadership is necessary

We’ve made huge advancements in mastering the elements — from sending objects to Jupiter to editing DNA to harvesting solar energy. But the skills required to make technological miracles are not the same skills required to seed the character and values of a self-guiding, extraordinarily smart artificial being.

Steve Jobs transformed the world of mobile connectivity by giving people a reason to buy a smartphone — he made technology irresistible. The other side of the coin was a play on consumers’ need to feel exclusive, elite and prestigious. His products ultimately perpetuated his own desires to feel special, wanted, better than you. How would a Siri robot react if someone was being condescending to her? Will the designers of Siri optimize for acquiescence, mutual understanding or retaliation?

The qualifications of a consciousness creator are not in engineering, design or business, but rather deep knowledge of and passion for humanity. A new leadership role of Chief Consciousness Officer with a background in psychology, mindfulness or coaching will become necessary.


There has never been a greater responsibility on the part of tech companies to develop a deep understanding of what human betterment and happiness really means. Tactics that have helped companies in the past — developing addictive behaviors, viral loops, switching costs — cannot exist at the heart of unimaginably powerful technology.

AI will be made in man’s image. Man’s image will either be rudely reflected back to us in unintended behaviors and outcomes, or thoughtfully created to fulfill on the values that truly benefit us. Now is the time to revisit who we are and what we stand for.

Before we build infinitely powerful machines, we must first disrupt human nature itself.

If you enjoyed this story, please click the 👏 button and share to help others find it! Feel free to leave a comment below.

The Mission publishes stories, videos, and podcasts that make smart people smarter. You can subscribe to get them here.