The psychology of AI Authoring

Two years ago Barney Pell, an AI inventor and investor, gave me some killer advice — when I told him I was working on a ChatBot entertainment product:

“always make sure that your users KNOW what your ChatBot understands.”

What Barney was saying to me was “It’s impossible to truly use a ChatBot (and its hidden value) unless you see how to control it — but since ChatBots don’t have any user interface (beside a Messaging input field) its imperative to somehow let the user know what the ChatBots commands, controls and key words/functions are — or else you’ll fail.”

So here we are, two years later, and a lot has changed, but Barney’s words still ring even truer today. We no longer call them Bots, we’ve built a prototype and tested it and now we’re focused on getting a working version of our system “useable” so that others can experience what we’re calling “AI Authoring.”

AI is such a broad term and authoring has long been abandoned by our industry, so lets start off with defining just what we mean by AI Authoring.

We’re in the entertainment business. Our data is fictional storytelling that humans have been doing — since the dawn of time. We’re not enabling our users to create support Bots, sell things, give advice or book a table or flight. We’re enabling our users to take their Instagram Stories, and turn them into sentient Beings and Share them with their friends.

Fun, interactive, posse-based conversations unfold and a new kind of self-expression is born. Creators in the future will utilize AI, as much as they do videos and images — in their storytelling.

In our case AI is an NLP engine which is wrapped around and inside of a slideshow of media (videos, images, text = Stories) which takes on a “sentient existence” and can then “converse” or “answer questions” that are poised to it by users (which we call Interactors) — in a Chat App.

The problem we’re solving is that simple “commenting” has been around for 20+ years and its time to go beyond commenting, where the audience becomes part of the creative artifact.

“Wow” — you say, pretty coolio — huh?

The trick in making this kind of system work is in coming up with a UX/UI that can be easily understood and used by humans not programmers, mathematicians or data scientists. That’s the Authoring part of the equation.

For those of you who don’t know — I got my fortune and fame (which lasted no more than :15 mins) from helping to invent an Authoring system for multimedia. It was called MacroMind Director. It enabled an entire industry of (what were called) multimedia developers to build games, marketing presentations, scientific visualziations, product simulation and educational courseware. All from one product.

If you don’t know what I’m talking about, go ask somebody over 45 years old.

MacroMind became Macromedia and they eventually bought a clone of Director; called Flash, so that they’d have something for the web. Flash enabled YouTube and animated ads — and the rest is history.

It was the timeline based authoring paradigm that enabled non-programmers to create animations, interactive branching and all sorts of yummy multimedia effects. Director and Flash were called multimedia “authoring tools.”

Jump forward to today. Try and build an AI product. There are a ton of ChatBot tools out there and all sorts of authoring systems for advanced biotech, manufacturing, data analytics and just about every way you can imagine AI can be applied to technology and industry.

Authoring tools for AI are a must, because how the hell are you gonna find the programmers and data scientists you need to build AI? They’ve all gone to work for Facebook, Google, Apple, Microsoft, Amazon or “the Chinese.”

This is why AI Authoring systems are such a big deal.

Humans creating AI Beings need to take into account the underlying factors, aspects and psychology of AI Authoring.

I’m writing up the following “factors” and “aspects” of the Psychology of AI Authoring for our team, but I decided to turn this into a Medium Post so that everyone could benefit from the work we’re doing.

Since Barney helped me, I’ll pay forward and help others….

When I talked about “seeing” an interface (above) I was referring to the psychological effect of a GUI (graphical user interface.) There was a time — before the Xerox Star and Macintosh — when computers were controlled from “command line” interfaces, but that time is long gone.

Nowadays one looks at the screen (whether mobile or web based), and gets cues and instructions (from the interface) as to how to operate and interact with the software — which (when designed correctly) — guides the user through the proper range of choices and potential functionality.

This psychological process is punctuated with instructions, alerts and notifications to make SURE that the user does the right thing — at the right now. Real-time feedback when using these interface controls adds to the cognitive understanding as to “what the hell is going on” and “what these tools — can do.”

Modern software (which relies upon AI and all sorts of “smart software”) also takes context into account — which I have been doing in all my software designs, since I read Robert Scoble and Shel Israel’s book — Age of Context — back in 2013.

Context is everything — and especially true when trying to author compelling interactive conversations — which cause your users to break out in loud — laughter and gaffaws!

So what I’m doing here (below) is outline 14 features, factors and aspects that need to be taken into account when creating an AI Authoring system for Storytelling Beings. This is (of course) everything BESIDES the Narrow NLP system which we’re building. eg. there’s a whole LOT that goes into building an AI Authoring system for Storytelling Beings!

These factors and aspects are the features, display renderings and internal “logic” that keeps an Instigate author (we call them “Instigators”) engaged, attentive and creative.

We call our company Instigate — ‘cause we’re enabling normal people to Instigate a new kind of self expression — by adding interactivity and intelligence to their Instagram Stories.

Creating AI (smart software) for humans to control their very own Being (proxy — sentient Being) requires a system that provides feedback and clarification to their efforts and actions. As the human comes up the learning curve of the Authoring enviornment, they acclimate themselves to the various factors and techniques that the tool provides — in acomplishing their tasks.

This learning curve IS the psychological affect.

Instigate is going to use a classic “progressive” gaming technique of working your way up through levels of the system, where each level unveils a new concept, feature or set of capabilities — which will expand their universe of authoring.

There’s a lot that goes into authoring AI, and we’re going to gradually unveil these techniques — as the user (the Instigator) progresses through the levels.

They’ll progress through accomplishments, milestones — and ultimately points. And yes — we’ll eventually have a leaderbaord so that they (the authors) can compare their progress — to ther authors.)

Our users will need to know a) WHAT works, b) what DIDN’T work the way you THOUGHT it would work and c) HOW and WHAT you have to do — to GET IT TO WORK.

Lord knows that debugging one’s own personal sentient Being — is going to be a challenge — and that’s why I’m taking these psychological factors — so seriously.

In our case — our GOAL is to create an AI Authoring system which enables normal people to create their own sentient Being (Bot) which can converse, answer questions and suspend disbelief — that there’s some kind of “Pet Rock”, Tamagotchi Being thingie living inside their smartphone.

The Instigate AI Authoring system takes these sentient Beings and wraps them around and inside of pre-existing media slideshows, which are called Stories. Collectively over 1B of these Stories are posted every day; on Snap, Instagram FM Messenger, Facebook and WhatsApp.

Here are some crucial aspects and tendencies which have to be implemented ALONG WITH killer NLP code — and the Authoring tools — per se.

1. Though we’d prefer NOT to get into Avatars, 3D animations and AR/VR — there probably has to be some sort of humanoid, facial personification of the meta-Being — the Agent — who is guiding and leading the Author (the Instigator) through the authoring process. The Agent’s face will appear alongside the notifications and instructions — to humanize the entire authoring experience. Everyone gets that. Its totally a cliché, but that’s OK. The idea is to make the author feel comfortable in the initial “training wheels” (on-boarding) Level I phase of the system. Ideally one would be able to “turn-off” the goofy training wheels — at later Levels.
2. The Instigate author is handed a “bag of magic tricks” which they’ll utilize in training their Being. So its imperative that the author get some sort of indication when one of their magic tricks — works. We call this a “hit.” The hit may be a match between a whitelist of key words (that the author has defined) and something the user (the person interacting with the Being) types into the Chat App. Or the hit may be when an whitelist term has been extracted from the author’s video selfie, and matches something the user (which we call “an Interactor”) types into the Chat App — communicating with the Being. We imagine that a special “PING” appears on the author’s screen — whenever a “hit” occurs. A successful hit will then trigger a resultant effect/specific response/or Action.
3. One of the most exciting EFX Instigate will hand to our authors is what we call an “Ah-Hah” moment effect. Our testing has shown that 50% of the people we show an Ah-Hah moment to — respond with a big smile, or chuckle. An Ah-Hah moment video contains a series of whitelist terms that the author mentions when she is recording a (seamingly) innocuous diary entry or video gossip. Once the Ah-Hah moment video is displayed (as part of the Being’s storytelling) a series of pre-defined responses, comebacks and quips are triggered —on any hits between the author’s whielist and what was mentioned in the video. These hits then create the effect that the Being knows exactly what was just mentioned in the video — and is remarking on that video. Its quite an amazing effect, and something we suspect will prove to be a cornerstone of our author’s “bag of tricks.” Pulling off these Ah-Hah moment effects, debugging them and perfecting them — is going to be really important — and MUST — be really easy to implement!
4. Debugging in general is also important, so we’ll need a series of “breakpoint” techniques, register displays and breadcrumb trail effects — which will enable our authors to track down just WHY something occured the way it did and what caused a particular effect to occur. Please wish us good luck on this one! How the hell do you make THIS easy to do?
5. Our Chat App enviornment will support dynamic messaging coming from our “Meta-Being” Agent — which will guide the author through debugging, inform them of the current state of the Being and any problems that may be occuring. These dynamic messages also provide Help in general. These messages will come in the form of Alerts, Notifications, Instructions and Annotations. The messages will appear as light grey text — inserted in-line — inside of the conversation — and the Chat App itself.
6. By progressing our authors through a series of Levels, we can control just how much they have to learn and comprehend, based upon what they’ve shown us — by their actions. AI Authoring entails over 20 concepts which must be understood. Just like adventure games gradually expose gamers to “campaigns”, bullets, magic potions, mining for gold, player lives or golden swords — we will gradually expose our authors to concepts such as: topics, backstory, themes and story structures.
7. Reminders will prove to be a powerful tool to keep authors engaged and coming back. “Please don’t stop, keep going” our Meta-Being will plea! “You’ve done such a GREAT job so far, keep going!” AI Authoring takes a lot of content and creativity to work. The more you put in, the more you get out of it. Once a Being’s Episode has been shared, it can always be updated and improved. Its more like a YouTube Channel or Blog, than a ChatBot.
8. As mentioned above — understanding HOW and WHY some particular response or statement was made by the Being— is imperative to controlling and authoring Storytelling Beings. We’re hoping that once we apply machine learning (and later deep learning) to our Beings — that they’ll get smarter over time. At least those are our goals. So in that quest, just how and why something happens is actually (what is called) the “feature engineering” of the machine and deep learning. Again — this is where you light a candle, contribute to our TipJar and wish us “Best of Luck!”
9. We believe in reinforcement learning, so constantly telling the author “these are the concepts you’ve learned, this is what you’re being taught right now, and here’s a list of what you WILL be taught” — moving forward — is going to be very effective. When dealing with large populations of users (authors) who have a wide range of intelligence, context and demographics — we think that reineforcement learning will be more beneficial than annoying. Again - this is something that we imagine — can be turned off — at later Levels.
10. The Instigate AI Authoring system will have three distinct modes of operation. Displaying (on-screen) which mode the system is currently in will also prove to be helpful in the overall pedagogical environment. Mode displays are DEFINITELY something that can be turned on and off. FWIW our three modes are: conversing (with the Being), authoring (to train the Being) and testing (figuring out if what you did works and is effective/fun!)
11. Not only does the system’s messages get driven by context and a wide range of behavior and actions, so will the UX/UI itself morph and adapt itself to the user. Depending which level and mode they’re in and what they’ve just done — expect to see new tool features, functionality and animating interfaces — come and go.
12. Having a Dashboard on everything — is a well proven technique for Enterprise and Gaming software — so why not have one for creative AI Storytelling? Seeing the instances of the Being (so that the resultant conversations can be viewed and joined), displaying the statistics and analytics of the Being and accessing a list of all vocabulary/commands (that Barney insisted we include) are all part of the Instigate User’s dashboard.
13. Taking the “contextually sentitive” meme even further, we imagine our system will be able to automatically adapt itself to various platforms it resides on top of. Alexa voice activiated interaction (or Bixby/ Cortana/ Google Asst) seems very likely to appear, as well as working alongside a wide range of text based messaging platforms, if they’ll allow us to plug-in. WeChat in China is of particular interest to us. If you’ll recall it was Director — that enabled the same file to play back on Windows AND the Macintosh — that got us our VC funding from KPCB. Thus was born “author once, playback in many places…..”
14. And finally — who can resist the notion of a Meta-Being, explaining the Concept of Concepts to Instigator authors? Explaining explaining seems to be so Meta and the best way to end an article on the Psychology of AI Authoring. Everything we’ll do will be about making it easy and intuitive for our authors — so that they can create an entirely new form of self-expression.

We’re just the toolsmiths.

in closing it should be noted that we (MacroMind) had no clue as to what our product (Director) would be used for. We thought it was a “videogame development tool.”

Little did we realize that animated marketing presentations, scientific visualizations, product simulations, educational courseware — would be possible.

It was our multimedia developer users — who invented that.

So too — are we hoping that 1,000 nodes of light in the neural network of synethetic life — will be born by our Instigators!