Illusion of life by Software

A theory on how to create (and analyze) virtual agents

Published in

Eibriel

7 min readAug 13, 2019

Through this work I propose the creation of a new concept related to animation and animatronics, but applied to computer programs.

Animation is often called the art of the illusion of life, this is due to the ability that must have an artist of animation to make an inanimate and static element, such as a drawing, or a doll seems, in front of the viewer, to have their own thoughts, desires and fears.

Mitsuku, one of the bests social chatbots currently available

Not only must this static element gain movement, but it must also be capable of generating empathy.

At the same time it would seem that the creation of an Artificial Intelligence agent would automatically result in an agent that, if it does not have “life”, at least generates an illusion thanks to its intelligent behavior. Unfortunately it is not so, many techniques have to be executed in accordance to give rise to the emergence of Illusion of life by Software (from now on simply illusion).

Support my work by wishlisting Compicactus on Steam

The N principles of illusion

Here are some principles that I believe must be followed to achieve the illusion.
I call them “The N principles…” because the number can change over time.

Glossary

Software: A computer program, usually for a specific purpose (informing, entertaining, etc.).
User: A human being interacting with software, usually for a specific purpose (to obtain information, entertain, etc.).
Stimulus: The action of the user who is manipulating the Software (Click, Voice, Drag, Text, etc.).
Answer: The response of the Software to the stimulus of the user or the environment (Sound, Text, Animation, etc.).

The software can be natural language processing, image and animation generation, or mixed.

Timing

The manipulation of the intervals in which different events take place generates a timing capable of generating an emotional response in the user.

The waiting time between the stimulus and the response
The length of the answer
The type of response (text, audio, images, video)

Gradients and thresholds

There has to be a relationship between the stimulus and the response.
Gradient: The intensity of some responses will be proportional to the stimulus.

For example:

Stimulus: Hello
Answer: How are you?

Threshold: There will be no response unless a certain threshold of intensity is crossed in the stimulus.

Stimulus: Hello
Answer: Hello!
Stimulus: Hello
Answer: Good!
Stimulus: Hello
Answer: Do you only know how to say hello? (In this case the threshold of 2 repetitions of the same stimulus was crossed)

Little Computer People, an interesting game featuring an artificial life

Opinion

The answer should say something about the user’s stimulus. Also called Trampoline Moments by Pullstring.

Example taken from the book Create Convincing Computer Conversations:

If a user answers the question “What is your favorite color?” by saying “Red”, answering “I love that color” is very vague. […] However if the character answers “Oooh Red! I love that color”, it is more likely that the user will feel understood.

Fallback

Design interesting responses to unexpected stimuli (taken from the book Create Convincing Computer Conversations).
The characters are usually defined by the reaction to limit situations, and the limit situation par excellence for a software is when the user performs an interaction for which he does not have a predefined answer. (search for quotation)
It is also interesting that one of the definitions of Artificial Intelligence is “A program capable of acting when it does not know what action to take”.

The following examples show totally different personalities:

Stimulus: How much is 2 + 2?
Software Answer 1: I’m sorry, I don’t understand the question.
Answer Software 2: What?! I have bigger problems here

Staging

The interaction must be clear, (Staging) the user must know what has happened. It is one of the twelve principles of animation.

Attractive

The software must be charismatic, whether an ally or a villain.
It is also one of the twelve principles of animation.

Anticipation

Anticipating the user’s response, being one step ahead of the user generates the illusion of intuition in the character.

Reward

The user must be rewarded for interacting, must feel that he makes long-term decisions in the development of the illusion.

Imperfections

Living beings tend to make mistakes, a small amount of imperfections can help with illusion.

If it is a living organic being: errors in response, doubt, multiple attempts, lack of reaction.
If it is a living inorganic being: to interpret literally a stimulus, to react exaggeratedly.

Continuity

It must seem that the life of the character continues even when the software is not running, or when the user does not interact with it.
It also refers to the creation of a world where the character lives, and with which he or she must interact on a daily basis.

Stimulus: How much is 2 + 2?
Answer Software 1: Hello! I just got back from shopping. The result is 4. (The character’s life continued while the user did not interact)
Answer Software 2: One moment, I’m going to look at the calculator to be sure. (pause) The result is 4. (In this case the character interacted with his own world)

Analysis

Next I two different pieces of software to see how they use these elements.

Little Computer People

It’s a life simulation game released in 1985. Wikipedia, Gameplay

Timing: Regular in general, the dog adds a more irregular rhythm. Small actions have a faster cadence.
Gradients and thresholds: Mostly managed by thresholds, you can also see a gradients in the water tank, and how it becomes grumpy if you don’t say “please”.
Opinion: The character chooses whether or not to make the option based on his mood. And he regularly writes letters to the user expressing his opinion about his performance. When interacting with text commands sometimes the character shows that he agrees with the user by gesturing “yes” with his head. When interacting with keyboard shortcuts there is usually a sound or animation as a reaction.

The character writing a letter, while the dogs sleeps

Fallback: I need to check, but apparently the Little Person will just ignore an unknown command.
Staging: You can clearly see what is the character doing, when a command is understood it say “yes” with the head.
Attractive: The character is continuously acting, even though the user did not interact with it.
Anticipation: The character constantly walking around the house serves as an anticipation (and demonstration) of the possible actions that the user may require.

The character proactively asking the user to play a game

Reward: The character writes thank-you letters, and responds to user interactions more frequently.
Imperfections: Each copy of the game has a different character, at least one of the characters writes with errors.
Continuity: There is a different character for each copy of the game, the state of the game is saved to continue it later.
Randomness: The character performs different actions on his own, he can also choose not to perform the action required by the user.

Mitsuku

It is a chatbot that uses AIML technology to maintain conversations with users. The software won 3 times the Loebner Award. Website, Telegram.

Timing: Responds instantly, the length of responses increases as the conversation progresses.
Gradients and thresholds: Internally Mitsuku keeps track of bad behavior and repetitions.

Mitsuku detecting and handling repetition

Opinion: The character usually comment on the user’s questions, for example if they contain spelling mistakes.
Fallback: If Mitsuku don’t understand the message she can tell a joke, be bullish about it, or try to play along.

Mitsuku telling a joke and playing smart

Staging: Mitsuku always responds in a correct easy to understand English.
Attractive: The character is always available to listen to what we have to say.
Anticipation: In rare occasions she can anticipate a question or an answer from the user. For example engaging in a Knock Knock joke.
Reward: The responses are always smart and not quite as expected, constantly surprising the user and teasing he or her to keep chatting.
Imperfections: The character may misinterpret some of the user’s comments, in order to deflect the subject or give a humorous note.
Continuity: The character can remember user aspects and conversation information from one session to another.
Randomness: There is not much randomness on Mitsuku aside of responding differently to the same question.