Poetic Language between Humans and Artificial Intelligence

Where are the humans in AI?

CONCEPTS_1ST ITERATION

  • Poetic/ Lyricism — a metaphor or a symbol relates to so many inputs and context. Where is AI’s position in this topic?
  • Social Status/ Hierarchy — Often AI are called assistant. Does AI has social status or hierarchy between them?
  • Manners/Traditions — While younger people spend more time with machines and AI, they can easily lose contact with human intelligence, losing their culture and roots. Moreover, they might lose the manner and etiquette in daily lives.
  • Emotional Quotient/Intelligent Quotient/Adversity Quotient

EQ and IQ are not equally valued in human intelligence society

Moreover AQ is not as often talked about as EQ and IQ are. What does EQ, IQ, and AQ means for AI? Does AI need to worry about the value differences?

CONSIDERATIONS

(value)

  • moral decision making
  • reasoning style
  • fairness
  • visual perception
  • cultures

(environment)

  • reflection
  • experimentation
  • play
Bar-like section
Immersive environmetns — light & smoke

CLASS DISCUSSION

  • What level is it appropriate to the situation ?
  • Why does everything needs to be transparent?

ex) I dont want to know how the bread is made, but I want to know if its delicious.

  • Trust is needes.

PERSONAL FEEDBACK DISCUSSION

Topic: Emotional Quotient/Intelligent Quotient/Adversity Quotient

  • What would a poem written by AI be like?
  • If AI assistants are gathered, what is the hierarchy like?
  • Why has current AI been seen as female mostly?
  • What if AI help people to develop different forms of Intelligence and test them in different ways?
  • Why do humans go to the AI bar? Maybe because they are stress, sad, happy

THOUGHTS

— Emotional Quotient — ability to recognize, evaluate, regulate your own emotions and those around you.

— Intelligent Quotient — about arithmetic, vocabulary, general knowledge. Might assess base on income and job performance.

bar-like setting, social setting having AQ & EQ related activities

— It’s a service where AI gets information of humans and train, provoke, play, adjust, work with humans by having conversations.

— The environment changes and adjust based on the working topic

— Humans are critical when thinking for other’s problem, but not critical about their own.

Precedent work and phenomenon:

ex) TV show storyline: about a man going out for a date while having poeple montoring his action and give him suggestions communicating with the woman he’s interested in.

ex) youtube videos: practice eye contact through looking at youtube video on people starring at the camera lense.

What role does AI play in the consultant situation?

GUEST SPEAKERS

Madeleine Clare Elish from Data and Society

  • Technology can disempower, de-skilling

ex) It becomes harder for human (pilot) taking over control.

— hight stress, high information.

control is distributed, however human is responsible

— systems are imperfect, need manual to fill the gaps

  • Technology is not magic

—Society overestimating capacity of machine, while underestimating human

  • seamful vs seamless design

— seamful design, seeing at the making that of process & product

  • faults of increase personalization

—creating filter bubble

— unfortunate. Human should be exposed through technology

  • Explainability, not transparency

— to translate, teach lower school level, not just college level

Metaphor/ phrases can be useful

What might integrating social metaphore in environments look like?

What role does AI play?

Min Kyung Lee from Carnegie Mellon University Human-Computer Interaction in the Machine Learning Department

  • Algorith- human gap
  • one fairness | one-time allocation vs multiple fairness | Negotiation process
  • transparency & control design dimensions

David Danks and Emily LaRosa from CMU Philosophy Department/ the K&L Gates Endowment for Ethics and Computational Technologies

Ethics — human value, human interest

How to alter/shape social Interest?

  • Lose Self Identity

ex) AI → Doctor → Patient (Doctor, Professor… other professionals)

— Psycology harm, devaluating

How could we do to preserve important value, while integrating AI, focusing on human to human interaction

  • Seeing the relationship with animal rights and AI rights

— looking at their capibilities

AI in particular uses is more important than AI having larger scale impact

  • different culture has different values and morals

— What do you see as AI interaction?

  • Theory of Mind — “ is the ability to attribute mental states — beliefs, intents, desires, emotions, knowledge, etc. — to oneself, and to others, and to understand that others have beliefs, desires, intentions, and perspectives that are different from one’s own.”(Wiki)

— Human behavior generates by system and knowledge

Observed behavior vs whats going on in the mind

  • Personify AI

— “you(as creator)better get it right”

— if AI seems as human, seems smart, human can get easily frustrated quickly when AI cannot complete an ordered task, which would lose its use or its attraction from human

— How honest is the AI packaging

— What are people doing with this specific AI

  • Automated vs Autonomous

— Automated. like toaster

— Autonomous. What, why, how is AI learning? AI make inferences based on given data set

— Whats the responsibility in autonomous weapon?

  • metaphor for AI

— Goldfish

responding to the environments

— good at it’s function

doesn’t care

distinguish ex) needle & mosquito in an unsophisticated way

— a Tap Root

— a precise function, one type of AI

— context recognition and adaptation

When is the appropriate time to use it?

(Do we need a metaphor for AI, maybe it can be just a simple flow chart as what we need)

  • Trust Built

— Through repeated interaction

ex) predictability vs trust (car vs human)

— Trust is multidimensional

Artifact- predictability

— Human- Why and how system come to behave?

— What kind of trust you want to build in the system?

Cennydd — former Design Manager at Twitter, current Independent Design Consultant

  • Behavior is the heart of Design | heart of Capitalism
  • Nudge Theory
  • Hypernudge: Network, Dynamic, Personalized
  • “Technology dilute moral responsibility”
  • “Tech is never neutral”
  • authenticity & trust
  • Ethical Test

— What if everyone did what I’m about to do?

— Am I threating people as ends or means?

— Am I maximizing happines for the greatest number?

— Would I be proud if this were a front-page story?

— Is this fair behind the ‘veil of Ignorance’?

  • Reject false simplicity
  • Materialize invisible to visible (not transparency)
  • Should AI be explainable?
  • Informational nudges vs structural nudges

— changing single, less threat vs changing people action

  • datas

— less meaning in personal use, more valuable in mass data, looking at trends

  • Tech cannot be used if cannot be explan to citizen

CONCEPT_2ND ITERATION

How does human consult AI for human to human social interaction? How can AI assist and practice with human on their social skills; to provoke, play, adjust human emotions.

THOUGHTS

Interesting points from previous keynote speakers and research:

  • Human qualities: multi-dimensional human trust, emotional, Theory of Mind
  • Ethics:Technology should not be us when it cannot be explained
  • behavioral data (action & mind) vs data
Considerations when designing.
Characteristics of AI, HUMAN, and BOTH.

Questions for myself:

When in confusing situation: conversations → decision → how to react

Q: Why do we choose to have conversations with friends and family?

A: Because of the trust we built, the expereinces they had, and the context they already know about us

Q: Is the design therapeutic?

A: No, it’s more about consultancy.

Q: How do we trust AI? Where does AI knows what’s going on with my life?

A: AI gets data about my life through facebook — how often I check other’s social media. It gets my behavioral data based on my how often I engage with a subject.

  • behavioral data — facial recognition to define mood.

CONCEPT_3RD ITERATION

Human finds it easier to think rationally for another human, however, human can be lost in one’s emotions and thoughts. Here, when human is lost in emotions,

human consults AI to rationalize one’s intangible feelings at this bar-like environment.

Interactions

  1. Human approaches this bar- like setting.
  2. AI understands human’s state through facial recognition
  3. Human’s input an improvised poem, allowing AI connect internally with human
  4. AI output explanation of human’s confusion through metaphors.

[ with AI through this design, human finds meaning and clarity:

  • What does this confusion means to you?
  • How does this confusion affect you (positive vs negative)?

This design uses behavioral data (facial recognition) and literary data (improvised poem ) ]

PERSONAL FEEDBACK DISCUSSION

  • What if the user cannot write?
  • What do I want to show?
  • Why are they going to this designed environment?
  • How is consulting this AI better than consulting friends and families?

Is this service about AI responding to humans with more humans’ experience?

Is this getting closer to something like fortuneteller and horoscope predictions?

What is the story?

When does people interact with this service?

THOUGHTS

I do not want the service to be a loop about human consulting on what they already experienced, I would like the design to be more creative.

Why and when human approach the designed AI
Pros and cons comparison of approaching humans or AI when a human is in confusion situation.

AI robust & rigid vs Human flow & context

Few words are not sufficient to communicate and express thoughts, feelings, ideas. How to use literature and objects to communicate — using symbols, metaphor, poems?

Precedent work & thoughts:

Emotional First Aid Kit is designed to help those in stressful situations

Tools for Therapy encourage people to open up about their emotions -Nicolette Bodewes

How does emotional intelligence play in AI?

Emotional intelligence — the reaction about your feelings and the ones beside you.

whats the context of things?

poem and metaphor, one needs to know the context of it — culture, people, environment

ex) Chinese character is a form of metaphor of people and their environments.

metaphor — describe one as another

  • understand new things, conceive them in terms of things we already know- dan safer
  • metaphors are the foundation fo our language…- metaphors we live by
  • When we make decisions, there are subconscious elements, deeper meanings, other intangible factors
  • metaphors can create new life and meaning for a product
  • professor Theodore Levitt notes, “Metaphors and similes become surrogates for the tangibility that cannot be provided or experienced in advance.”
  • comparison is made between dissimilar things
  • use metaphor when it is not literally applicable
  • express a complexity of insight and feeling
  • inner connections

visual poetry- furniture

human — arguments, bias, judge, stereotypes

“This field of affective computing is developing quickly through sophisticated biometric sensors capable of measuring our galvanic skin response, brain waves, facial expressions, and other sources of emotional data.”

CONCEPT_4TH ITERATION

Human communicates intangable emotion to AI (technology) through poems with a designed kit.

PERSONAL FEEDBACK DISCUSSION

What form does it take?

Is this a design that is used all the time?

What is this design kit about?

Is this designed language about personalizing one’s technology?

Is this design about understanding one’s self?

What are the other poetic mediums?

TOUGHTS (REPRESENTATION)

  • calligraphy
  • ink
  • water
  • sprinkle
  • clay, supporting each other
  • kit
  • Rorschach test
  • poem as form code
  • smell, scent
  • balloon & sand
  • water
  • light

Speculative Needs Project was conducted with 12 Universities in Lebanon

Pages of poetry and art from a German doctor, Justinus Kerner’s Klecksographien (1890), inspired by accidental inkblot.
  • Some psychologists use the Rorschach test to examine a person’s personality characteristics and emotional functioning
  • “By asking the person to tell you what they see in the inkblot, they are actually telling you about themselves, and how they project meaning on to the real world,” — psychologist Mike Drayton for BBC
Scene from the film, Arrival (2016)
in Chinese characters, elements of symbols combine to create a new character and new meaning
German Compound words

CONCEPT_4TH ITERATION

Communicating intangible emotions between humans is challenging. However, because of contextual understanding of one another, humans feel intimate and comfortable to communicate with one another. Context is formed with background, culture, and bits and pieces of one’s experiences.

Where is the understanding of context between humans and Artificial Intelligence?

Today, code is the communication medium between human and Artificial Intelligence. Code is made of numbers and logistics. What if humans could introduce their way of communication to Artificial Intelligence? Imagine humans and AI or AI and AI

using metaphors, phrases, poems as language to communicate

with each other in deeper levels, having the ability to understand the intangible emotions.

EXERCISE POETRY COMMUNICATION WITH GOOGLE

Human:

Three days ago it’s summer

Yesterday was fall

Today is winter

What a dramatic four seasons

this week in pittsburgh

like my emotions

AI:

In Philadelphia, spring moves by at fast forward; one day is cold and blustery, branches bare, then air is warm and breezy. One late afternoon in mid-April, the sun fills the air with brightly lit dots of color — -greens, browns — leaves brightly backlit

(from The Language of Landscape

by Anne Whiston Spirn)

Human:

Global warming

I wonder what is happening

AI:

She cries ice and dries her tears in the fire Too Soon! They cry, as the Autumn Leaves fall They exchange seasons with cool white snow Our hottest summer Our coldest Winter

Human:

Three days ago it’s summer

Yesterday was fall

Today is winter

What a dramatic four seasons

this week in pittsburgh

like my emotions

AI:

summer, fall, winter, you are feeling

where is spring

with the flowers blooming

and seeds germinating

Human:

Missing the spring

might be the time to take a break

to rejuvenate

AI:

The anticipation of Spring is a great time to turn over a new leaf.

CONCEPT REPRESENTATIONS_1ST ITERATION

Each balloons represent is an element. Multiple elements combined forms a new character with new meanings
This is an example of how might human and AI communicate with each other in poems.

How do I present the concept? What is the form/ kit?

  1. two posters

- Importance of understanding of context. Humans and AI use metaphors, phrases, poems as language to communicate the intangible emotions

- communicate with AI in human terms

existed metaphors, symbols in different language (chinese, german, italian)

2. Two activities

language through form

create new meanings by compounding balloons with sands.

language through poem

inkblot booklet, poem respond as in AI or human

SPEAKERS

The Carnegie Mellon Universtiy — K & L Gates Conference on Ethics and AI

The talks was about important topics like agency & empowerment, misuse on human rights, legal, ethical, privacy challenges, and persuasion. Also, topics like capabilities & trust.

  • Preferences on “time well spent” — Policies, Disclosure, Controls, Education
  • Agency & Empowerment — what is our intention? what world are we creating?

SPEAKER

MiMi Onuoha — an artist and researcher using data and code to explore new forms of storytelling, social critique, and interaction

  • hiding the context (hiding a great deal of meaning) & terms of collection
  • “datasets are the results of their means of collection (which are usually hidden)”
  • “those with resources, lack incentives”
  • “data resist metrification”
  • “benefits to nonexistence”
  • problem: data not being collected — how do we make sense of this? how to open up to people?
  • demystifying

TOUGHTS (REPRESENTATION)

  • Inkblot
  • Talk to books
  • Balloon as communication tool?( How does baby and adult see the communication tool )
  • poetic logic
  • riddle, puzzles
  • How to represent human to human value?

SPEAKER

Anne Burdick — “Trina, Designing Text Technologies for Critical Interpretation”

Anne uses “speculative software/design fiction to ask how building tools around core humanities concepts might expand current models of knowledge”

  • Humanistic
  • Language, writing, body
  • Multiple interfaces , each represent a world view
  • Multi dimensional
  • Simultaneous coexistence
  • Annotation, while reading interfaces
  • Latin-Text have different meaning different context
  • digital humanities commence
  • How fiction is it?
  • Prototypes
  • Interface enable criticality
  • is reading a kind of travel?
  • The Trina Apparatus
  • Bruce sterling- design fiction (object, literary)
  • Story worlds
  • Cultural, memory, knowledge
  • critical thinking & subjective interpretation
  • Inaccessibility of human subjectivity
  • Interface create criticality
  • Animinoty — Wisconsin
  • Subjectivity to digital humanity

TOUGHTS (REPRESENTATION)

  • What happens when poem medium is the language between human and AI?

What’s the form? What’s the story?

What do you want ppl to feel?

  • poetry, abstract, literature, object, feedback
  • Pro/con about communicating with AI through poems
  • How does AI communicates with poems — pull out concepts from human poetry, pull out actual quotes from literature, response with poems human constructed
  • Why does AI wants more context? To give better advice and have better conversation
  • wellness of poetry
  • human input artistic medium, AI output poem to interpret and respond.
  • AI need feedback loop
  • video and toolkit (balloon, sketchbook, poetry, different forms of artistic reaction)
  • poster and kit (balloon, sand, ink, poem)
  • poster (quotes)
  • kit experience — balloon, ink interpretation (new meanings, poetry)
  • poetry, abstract, literature, object feedback
Sketch out what experience I want to create with my tools and how it will look like
Inkblot Making with Bristol paper and acrylic paints

EXHIBITION PLANNING/SETTING UP

EXHIBIT

EXHIBIT/PROJECTS DEBRIEF