Artifacts of Utterance: Interpretation and Translation in the age of AI

Project Brief: Regarding the question of human intelligences and artificial intelligences (of various kinds) together, in environments — what dimensions could there be to these interactions, and what issues do they highlight, now and in the future? What is the role of designers in these situations?


Harold Cohen’s AARON AI Collaborator, Christ Schmandt’s “Put That There” (1979); MIT Media Lab Speech Interface group by Uber, Tesla Computer Vision
Google Magenta’s NSynth for music synthesis, Lobe Visual Interface for deep learning, “Project Dreamcatcher” by Autodesk for form interpolation

Voice Intelligence

Breaking Down Natural Speech

Illustration by Bee Johnson

What You Mean

“Good for YOU” vs. “Good FOR you” vs. “GOOD for you”
“Fig. 6. Schematic illustrations of the phonetic profiles of positive and negative intensification that emerged from the key words with (a) short vowels (V S ) and (b) long vowels (V L ) in the accented target syllables. The shapes of the polygons in the lower panels represent the acoustic energy (E) courses. The upper panels sketch the characteristic F0 peak contours. Broken lines point to the possibility of voiceless-onset consonants. The different shades of the segment polygons refer to the differences in voice quality (i.e. lighter = breathier). All illustrations are based on the means of table 1. F0 ranges are oriented towards actually found values.” — Oliver Niebuhr, On the Phonetics of Intensifying Emphasis in German
Figure 6. Australian English diphthong schematic trajectories superimposed onto the traditional vowel map with IPA cardinal vowels indicated (International Phonetic Association, 1999).

Emotive Modeling

Where You Are

Design for Meaning-Finding & Visualizing

Technical Framework

Form Manipulation with Voice

Bark Extractor → OSC → Wekinator → OSC → Unity
UnityOSC connection (github), Noise Shader (Char Stiles), (Animator + Scaling, Scott Leinweiber)

Thank you :)

Research + References

“One thing we know about creativity is that it typically occurs when people who have mastered two or more quite different fields use the framework in one to think afresh about the other. Intuitively, you know this is true. Leonardo da Vinci was a great artist, scientist and inventor, and each specialty nourished the other. He was a great lateral thinker. But if you spend your whole life in one silo, you will never have either the knowledge or mental agility to do the synthesis, connect the dots, which is usually where the next great breakthrough is found.” — Marc Tucker, the president of the National Center on Education and the Economy:

Early project by Chris Schmandt (1979); MIT Media Lab Speech Interface group video collection

Unrelated But Related


AI Learning Techniques

Documentation + Dev Tools + Educational



Love podcasts or audiobooks? Learn on the go with our new app.

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store