Milestone
Preliminary ideas
My goal was to develop my “Wekinate your world” project in which I controlled an avatar’s mouth between two positions, one open and one closed. Specifically, I thought about different methods to make my next toy more expressive, allowing a user to move other parts of the avatar’s head and/or have a broader diversity of movements.
Prototyping
In this latter goal, I manually annotated the timing of the consecutive open and closed mouth positions. Instead of navigating between two points of a video to animate an avatar, I now keep going through the video to obtain different face and body expressions. To move forward in the video, the user needs to open or close their mouth fully. The video keeps being interpolated between the two current open/closed positions as long as this is not the case. (I uploaded my manually annotated data on my Drive below for the milestone. I did not have any prototype to show at this point.)
Project
Sesame Street Puppets
While making the previous toy, I realized how odd the final result felt. We were straight in the uncanny valley.
As our goal was to escape the Turing trap, I tried stepping out of this valley in two ways. First, I tried to make it obvious that this avatar was not human but controlled by me in a humorous tone. Second, I chose to further animate puppets from Sesame Street. I thought puppets were particularly well-suited for this toy since the range of their expression and believability mostly rely on their mouth position as well as what they say. Their movements are significant but can be interpreted in a lot of different ways based on the context.
Interactive toy
In this final project, I wanted to focus on the human part of the toy. Puppets are great tools for sharing stories and playing with children. By allowing any user to control a puppet with both their face or their hand, I hope users can spend quality time with other people.
I also started working on a system that would allow multiple users to interact with two puppets by splitting the screen. I could imagine a lot of fun features, but the result with this simple method was quite satisfying for now.
Performance
In my final performance, I was hoping to convey parts of these reflections: making an uncanny avatar sound more funny than uncanny, demonstrating the expressiveness of the toy, creating stories from it, highlighting the humans behind the tools, their enjoyment and emotions…
Final Reflection
One word to describe how I felt in the creation process: Myself!
It’s been a long time since I had not felt that way while working on class or research projects. Through this work, I felt like I was placing hope for my future works. Hope to feel as joyful, passionate, creative, proud, personal… I had such a great time spending hours and hours prototyping this toy and preparing this performance. I feel that I have developed a better perspective of what AI is and what I want AI to be and that I have grown more as a human being through this class than through ten other Machine Learning / AI classes. Thank you so much to Ge, Yikai, and all my classmates who made this class so special. I look forward to seeing all your creations!
Deliverables
Please watch the full video! :)
https://drive.google.com/drive/folders/1wUPfHfp4eqVo0nM4ukOL-2OhH2iyqmLt?usp=sharing
Thank you everyone!
— Jean-Peïc Chou
