Addressing autism: Project Pino

Perspectives from a Cognitive Build team at World of Watson

Cognitive Build is IBM’s global innovation initiative to help drive a culture of entrepreneurship within IBM and show all employees the transformative power of IBM Watson. Several Cognitive Build teams presented working prototypes at World of Watson (WoW) 2016. Jennifer Skeivik, who developed and demoed a prototype called Project Pino, shares her experience with Anne James, IBM Intranet Managing Editor.

What sort of project did you present at WoW and what was the inspiration behind it?

Jennifer Skeivik: I went to WoW to represent a Cognitive Build project prototype called Pino. Project Pino uses Watson Conversation service to help children with autism communicate more independently by providing real-time verbal prompts. It can also be used with other conditions that affect communication ability, such as stroke and Alzheimer’s disease.

My youngest son was diagnosed with autism and is mostly nonverbal. At a birthday party a couple of years ago, I saw how upset he was when he didn’t receive a cupcake because he couldn’t say “yes” when offered one. He needs a therapist or caregiver to prompt him to answer basic questions. He has a communication device that can help him speak, but it requires him to know he needs to respond.

When Cognitive Build started, I thought it would be great if my son’s communication device could be cognitive so it could help him to be more independent when I’m not around. I led the Cognitive Communications for Autism, Watson2Engage project, which aimed to enrich these communication devices with cognitive capability.

During the investment phase, I found Karabi Bharadwaj’s Pino project, which uses real-time verbal and visual prompts to increase spontaneous speech in people with autism, aphasia, and Down Syndrome.

Project Pino finished 30th in overall Cognitive Build funding out of about 2600 projects. Later, it was also a finalist in the India/South Asia Outthink Challenge.

I reached out to Karabi, and we have been collaborating on this common cause ever since.

Is this project related in any way to your IBM day job?

JS: It is actually related. I am part of the Open Technology group in IBM Cloud that encourages open source ecosystems and open standards around technology strategic to IBM. I am frustrated with the lack of open standards in the solutions currently available to my son.

Communication devices are highly proprietary. Even getting basic data such as progress toward his speech goals is a manual and inconsistent process.

I talked to several physicians at WoW who are also interested in collaborating on open standards and open source projects to accelerate progress in assistive communication technologies.

As it turns out, this type of collaboration is a great fit for developerWorks Open, a community for innovation and partnerships around IBM’s open source projects.

How was your project selected for WoW, and how else did you participate?

JS: The Cognitive Build Project Office invited teams to submit their interest in WoW participation. More than 60 teams applied. The selection process required us to demo a working, cognitive prototype. About 25 teams were invited to attend WoW and participate in a “pitch fest” on the Cognitive Build stage.

We were also advocates for the Cognitive Build in conversations with clients and partners who were interested in doing something similar in their own organizations.

How did you learn how to incorporate Watson APIs into your prototype?

JS: I heard this question a lot at WoW. Fortunately, there are a lot of great resources available to get started with the Watson APIs. We used the open source samples and examples from Watson Developer Cloud, the developerWorks Open tutorials, and developerWorks Recipes. I especially like the “Deploy to Bluemix” capability. If you’re not sure what you need, quickly deploying an application and playing with the code saves a lot of time.

Did you get any helpful feedback about your project at WoW?

JS: We got a lot of really insightful feedback. A couple of themes emerged. Many people commented that it’s important not only to prompt for a response in a conversation but also to gauge the way the response is delivered — in both tone and facial expression. And use cases for other conditions were suggested, such as monitoring the cadence of a conversation during a manic episode in someone with bipolar disorder.

A highlight was meeting, by coincidence, Dr. Gowtham Rao, Chief Medical Information Officer at the company that insures my son! After I demoed Project Pino, he high-fived me. I was excited to meet a direct stakeholder who is very familiar with the challenges and healthcare costs associated with communication disorders.

Demoing Project Pino was so much fun. I met a lot of great people…and even a couple of Sesame Street characters.

What are the next steps for Project Pino?

JS: Winning a top-three spot in the WoW Cognitive Build “pitch fest” helped attract even more attention to Project Pino.

Individuals with links to healthcare institutions have expressed interest in exploring collaboration potential.

We are currently looking for feedback from clinicians and researchers who work with children with autism. We are also looking for other open source collaborators and subject matter experts in speech therapy and related standards for assistive communication.

Cognitive Build was a great and challenging experience. Contributors from around the world rallied together around a common cause and helped us move the project forward as we learned new skills — all this while keeping up with our day jobs.

It was exciting to meet the other teams in person at WoW and learn about their projects. I feel very fortunate to have been involved. Karabi and I can’t wait to see where else this project will go!

Jennifer Skeivik is an Open Source Project Manager at IBM