Teaching a bell to ring when your pancakes are ready
Experimenting with putting teachable technology in everyday objects.
Imagine, for a second, that you had a magic bell that you could teach when you wanted it to ring. When would you want it to ring? I have pretty bad posture, so I’d want it to ring every time I slouched, a reminder to sit up straight. (Perhaps too Pavlovian?) Our friend Sam wanted to tell it to ring whenever a bluejay (not any other kind of bird) was on their bird feeder. And our friend Kay, a musician, said he’d tell the bell to ring whenever his goldfish swam into a particular part of the fish tank (apparently he’s always wanted to use his goldfish as an instrument).
Sure, a teachable bell might be a silly idea. But the point of this post is to inspire you to think about a larger idea: what might you do if you could teach everyday objects something uniquely useful to you?
We’ll explore this idea of “teachable technology,” and how you can create a hacky prototype using off-the-shelf electronic parts. We’ll share what we’ve learned along the way and probably end up asking more questions than we answer like:
- What would a simple, understandable teaching interface look like?
- How might putting a bit of teachability into something as basic as a bell change what people would use it for?
- What other things might people be inspired to make teachable?
- What if objects were so easy to teach that anyone — not just programmers — could teach based on unique needs for themselves or their communities?
- And how could this all be done in a way that’s respectful of privacy?
We’re sharing our sketches in this post, as a starting point for anyone — designers, teachers, students — to explore these ideas further.
Our prototype
Now to make a magic bell… We did some quick prototyping using parts that are accessible for anyone to buy: a Raspberry Pi, a Coral, and other DIY electronic parts via sites like Adafruit. As you can see above, it’s pretty hacky! So it’s probably not something you’ll use everyday in its current state. But that’s what makes this a fun thought experiment. We’re speculating a bit into the future and poking around to help all of us, as potential designers of these systems, start asking questions together.
One note to keep in mind as we demonstrate the prototype: We needed to write code to build this prototype, but our ultimate goal is to illustrate how a user could, one day, use a system like this without any coding. They would just hook up whatever sensor or signals they wanted, and teach it just by pushing buttons.
So, it has a few basic parts: The input, which is a camera in this case (but could be other kinds of sensors). The decider, which is the computer which will basically just decide the answer to a yes/no question. And the output — our bell.
For a fun test, let’s try to make the bell ding every time it senses a thumbs up. So, we’re going to teach the decider to answer the simple yes/no question “Is my thumb up?” First, we show it thumbs up while holding the “Yes” button. Then we hold the “No” button to show it things that aren’t a thumbs up — like an empty background, my hand in other positions, and so on.
But, one nice thing we found about a simple system like this is that it lets you rapidly iterate — so, that if we find the bell is dinging when it’s not supposed to, we can quickly teach it with more yes/no examples.
An important note about privacy: In this prototype, the images from the camera stay on-device and are not saved or sent to any server. So the camera is acting less like a traditional camera, and more like a sensor. (More thoughts on this below.)
Trying it out
We quickly tried out our prototype in different scenarios around our house. It got pretty fun!
“Is the pancake ready to flip?” We taught the bell to ring when the pancake was ready to flip. (You know, like when you start to see bubbles!)
“Are my eggs done the way I like them?” We taught the bell to ding when the eggs were ready. This was kind of a silly use case, but it was interesting because each of us prefers our eggs a bit different — scrambled, sunny side up, and so on. It sparked interesting conversations around how we could one day teach computers to recognize subtle differences important to each of us.
While these are just a few fun silly experiments we did around the kitchen, there’s no reason why it couldn’t be extended to other things like sensing when a hummingbird is at its feeder, letting you know when you’ve started to slouch, and so on.
Thought Starters
This idea of “teachable technology” — enabling any user to easily teach their technology using AI — is not a new idea. But recently we’ve seen a growing interest, especially with people using Teachable Machine to explore their own ideas, teaching their own machine learning systems for accessibility, design/creativity, and more. The goal isn’t a future with lots of ringing bells, but one where technology is more accessible for everyone, as something that can be taught by anyone — not just techy folks.
If you’re interested in exploring teachable technology, here are some thought starters:
- Privacy. It was important to us to design this system in a privacy-respectful way. We usually think of cameras as capturing and storing images. But here, the camera acts more like a sensor, and no images are stored or leave the device. But how might we design systems like this in the future so that this critical aspect of privacy is easily understood by everyone?
- User as programmer. Most of the AI systems you use on your devices today are pre-built. But a system like this would arrive in your hands more as a blank slate, ready to be taught by you. You become the programmer. But instead of writing lines of code, you teach it by example.
- Custom AI systems uniquely helpful to you. We’re excited about how teachable technology could enable anyone to create custom AI systems for their unique needs. Systems like this could enable people to quickly tinker and try out ideas for themselves, their households, or their communities.
Links
This prototype only scratches the surface. And of course, we’re not alone in exploring these ideas. Here are other past and current projects you might want to check out. We hope this post inspires you to explore these ideas further.
- Teachable Machine
- Teachable Sorter
- Coral.ai
- Teachable Snake (by Vince MingPu Shao)
- Objectifier (by Bjorn Karmann)
- Wekinator (by Rebecca Fiebrink)
About the Creators:
Lucas Ochoa , Gautam Bose, and Isaac Blankensmith are creative technologists at Google that are passionate about making technology more accessible for all makers, especially when it comes to machine learning and physical computing.
Collaborators: Alexander Chen, Jonas Jongejan, Nicole Bleuel
