We built an award-winning product to help the visually impaired in under 48 hours. Here’s how we did it.

Phillip Caudell
Dec 7, 2016 · 2 min read

We spent the weekend at TechCrunch Disrupt’s 2016 London hackathon. It was a pretty intense 48 hours of brainstorming, plotting, researching, eating, building, testing, bit more eating, topped off with a presentation to a few hundred attendees in the room. Gulp. But we won a prize. The special IBM Watson award!

This was the first time we had donned our branded hoodies and stormed a hackathon as a team, so we were pretty happy with the outcome. Here’s some more details about what we built.

Photo credit: Farah Kanji

Inspiration

One of my family members has an eye disease called Glaucoma which damaged the optic nerves in their eyes. The damage has resulted in severe vision loss, meaning they struggle to find everyday items around them. With advancements in computer vision, it seemed reasonable that we could replace their eyes with an artificial one.

What It Actually Does

AEye is an artificial eye. Using a combination of the microphone and camera on the device, someone is able to ask where an object is and be guided to it. Moving around the room will result in a “hot” or “cold” reading.

Video demo of final product

How We Built It

Split into two teams: 3 people concentrating on the mobile app & the UX design; 2 concentrating on the backend and visual recognition training.

We have trained multiple image classifiers on the Watson visual recognition service. This can give a reliable match when the image contains the object we’re searching for. This service is wrapped by a Python web service, which maps the desired object class to the associated classifier.

Early prototype app we built to prove the technology works

The iOS app uses the Watson Speech to Text service to translate voice input from user. Once the user selects a class to search for, we give aural feedback (“warmer”, “colder”, “found it”, etc) as the user moves the phone’s camera around the room or surface.

Tech We Used

  • Swift
  • Python
  • IBM-Watson
  • Radix

So yeah, without a doubt the next step would be to add celebrity voiceovers. Jeff Goldblum perhaps?

Looking forward to the next Hack! 😬

Founders Factory

Great startups set out to solve big problems.

Phillip Caudell

Written by

Maker of digital products and businesses @ Founders Factory. Working with easyJet, L'Oreal, The Guardian, Aviva, M&S, CSC & Holtzbrinck

Founders Factory

Great startups set out to solve big problems. We’re here to give founders what they need to solve them faster, with our studio and acceleration programs.

Phillip Caudell

Written by

Maker of digital products and businesses @ Founders Factory. Working with easyJet, L'Oreal, The Guardian, Aviva, M&S, CSC & Holtzbrinck

Founders Factory

Great startups set out to solve big problems. We’re here to give founders what they need to solve them faster, with our studio and acceleration programs.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store