Microsoft Azure
Published in

Microsoft Azure

Making a wearable live caption display using Azure Cognitive Services and Ably Realtime

How I helped my mother, who is hard of hearing, to understand me while I visit her with my mask on

A flexible LED display connected to a microprocessor with a phone and a laptop and a small lipstick battery on a desk.

Why would we want this?

An LED display inside a wearable mask

What is it?

A flexible display attached to an Adafruit Feather Huzzah connected to a USB battery and a phone displaying the app

How Does it work?

Using the microphone

Processing the data stream

How the data is sent

Processing text commands

const textMessage = {
value: "My line of text",
mode: 1,
scrollSpeedMs: 25,
color: { r: 255, g: 255, b: 255 }
}

Displaying the results

Creating a pixel “font”

the pixel font as a png
the letter n as a pixel font
the letter n with its binary values
[0,0,0,0,0,0,0,0,1,0,1,0,1,1,0,1,1,0,0,1,1,0,0,1,1,0,0,1,0,0,0,0]

Hardware differences

// Display config
const int display_gpio_pin = 4;
const int display_width = 32;
const int display_height = 8;
const index_mode display_connector_location = index_mode::TOP_LEFT;
const carriage_return_mode line_wrap = carriage_return_mode::SNAKED_VERTICALLY;
const neoPixelType neopixel_type = NEO_GRB + NEO_KHZ800;

Turning the pixel font into scrolling text

How pixels scroll across the display

Testing the hardware

A screenshot of the virtual simulated display in the app

The Remote LED Matrix Driver

A sequence diagram of the app and hardware

What if I don’t want to wear a display?

Show me the code

Where can we take this project next?

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store