Principal Components: Leveraging AI to Unlock Unstructured Data

Video, audio, images and text can all be rich sources of insights, but they can also be challenging to analyze. New tools offer easier access to the information within them.

Susan Currie Sivek, Ph.D.
May 6 · 4 min read
Nick Fewings on Unsplash

You know how on TV crime shows, it seems magically possible to extract clear, easy-to-read information from even super-fuzzy images or video? We’re getting ever closer to that level of understanding for even difficult forms of data. This week on the Data Science Mixer podcast, we talked with Trevor Jones, vice president of business operations, and Robbie Booth, senior director of AI cognitive engines, at Veritone, whose aiWARE offers “layered cognition” for processing audio, image, and video data.

Here are three “principal components” of our conversation that offer new ways to think about and analyze your unstructured data.

Recordings of interactions provide multiple layers of potential insights for rich analysis.

One of the use cases that we really see emerging is around interaction analytics. And what that really means is, if you have a conversation between people, how can I query those recordings? Some of the applications could be sales calls for sales enablement training, or asking whether my sales reps are being effective. It could include telemedicine: Are my doctors providing a good experience? Or a call center: Are my call center agents complying with our policies, and are they providing a good experience? Either using our platform natively, or working with Alteryx as well, you get dashboards that show you: What are they talking about? What’s the sentiment? There are computer vision aspects to figure out: What are the facial expressions? Who are the people on the call, using facial recognition? There’s content, entity extraction, and topic extraction. We get to use such a diverse amount of AI engines together in one solution.

Computer vision tools can be used for a huge variety of applications, from power to advertising.

Trevor: We work with advertisers that place media on live sporting events. You don’t always know where the cameras are going to be or which way they’re facing. You want to try to measure the ad value of your sponsorship. We ingest the media for the sporting event, and we run it through logo detection. It identifies where it is on the screen, what angle it’s at, and how prominent it is on the screen. We compile an aggregate index that essentially equates that to value, and then we match that against [other metrics], whether it’s Google Analytics impressions or orders. We attribute that back to the actual impression.

Different kinds of data science tools will continue to grow and empower varied users to respond to data questions.

By the end of my tenure in the video game space, you license your engine, and the bulk of your team are really working in that script layer or the drag-and-drop. So all of a sudden, your team doesn’t have 150 engineers and a couple of token artists. You basically have designers. You have people who are creating the experience. You’re empowering creatives. I think the goal of our tools and other tools like it, which are inevitably going to pop up, is to be the way that you can go run any kind of cognition without having to write any code to support it. You can do so in the cloud and pick what your environment looks like, so that if you need to run it on a GPU, you can, and just make it super simple. I think it’s really exciting what things are going to look like in five years’ time.

Alternative Hypothesis

What were Trevor and Robbie’s alternative hypotheses? Check out the episode to find out.

These interview responses have been lightly edited for length and clarity.

The podcast show notes and a full transcript are available on the Alteryx Community.

Geek Culture

Proud to geek out. Follow to join our +500K monthly readers.

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store