Principal Components: Leveraging AI to Unlock Unstructured Data
Video, audio, images and text can all be rich sources of insights, but they can also be challenging to analyze. New tools offer easier access to the information within them.
You know how on TV crime shows, it seems magically possible to extract clear, easy-to-read information from even super-fuzzy images or video? We’re getting ever closer to that level of understanding for even difficult forms of data. This week on the Data Science Mixer podcast, we talked with Trevor Jones, vice president of business operations, and Robbie Booth, senior director of AI cognitive engines, at Veritone, whose aiWARE offers “layered cognition” for processing audio, image, and video data.
Here are three “principal components” of our conversation that offer new ways to think about and analyze your unstructured data.
Recordings of interactions provide multiple layers of potential insights for rich analysis.
Trevor: One of the use cases we’ve been exploring recently is around interaction analytics. These days, there’s so much content being generated through Zoom and Hangouts and all these things, especially since COVID. It’s a lot of data that’s just not being tapped.
One of the use cases that we really see emerging is around interaction analytics. And what that really means is, if you have a conversation between people, how can I query those recordings? Some of the applications could be sales calls for sales enablement training, or asking whether my sales reps are being effective. It could include telemedicine: Are my doctors providing a good experience? Or a call center: Are my call center agents complying with our policies, and are they providing a good experience? Either using our platform natively, or working with Alteryx as well, you get dashboards that show you: What are they talking about? What’s the sentiment? There are computer vision aspects to figure out: What are the facial expressions? Who are the people on the call, using facial recognition? There’s content, entity extraction, and topic extraction. We get to use such a diverse amount of AI engines together in one solution.
Computer vision tools can be used for a huge variety of applications, from power to advertising.
Robbie: In the power industry, they’re using drones — like the DJI drones, for example, that have a NVIDIA Jetson device onboard — and really powerful computer vision and object detection techniques and models. You can use them to check power lines after a storm so that crews don’t have to manually climb each pole to verify that it’s okay. Those types of scenarios — being able to take a whole ton of visual data, and run an analysis on which ones have the anomalies that need to be manually checked by a human — can just save tons and tons of time.
Trevor: We work with advertisers that place media on live sporting events. You don’t always know where the cameras are going to be or which way they’re facing. You want to try to measure the ad value of your sponsorship. We ingest the media for the sporting event, and we run it through logo detection. It identifies where it is on the screen, what angle it’s at, and how prominent it is on the screen. We compile an aggregate index that essentially equates that to value, and then we match that against [other metrics], whether it’s Google Analytics impressions or orders. We attribute that back to the actual impression.
Different kinds of data science tools will continue to grow and empower varied users to respond to data questions.
Robbie: Look at other industries where people have empowered scientists or just other folks with tools that don’t require a high-level entry point. If you think of video game development and the rise of the game engine, like Unity, Unreal Engine, and things like that, it’s really amazing what the difference has been. I got my start in the video game business, and we’d have 150 people, and it’d take us five years. It was painful. Then we started playing around with the concepts of a common engine.
By the end of my tenure in the video game space, you license your engine, and the bulk of your team are really working in that script layer or the drag-and-drop. So all of a sudden, your team doesn’t have 150 engineers and a couple of token artists. You basically have designers. You have people who are creating the experience. You’re empowering creatives. I think the goal of our tools and other tools like it, which are inevitably going to pop up, is to be the way that you can go run any kind of cognition without having to write any code to support it. You can do so in the cloud and pick what your environment looks like, so that if you need to run it on a GPU, you can, and just make it super simple. I think it’s really exciting what things are going to look like in five years’ time.
On Data Science Mixer, we always ask guests our “Alternative Hypothesis” question: What’s something that people often think is true about data science or about being a data scientist, but you’ve found to be incorrect?
What were Trevor and Robbie’s alternative hypotheses? Check out the episode to find out.
These interview responses have been lightly edited for length and clarity.
The podcast show notes and a full transcript are available on the Alteryx Community.