Cognilytica’s Latest Voice Assistant Benchmark Produces Surprising Results
Voice-enabled devices of all sorts are increasingly finding their way into our daily lives. Many large vendors are releasing their own versions of voice assistants to act as general purpose helpful devices. From assisting with answers in the kitchen, to the office, or helping power a number of IoT devices the use cases for voice assistants are ever growing. But, just how intelligent are these systems?
In an authoritative benchmark, Cognilytica put four popular voice assistants to the test to find out just how just how intelligent is the AI back-end. In this second version we tested Amazon Alexa, Google Assistant (Home), Apple Siri, and Microsoft Cortana. The benchmark measures conversational system intelligence by asking 120 questions grouped into 12 categories of various levels of cognitive challenge.
Key Findings from the benchmark include:
- Voice assistants have a long way to go before even half of the responses are acceptable.
- For the current benchmark, Alexa provided the most number of adequate responses, with 34.7% of responses determined to be adequate while Google followed close behind with 34.0% adequate. Cortana showed much improvement from the previous benchmark with 31.9% adequate responses while Apple’s Siri still trails with 24.3% of answers determined to be adequate.
A full copy of the report and our findings can be found at: https://www.cognilytica.com/2019/09/05/report-voice-assistant-benchmark-2-0-2019/
If you’re currently a subscribing client to Cognilytica research you can access this report for free. If you’re not yet a client you can purchase the report for $995.
