Who’s smartest — Alexa, Siri, Cortana, or Google Assistant?

When it comes to knowing stuff, Google Assistant on Google Home rules the virtual assistant world, according to results of a study conducted by Stone Temple Consulting earlier this year and released today.

Microsoft’s Cortana was the second smartest, and Apple’s Siri and Amazon’s Alexa brought up the rear.

Though Google ruled, “Cortana is pressing quite hard to close the gap, and has made great strides in the last three years” in terms of smartness, Stone Temple says. Unlike Google, “Alexa and Siri both face the limitation of not being able to leverage a full crawl of the web to supplement their knowledge bases. It will be interesting to see how they both address that challenge.”

Derek Walter

Let’s break down the research.

How the study was performed

Stone Temple asked each virtual assistant 5,000 different questions and compared the answers to Google search results. The assistants were ranked based on five criteria:

* If the assistant answered a question verbally;

* If an answer came from a database, such as Google’s Knowledge Graph;

* If an answer came from a third-party source, such as Wikipedia;

* How often a query stumped the assistant;

* If the device tried to respond to a query but simply got the answer wrong.

Stone Temple didn’t ask the virtual assistant devices to perform an action, such as hailing a Lyft or ordering a Starbucks drink. The goal was simply to determine which “was the smartest from a knowledge perspective.”

The research results

The Google Assistant on Google Home answered 68.1 percent of the questions, with 90.6 percent of answers being “complete and correct.”

Cortana answered 56.5 percent of questions and got 81.9 percent complete and correct.

But Siri only answered 21.7 percent of questions and nailed 62.2 percent of them completely, correctly.

In last place for percentage of questions answered was Alexa on Amazon Echo, answering just 20.7 percent of questions. However, of the queries Alexa answered, 87 percent were complete and correct, just behind Google Assistant/Google Home.

Not surprisingly, Google search answered nearly 80 percent of the queries, and got nearly 100 percent of them correct and complete.

So, what caused variations in how correct and complete an answer was? Five things:

* There’s more than one way to answer the query. Example: The question “How fast does a Jaguar go” could refer to an animal or a luxury vehicle.

* The assistant may answer a query it doesn’t understand with an answer that’s reasonably close, but no cigar. Example: When Stone Temple asked Siri about “awards for Louis Armstrong,” Apple’s virtual assistant answered with a link to a movie about the beloved musician. Similar responses “accounted for a large number of the ‘not 100% correct’ scenarios on Siri,” according to Stone Pilot.

* The assistant may have provided a partially correct response.

* When it didn’t know the answer, the assistant may have responded with a joke.

* Last possibility: The assistant was just dead wrong in its answer.

Which virtual assistant is the funniest?

Siri answered the most questions with a joke, followed by Alexa, Google Assistant on Google Home, and Cortana.

About Stone Temple

In case you’re wondering, Stone Temple is a well-regarded digital marketing and SEO firm. The company often conducts useful research studies, and its blog is a must-read for digital marketers.

What does it all mean?

The research could be most helpful to those weighing Amazon Echo vs. Google Home, either for home or work. If you want the best chance of getting a useful piece of information from your smart speaker, Google Home is the better option.

Nonetheless, I prefer Amazon Echo devices—which are built into an increasing variety of hardware products from Amazon as well as others. For example, I had lots of headaches setting up Google Home on my home Wi-Fi network—so many that I took the device back to Best Buy, where I purchased it. Echo devices, on the other hand, have worked mostly without a hitch.