Google Voice Search Beats Apple’s Siri

When Siri was announced, it was the highlight of a feature that made the iPhone 4S a much desirable phone. The most obvious contender to Siri is Google Voice Search.

Now a test by Piper Jaffray analyst Gene Munster has found out that Google Voice Search smokes Siri, so much so in fact that he says Siri may need more than two years to become a viable contender to the Google-made feature.

In a note to clients on Thursday, Munster wrote about a test he and his team conducted. They asked Siri and Google Voice search 1,600 questions each, half of them while on a noisy Minneapolis street and half of them while in a quiet room.

Here’s what Munster and his team wrote:

COMPREHENSION

We measure Siri comprehension at 83%. On a noisy Minneapolis street, Siri comprehended 83% of our queries. We believe this is the most accurate representation of Siri’s comprehension because Siri is rarely used in a perfectly quiet setting. As a point of comparison, we performed the test in a quiet room where Siri was able to comprehend 89% of queries. Siri will need to improve from a B in comprehension to at least an A if it is to be considered a viable alternative to Google (A+), which could take two or more years.

ACCURACY

We measure Siri accuracy at 62%. Siri accurately answer 62% on the street and 68% in a quiet room. (See page 2 for difference between comprehension and accuracy). We believe accuracy is where Siri needs the most improvement if it wants to rival Google. Currently, we measure Google at 86% accuracy in the US based on comScore result page per search data. According to comScore, Google delivered an average of 1.14 search results pages per search over the past year. We believe this is a relevant proxy suggesting that 14% of the time, users could not immediately find that for which they were looking, thus 86% of the time, Google was accurate. In order to become a viable mobile search alternative, Siri must match or surpass Google’s accuracy of B+ and move from a grade D to a B or higher.

However, Munster predicts Siri is just going to get better.

Google provides 60% of answers today, 48% when iOS 6 ships. We estimate that changes to Siri iOS 6 will decrease dependency on Google by 12%, from 60% to 48%. Apple has made several significant changes to Siri on iOS 6 that will decrease Siri’s dependency on Google. First and foremost, Apple’s in-house maps app will eliminate Siri’s dependency on Google for navigation. Second, Siri’s new integration of Yahoo Sports, Open Table, Rotten Tomatoes, and Fandango will provide answers for sports scores and statistics, restaurant reservations, movie show times, and ticket purchases.

As for the questions Siri was stumped on, here’s a sampling:

What team does Peyton Manning play for? Siri answered the previous question asked before this query.

Where is Elvis buried? Siri went looking for “Elvis Buried” the man and not where Elvis the superstar is buried.

Where am I? Siri on occasion dropped a pin in the wrong place.

When did the movie Cinderella come out? Siri queried Yelp for screening times of Cinderella.

How do I get from Boston to New York? Siri answered: “I can only give directions from your current location. I can’t give you directions to a place you are not in.”

What spices are in Lasagna? Siri searched Yelp for places where Lasagna is on the menu.

When is the next Haley’s comet? Siri answered: “You have no meetings matching Haley’s.”

I want to go to Lake Superior? Siri displays navigation to a company called Lake Superior X-Ray.

What do you think of this new test and result? Do you think Siri is really beaten by Google Voice Search? Tell us in the comments below.

Images 1 & 2 from AshtonPal & Johan Larsson on Flickr (CC)

[cb]Apple[/cb]

[cb]Google[/cb]