New HomePod Testing: Siri Satisfies 52.3% of Requests

Consulting company Loup Ventures conducted a new test that explored the possibilities

Apple HomePod speaker in categories such asSiri performance, sound quality and ease of use. Regarding Siri, Loup Ventures analyst Jim Munster reports that despite the fact that the voice assistant understood 99.4% of requests, it was only able to give the correct answer to 52.3% of them. Loup Ventures tested 3 separate HomePod speakers, asking a total of 782 questions.

Unlike the results of previous comparisonstest results with competitors' products, this time HomePod falls to the very bottom of the list of voice assistant performance. Thus, Google Home was able to satisfy 81% of requests, Amazon's Alexa speaker was able to provide 64% of correct answers, and Microsoft's Cotrana answered 57% of requests correctly.

Advertising

Munster subsequently relented in conclusions,noting that Siri's assistant is good for queries of a "local" and "commercial" nature - for example, finding nearby coffee shops or helping to buy new shoes. Siri is superior to Alexa and Cortana in these categories, but still inferior to Google Assistant on Google Home.

Despite the low percentage of correct answers to 782request, Munster noted that overall Siri performance has improved more than might be expected given the HomePod's "narrow music focus."

Advertising

Advertising

The researchers explained that over time, AppleHomePod and Siri will need to be improved to catch up or even surpass the competition by simply adding query domains such as calendar, email, calling and navigation.

Advertising

Advertising

Domains like navigation, calendar,Email is simply not supported. In response to such requests, we hear “I can’t ___ on HomePod” (“I can not ___ on HomePod”). Also, in the case of requests in response to which Siri on the iPhone results in Google search results, Homepod replies that it can’t get an answer, which forces you to use the phone or refuse the request altogether. Removing navigation, calls, email, and calendar-related requests from the list increases the proportion of correct answers to 67% - a big jump compared to 52.3%, the total share of correct answers to requests.

This means adding support for thesedomains can allow HomePods to outperform Alexa (64%) and Cortana (57%), but not Google Home (81%). We know that Siri can correctly answer a number of requests that HomePod cannot satisfy, as evidenced by the above. Apple should expand the query categories for HomePod, and then, in our opinion, the speaker will become more useful and integrate better with other Apple devices.

The study found that HomePodsignificantly superior to competitors in the area of ​​listening ability. HomePod allows the user to speak at normal volume even while playing loud enough music - Siri will still hear the request. “This is HomePod’s most stellar feature,” Munster said.

Advertising

Advertising

Loup Ventures also praised the sound qualityHomePod, reporting that "the speaker sounds incredible." As with other reviews of the speaker, Munster's satisfaction with the sound quality is contrasted by the poor performance of the speaker's built-in Siri assistant. According to the analyst, the situation will soon change.

Do not be deceived by the fact that only those who have entered the worldHomePod smart speakers focus on sound quality; Apple certainly has a grander vision than just improving sound. It may not be present in the first version of HomePod (meaning you can't even make a call with it yet), but we're confident that Apple's goal is to expand Siri into an "absolutely present" experience, connecting all your connected devices and services, making a breakthrough towards the transition to voice control.

You can find even more on the Loup Ventures websiteinformation on the “smart speaker challenge,” including forecasting demand and market share HomePod could capture for the remainder of 2018.