They’re Listening: A Paranoid Guide to Smart-Speaker Privacy

By

Scrambling to deflate a privacy scandal, Facebook has quietly shelved an Amazon Echo–like home speaker it had been planning to release in May, Bloomberg reported last month. The company decided that now might not be the right time to ask to be invited into your living room. The unsaid reasoning: It’s one thing to give your data up to a tech company; it’s another to let one of its devices listen in on you all the time.

For now, three other companies’ home speakers rule the market. None of them is free of privacy pitfalls, either — that’s a given when you bring home an internet-connected microphone attached to a computer and a speaker — but one of the three does stand out when it comes to privacy and security.

Understanding why that is, and how the three companies’ home speakers deal with your data differently, requires taking a close look at what happens to a spoken request, from the moment it leaves your lips to the instant it’s deleted from a faraway server — if it’s ever deleted at all. It also demands a little bit of business voodoo to guess at what the companies are eventually going to do with all that data they’re collecting.

Are you sure you want to put that thing in your house?

Last August, Wired reported on a security researcher who found a way to turn pre-2017 Amazon Echo devices into an always-on bug that sends everything it hears to a remote server of the hacker’s choosing. The attack requires the hacker to handle the device in person — it can’t be done remotely — and while new devices aren’t susceptible to the attack, old ones are still vulnerable.

And in October, Artem Russakovskii, a writer for the website Android Police, was given a defective preview version of a Google Home Mini that activated itself thousands of times a day, recording what it heard and sending it to Google for processing. Google said the devices it sold to the general public did not have the same defect.

Both of these problems have been resolved, but they’re a reminder that device-makers aren’t infallible. Anything connected to the internet can — and probably will — be hacked at some point. If it’s just a gimmicky “smart” coffee maker, maybe the worst that’ll happen is that a hacker will scorch your morning brew. But a home speaker, with its internet-connected microphone, is much more tempting. If its manufacturer makes even the smallest security mistake, someone will probably find and exploit it.

Google Home and Amazon Echo

Of the three main companies that offer home speakers, Google probably knows the most about its users. If you use Google’s services — Search, Gmail, Maps, and Calendar — regularly with default privacy settings, Google knows where you live and work; who you email, chat, and meet up with; and what you like to search for, among other things.

This in-depth knowledge makes Google’s Assistant really useful, allowing it to dip into the information it has on file to complete your requests. (Amazon’s Alexa, too, can connect to Google services like Calendar to gain similar capabilities.) But the convenience comes with a privacy trade-off.

Google and Amazon keep a running tab of everything you’ve asked your home speaker. When you interact with a speaker by saying its wake word — “Okay, Google” or “Alexa” — and asking it a question, it sends a recording of your voice to its servers for processing. All three of the companies encrypt this recording, both in transit (as it moves through the internet), and at rest (on a server rack somewhere). Google and Amazon tie your recordings to your identity, but Apple doesn’t — more on this later.

What this means is that all of Amazon’s and Google’s data about you is neatly tied up with a bow, all associated with your user account. See for yourself: Google’s “My Activity” page lets you play back recordings of everything you’ve ever said to Google Assistant; here’s how to access Amazon’s equivalent. You can delete the recordings, which the companies would otherwise keep forever, but they warn that doing so might make their devices worse at understanding you.

This trove of data is what makes Google’s and Amazon’s speakers into useful tools. But it also means that someone looking for a lot of information about a person — like the authorities, for example — can find it all in one place.

Last year, a prosecutor in Arkansas asked Amazon for access to voice recordings from an Amazon Echo that belonged to a murder suspect. Only commands that follow Echo’s wake phrase, “Alexa,” are sent to Amazon’s servers, but the devices are known for accidentally waking up when they hear something that sounds like the wake phrase (like, say, “A Tesla!”). The company originally pushed back against the prosecutors’ request, but the defendant eventually gave his permission for Amazon to turn over the recordings. The Echo doesn’t appear to have caught any incriminating recordings: The charges were eventually dropped.

But it’s not just the speakers’ manufacturers that gather information when you ask them for things. To get the most out of an Echo or a Google Home, you’ll probably want to connect it with other companies’ services, using what Amazon calls “skills” and Google calls “actions.” This is how you do anything outside of each company’s own ecosystem, like order an Uber or a pizza.

Like always, though, the price of that convenience is data. A Wall Street Journal investigation found that although technology makes it extremely convenient to meet up with a friend, order a pizza, and watch a movie at home, it’s not without its cost to privacy: In the scenario, the pair of friends gave up 53 pieces of personal information to various companies throughout the evening, 38 of which were collected by companies in the background.

Connecting a home speaker to third-party extensions is also potentially a recipe for abuse. It was a third-party quiz app that vacuumed up Facebook users’ personal data — and that of their friends — and shared it with a researcher associated with Cambridge Analytica. There’s no reason an unscrupulous developer couldn’t come up with a similarly invasive add-on for a home speaker. Both Google and Amazon allow developers to create extensions for their home speakers, but the Echo, having been around longer, has more plug-ins.

Apple HomePod

Apple is the odd one out in this trio: Its HomePod offers the most privacy of any home speaker — but at the cost of convenience. Besides using the HomePod to control Apple’s software or as a hub for an automated home, you can ask about the news, weather, or traffic — but not much else. You can’t install extensions the way you can on an Echo or a Google Home, so Apple has complete control over what data goes where.

But the biggest privacy difference between the HomePod and its competitors isn’t what it can or can’t do — it’s how the HomePod interacts with Apple’s servers. Like the other speakers, when a HomePod hears a request, it sends it off to Apple to parse and fulfill it. But instead of associating the request with the user’s account, like Google and Amazon do, HomePod requests are anonymous, tied only to a random, rotating ID. Just like a request you might make of Siri on an iPhone, HomePod requests will live on Apple’s servers for six months, associated with that ID, and then another year and a half, unlinked to any ID at all. By contrast, Google and Amazon only delete requests from their servers when asked by the user.

In the few months it’s been out, people have complained about one particular privacy shortfall of the HomePod. The HomePod can fulfill “personal requests,” like reading out and sending texts, or reading and creating notes. For someone who lives alone — or has no secrets — this might be useful. But otherwise, as long as the primary user is at home, anyone can walk up to the device and ask it to send an embarrassing text to mom, and it will. Unlike the Echo or the Google Home, HomePod can’t differentiate between people’s voices, so anyone’s request will go through.

But that’s a relatively small privacy gripe. Generally, if you value privacy (and sound quality) over omniscient assistance, Apple’s HomePod should be your go-to. Siri is leagues behind its competitors, but at least it doesn’t tattle.

Ads on your home speaker?

If home speakers were like toasters — something you buy once that does the same thing forever until it breaks — this article would be over now. But people expect their home speakers to develop new features over time, and the companies that create them have the ability to change how they work at any point.

In fact, that’s why Bruce Schneier, a security researcher, has a very simple approach to home speakers: “Since their terms and conditions are designed not to tell you what’s really going on, and they can change without your knowledge or consent, I wouldn’t trust any of them,” he told me.

If you, however, are not a world-renowned computer expert, and you decide the convenience of a home speaker outweighs the privacy risks, then it’s worth thinking about the companies’ motives to try to guess where they’ll go next. While none of them like to talk about what they plan to do with your data in the future, we can hazard some guesses based on what we know about the companies.

Google’s parent company, Alphabet, makes 84 percent of its revenue just off of advertising. So it wasn’t too surprising when, last year, some Google Homes spouted off what sounded suspiciously like an advertisement for the new Beauty and the Beast film. Google denied that the segment, which played after Google Assistant had summarized the user’s agenda for the day, was an ad. (It was “timely content,” the company said at the time.) A spokesperson for Google said that while Google Assistant doesn’t currently have ads, it probably will in the future. Google “may use transcriptions (not raw audio) of these interactions to deliver more useful ads on other platforms,” the spokesperson said.

Amazon’s business model is less dependent on advertising, but in January, CNBC reported that Amazon is looking to dive into advertising on its Alexa-powered devices. Amazon denied the report at the time, and a spokesperson told me that the company has “no plans to add advertisements to Alexa and [does] not use voice recordings for product recommendations.”

Apple, for its part, likes to say that it’s not interested in its users’ data the way its competitors are — instead, its main business is selling devices and cloud services.

None of the three smart speaker companies is a complete privacy disaster, but none is perfect, either. If privacy and security are your priority and you’re willing to pay $349 for a speaker, HomePod should be your choice. If you’re willing to give up more information about yourself to companies (which likely already know a lot about you) in return for a smarter AI assistant, then the Echo or the Home is best, depending on which company’s services you interact with more. Or you can go the secure route and keep the devices out of your home entirely. You’ll laugh when they all get hacked.

How Much Should You Trust Your Smart Speaker?