facial recognition

Smile! The Secretive Business of Facial-Recognition Software in Retail Stores

Photo-Illustration: Konstantin Sergeyev/Intelligencer; Photo: Joe Raedle/Getty Images

Have you been to Madison Square Garden recently? Or Kennedy International Airport? If so, there’s a good chance that you were photographed by a security camera. Depending on where you were and the technology being used, your face may have been analyzed by bots and checked against a database of criminals and “known shoplifters.” It doesn’t need to have been MSG, either: For all we know, the same technology could also be in use at dozens of other places around New York City, from department stores to megachurches, but no one can say for sure except the companies supplying the software.

Facial-recognition software, which has been in development since the 1960s and has been gaining popularity with police for more than a decade, has taken off with retailers and event spaces during the last couple of years, consultants say. It’s marketed to them as an unparalleled tool for cutting down on shoplifting, and sold to the public as a security tool — helping identify would-be terrorists at sports games, for instance, or protecting consumers against identity theft by making sure that they are who they say they are. It’s also almost completely unregulated.

“The technology is in some environments where I’m sure millions of people, in a year, or even in a month, are subjected to it,” said Donna Lieberman, executive director of the New York Civil Liberties Union. “Nobody has any idea that it’s happening, or what data is being collected, or how it’s being stored, or for how long, or who has access to it.”

On Wednesday, New York City Councilman Ritchie Torres, who represents the Bronx, introduced a bill aimed at changing this. It would require businesses to start telling the public if they’re using facial recognition, how long they’re storing it, and who they’re sharing it with. Torres said he was inspired to put the bill out after he learned about Madison Square Garden’s use of facial recognition in the spring.

“It’s both a no-brainer and a small measure — a baby step,” Lieberman said. “I think it’s necessary but not sufficient.”

Adrian Weidmann, a retail consultant based in Minneapolis, said most stores, from bodegas to shopping malls, already have most of the technology in place to start tracking customers: not just the scores of security cameras in an average big-box store, but also the cameras inside digital signs and kiosks, which show whether shoppers are paying attention to ads. “It’s the same camera lens,” Weidmann said. All it takes to upgrade is a piece of software. The software often comes with a database of criminals or known shoplifters, which comes from combining the shoplifter registries of participating stores, said Clare Garvie, who studies the technology and its privacy implications at Georgetown Law. It’s unclear exactly what it takes to be put in these databases, let alone how to get your name removed.

Peter Trepp, CEO of the facial-recognition software company FaceFirst, told BuzzFeed in August that retail now accounted for nearly half of his company’s business. “If you think about the top 40 or top 80 companies you know, almost all of them are thinking about facial recognition, or they’ve all at least looked into it,” he said. (FaceFirst declined to make Trepp or any other representatives available for this story.) Trepp also told the Sports Network, a Canadian publication, last year that his company had been marketing to sports stadiums and teams and was “very much in play” with a number of them, but wouldn’t give details.

The retailers and venues themselves are almost universally secretive about what they’re doing. The American Civil Liberties Union, of which Lieberman’s organization is a regional branch, polled the country’s 20 largest retailers earlier this year — substituting Disney for Amazon, since Amazon doesn’t have many physical stores — about whether they routinely photograph customers. Only one, the supermarket conglomerate that owns Food Lion and Giant, said it did not.

Walmart has admitted that it tested the technology in 2015 but decided it wasn’t profitable enough. Target also told BuzzFeed that it had tested it but wouldn’t say where or when. And in 2016, Saks Fifth Avenue told The Guardian that it was using the technology in its Canadian stores.

Torres’s bill, if it gains any traction, is likely to come up against a serious lobbying effort by the tech industry. Several states, including Alaska, Connecticut, Montana, New Hampshire, and Washington, have considered similar privacy laws within the last couple of years, but none but Washington have followed through. (Illinois and Texas also have long-standing privacy laws in place.) Trade groups and companies like Facebook and Google have come out in full force. Facebook has been especially aggressive, according to a report from the Center for Public Integrity.

The interest of Google and Facebook in unregulated facial-recognition systems in retail settings may not be entirely about their concern for the security of brick-and-mortar stores. In the future, facial-recognition technology could also be used for marketing, helping stores track customers in real life the way online retailers track them with cookies. There’s no evidence that this is happening so far, but Weidmann, the consultant, said that he’s already seeing companies merge their security and marketing divisions. On a project with Home Depot last year, for instance, he helped the marketing team use security footage to track customers through stores and figure out what products they were browsing. “Nobody had ever thought about linking the two,” he said. “I was able to get a much wider lens.”

He believes that Amazon is leading the way for all major retailers, brick-and-mortar shops included. “They’re all in a panic,” he said. “How do we bring relevance back to our brick-and-mortar?” Facial recognition, he said, along with location tracking on smartphones, could make the “physical cookie” possible.

This story previously cited an ACLU report from earlier this year that said the Lowe’s chain of home-improvement stores had self-reported that it uses the technology. A spokeswoman from Lowe’s clarified that the stores tested the technology four years ago and decided against using it in any way. Its privacy policy now reflects this.

The Secretive Business of Commercial Facial Recognition