Late last year, San Francisco face-recognition startup Everalbum won a $2 million contract with the Air Force to provide “AI-driven access control.” Monday, another arm of the US government dealt the company a setback.
The Federal Trade Commission said Everalbum had agreed to settle charges that it had applied face-recognition technology to images uploaded to a photo app without users’ permission and retained them after telling users they would be deleted. The startup used millions of the photos to develop technology offered to government agencies and other customers under the brand Paravision.
Paravision, as the company is now known, agreed to delete the data collected inappropriately. But it also agreed to a more novel remedy: purging any algorithms developed with those photos.
The settlement throws a shadow on Paravision’s reputation, but chief product officer Joey Pritikin says the company can still fulfill its Air Force contract and obligations to other clients. The startup shut the consumer app in August, the same month it learned of a potential FTC complaint, and it launched face-recognition technology developed without data from the app in September. Pritikin says those changes were in motion before the FTC came knocking, in part due to “evolution in public sentiment” about face recognition.
FTC commissioner Rohit Chopra, a Democrat, released a statement Monday praising the commission’s thoroughness with Paravision, saying it had been rightly forced to “forfeit the fruits of its deception.”
He contrasted the settlement with a 2019 agreement in which Google paid $170 million for illegally collecting data from children without parental consent. The company was not required to delete anything derived from that data. “Commissioners have previously voted to allow data protection law violators to retain algorithms and technologies that derive much of their value from ill-gotten data,” he wrote. “This is an important course correction.”
Supersmart algorithms won’t take all the jobs, But they are learning faster than ever, doing everything from medical diagnostics to serving up ads.
Ryan Calo, a law professor at the University of Washington, says requiring Paravision to delete face-recognition algorithms trained with allegedly ill-gotten images shows the FTC recognizing how the rise of machine learning has tightly entwined data sets and potentially harmful software products.
Tech companies once created software solely by paying people to tap the right keys in the right order. But for many products such as face-recognition models or video filtering software, one of the most crucial ingredients is now a carefully curated collection of example data to feed into machine-learning algorithms. “This idea you have to delete the model and the data is acknowledgment those things are closely linked,” Calo says. Face-recognition systems deserve special scrutiny because creating them requires highly personal images. “They’re like Soylent Green—made out of people.”
David Vladeck, a former director of the FTC’s Bureau of Consumer Protection and a law professor at Georgetown, says Monday’s settlement is consistent with prior ones that required deletion of data. In 2013, software company DesignerWare and seven rent-to-own retailers agreed to delete geotracking data gathered without consent from spyware installed on laptops.
Monday’s more expansive deletion requirement with Paravision was approved unanimously, 5-0, by the FTC, which is still controlled by a Republican majority. After president-elect Joe Biden’s inauguration this month, the commission could become majority Democrat, and potentially even more eager to police tech companies. It could get new support and resources from the Democrat-controlled Congress.
Calo hopes to see the agency get more technical resources and expertise to help it scrutinize the tech industry on a more equal footing. One use for more tech know-how could be to devise ways to check whether a company really has scrubbed not just ill-gotten data but also advantages or tech derived from it. That could be difficult to do for systems involving complex machine-learning models built from multiple sources of data.
Vladeck says expanding the FTC’s core powers and headcount is yet more important. The agency may gain considerable new legal powers if Democrats pass the first federal US privacy law, as some, like US senator Maria Cantwell (D-Washington), have proposed. He’s optimistic about such legislation, if it also comes with a lot more people. The FTC has typically had just a few dozen people working on privacy and identity theft, Vladeck, says, while much smaller nations, such as Ireland, have hundreds. “The FTC is two-thirds the size it was 40 years ago,” he says.
More Great WIRED Stories
📩 Want the latest on tech, science, and more? Sign up for our newsletters!
Yes, Cyberpunk 2077 is buggy. But mostly, it has no heart
A race car crash from hell—and how the driver walked away
Real Life. Real News. Real Action
Zillion Things Mobile!Read More-Visit US
Apple’s app “privacy labels” are a big step forward
These 7 pots and pans are all you need in the kitchen
The race for a Covid vaccine was more about luck than tech
🎮 WIRED Games: Get the latest tips, reviews, and more
Subscribe to the newsletter news
We hate SPAM and promise to keep your email address safe