The errors emerged as part of a test by the ACLU that compared the photos of all federal lawmakers against a database of 25,000 publicly available mug shots.
Rep. John Lewis of Georgia and Rep. Bobby L. Rush of Illinois are both Democrats, members of the Congressional Black Caucus and civil rights leaders.
But facial recognition technology made by Amazon, which is being used by some police departments and other organizations, incorrectly matched the lawmakers with people who had been charged with a crime, the American Civil Liberties Union reported Thursday.
The errors emerged as part of a larger test in which the civil liberties group used Amazon’s facial software to compare the photos of all federal lawmakers against a database of 25,000 publicly available mug shots. In the test, the Amazon technology incorrectly matched 28 members of Congress with people who had been arrested, amounting to a 5 percent error rate among legislators. The test disproportionally misidentified African-American and Latino members of Congress as the people in mug shots.
“This test confirms that facial recognition is flawed, biased and dangerous,” said Jacob Snow, a technology and civil liberties lawyer with the ACLU of Northern California.
Most Read Business Stories
- New questions emerge around REI CEO's undisclosed relationship
- Fired Amazon employee with Crohn's disease files lawsuit over lack of bathroom access
- Is your phone always low on battery and chewing through data? 'DrainerBot' could be to blame, Oracle says.
- Rich Barton returning as Zillow CEO as Spencer Rascoff steps down
- Tesla Model 3 loses coveted Consumer Reports recommendation
On Thursday afternoon, three of the misidentified legislators — Sen. Edward J. Markey of Massachusetts, Rep. Luis V. Gutiérrez of Illinois and Rep. Mark DeSaulnier of California, all Democrats — followed up with a letter to Jeff Bezos, chief executive of Amazon, saying there were “serious questions regarding whether Amazon should be selling its technology to law enforcement at this time.”
In the letter, the lawmakers asked for details on how Amazon tested its facial technology for accuracy and bias. They also requested a list of all government agencies using Amazon’s facial technology as well as all law enforcement and intelligence agencies Amazon had communicated with about the system.
Separately, two other Congressmen wrongly matched with mug shots — Lewis and Rep. Jimmy Gomez, D-Calif. — wrote their own letter to Bezos requesting an immediate meeting “to discuss how to address the defects of this technology.” The letter was first obtained by BuzzFeed.
Nina Lindsey, an Amazon Web Services spokeswoman, said in a statement that the company’s customers had used its facial recognition technology for various beneficial purposes, including preventing human trafficking and reuniting missing children with their families. She added that the ACLU had used the company’s face-matching technology, called Amazon Rekognition, differently during its test than the company recommended for law enforcement customers.
For one thing, she said, police departments do not typically use the software to make fully autonomous decisions about people’s identities. “It is worth noting that in real world scenarios, Amazon Rekognition is almost exclusively used to help narrow the field and allow humans to expeditiously review and consider options using their judgment,” Lindsey said in the statement.
She also noted that the ACLU had used the system’s default setting for matches, called a “confidence threshold,” of 80 percent. That means the group counted any face matches the system proposed that had a similarity score of 80 percent or more. Amazon itself uses the same percentage in one facial recognition example on its site describing matching an employee’s face with their work ID badge. But Lindsey said that Amazon recommended that police departments use a much higher similarity score — 95 percent — to reduce the likelihood of erroneous matches.
Facial recognition — a technology that can be used to identify unknown people in photos or videos without their knowledge or permission — is fast becoming a top target for civil liberties groups and privacy experts.
Proponents see it as a useful tool that can help identify criminals. It was recently used to identify the man charged in the deadly shooting at the The Capital Gazette’s newsroom in Annapolis, Maryland.
But civil liberties groups view it as a surveillance system that can inhibit people’s ability to participate in political protests or go about their lives anonymously. This month, Microsoft called said the technology was too risky for tech companies to deploy without government oversight and called on Congress to regulate it.
Over the last two months, Amazon has come under increasing pressure for selling its Rekognition technology to law enforcement agencies. The company has sold the service as a way for police departments to easily identify suspects in photos or videos.
Amazon’s site describes how its system can perform “real-time face recognition across tens of millions of faces” and detect “up to 100 faces in challenging crowded photos.” (The New York Times recently used the Amazon technology to help identify guests at the royal wedding of Prince Harry and Meghan Markle.)
In May, two dozen civil liberties groups, led by the ACLU, wrote a letter to Bezos, demanding that his company stop selling the facial technology to law enforcement. The groups warned that the software could be used to trail protesters, unauthorized immigrants or other members of the public — not just criminal suspects.
Similar demands of Bezos from Amazon employees, Amazon investors, and several hundred academics soon followed.
Snow of the ACLU said his group’s test of Amazon’s software should push Congress to put a moratorium on law enforcement’s use of facial recognition technology.
But in a blog post last month, Matt Wood, general manager of artificial intelligence at Amazon Web Services, said that there had been no reports of law enforcement abuse of Amazon’s facial technology. He added that Amazon believed it was “the wrong approach to impose a ban on promising new technologies because they might be used by bad actors for nefarious purposes in the future.”
In a letter to Amazon from the Congressional Black Caucus, the group’s members noted the potential for racial bias with the technology — an issue raised by a recent MIT study that found some commercial facial recognition systems correctly identified a higher proportion of white men than darker-skinned women. In their letter, the Caucus members urged Bezos to hire “more lawyers, engineers and data scientists of color to assist in properly calibrating this technology to account for racial bias that can lead to inaccuracies with potentially devastating outcomes.”
In the civil liberties group’s test, the Amazon software misidentified several members of the Congressional Black Caucus as other people who had been arrested, including Lewis and Rush.
“We think these test results really raise the concern that facial recognition has a race problem,” said Snow, the ACLU lawyer.