“Why are we building this?” an Amazon employee wrote. The simplicity of the question was devastating, in context—asked, as it was, in an anonymous essay on Medium, written to protest the company’s sale of its Rekognition facial-recognition software to the police.
Why should a question like that be anonymous? Why is it being asked after the thing has already been done? Because to ask is to chip away at the pillars of the technocracy.
The employee wrote:
Amazon is designing, marketing, and selling a system for dangerous mass surveillance right now. Amazon’s website brags of the system’s ability to store and search tens of millions of faces at a time. Law enforcement has already started using facial recognition with virtually no public oversight or debate or restrictions on use from Amazon. Orlando, Florida, is testing Rekognition with live video feeds from surveillance cameras around the city. A sheriff’s department in Oregon is currently using Rekognition to let officers in the field compare photos to a database of mugshots. This is not a hypothetical situation.
Amazon is building this capability because it doesn’t care about why it’s building it. There is a market; there is a place to grow. It is hard to remember from hour to hour which of the tech colossuses has crossed which line most recently, in their shared commitment to expansion against all other values. Yesterday was also the day to absorb the news that Google’s CEO, Sundar Pichai, had admitted the company is pressing ahead with testing of Project Dragonfly, the formerly secret search system it’s building to comply with Chinese censorship.
Project Dragonfly is the opposite of what Google has always claimed to do—contrary not only to its now-abandoned “Don’t be evil” motto, but to the substance of the work that the company was founded on. Google launched as the best search engine ever built, enabling users to find the best information on the internet. What it’s building for China, as the Intercept reported in breaking the news of the project, is a system that will replace the best information with the information the government wants provided, when the government deems it necessary to make the switch:
Sources familiar with Dragonfly said the search platform also appeared to have been tailored to replace weather and air pollution data with information provided directly by an unnamed source in Beijing.
Pichai, as justification for the project, said that the search system could give people in China access to more information about cancer treatments than they can currently get. At the expense, unfortunately, of true information about the things might give them cancer.
But isn’t some information better than none? Project Dragonfly, Pichai said, will allow Chinese users to get information about “well over 99 percent of queries” they submit. This is the standard excuse for everyone who agrees to sell out to the Chinese regime—look at how much information the censors don’t censor! That is what censorship is: taking some information out and leaving the rest. Leaving the rest is what makes the lies of omission effective. More than 99 percent of the days in 1989 were not June 4. How much difference does the one day make?
