How artificial intelligence systems could threaten democracy

U.S. technology giant Microsoft has teamed up with a Chinese military university^[1] to develop artificial intelligence systems^[2] that could potentially enhance government surveillance and censorship capabilities. Two U.S. senators publicly condemned^[3] the partnership, but what the National Defense Technology University of China^[4] wants from Microsoft isn’t the only concern.

As my research shows^[5], the advent of digital repression is profoundly affecting the relationship between citizen and state^[6]. New technologies are arming governments with unprecedented capabilities to monitor, track and surveil individual people. Even governments in democracies with strong traditions of rule of law^[7] find themselves tempted to abuse these new abilities^[8].

In states with unaccountable institutions and frequent human rights abuses^[9], AI systems will most likely cause greater damage. China is a prominent example. Its leadership has enthusiastically embraced AI technologies, and has set up the world’s most sophisticated^[10] surveillance state^[11] in Xinjiang province^[12], tracking citizens’ daily movements and smartphone use.

Its exploitation of these technologies presents a chilling model^[13] for fellow autocrats and poses a direct threat to open democratic societies. Although there’s no evidence that other governments have replicated this level of AI surveillance, Chinese companies are actively exporting the same underlying technologies across the world.

Increasing reliance on AI tools in the US

Artificial intelligence systems^[15] are everywhere in the modern world, helping run smartphones, internet search engines, digital voice assistants and Netflix movie queues. Many people fail to realize^[16] how quickly AI is expanding, thanks to ever-increasing amounts of data to be analyzed, improving algorithms and advanced computer chips.

Any time more information becomes available and analysis gets easier, governments are interested – and not just authoritarian ones. In the U.S., for instance, the 1970s saw revelations that government agencies – such as the FBI, CIA and NSA – had set up expansive domestic surveillance networks^[17] to monitor and harass civil rights protesters, political activists and Native American groups. These issues haven’t gone away: Digital technology today has deepened the ability of even more agencies to conduct even more intrusive surveillance.

For example, U.S. police have eagerly embraced AI technologies. They have begun using software that is meant to predict where crimes will happen^[19] to decide where to send officers on patrol. They’re also using facial recognition^[20] and DNA analysis^[21] in criminal investigations. But analyses of these systems show the data on which those systems are trained^[22] are often biased, leading to unfair outcomes^[23], such as falsely determining that African Americans are more likely to commit crimes^[24] than other groups.

AI surveillance around the world

In authoritarian countries, AI systems can directly abet domestic control and surveillance, helping internal security forces process massive amounts of information^[25] – including social media posts, text messages, emails and phone calls – more quickly and efficiently. The police can identify social trends and specific people^[26] who might threaten the regime based on the information uncovered by these systems.

For instance, the Chinese government has used AI in wide-scale crackdowns in regions that are home to ethnic minorities within China. Surveillance systems in Xinjiang and Tibet have been described as “Orwellian^[27].” These efforts have included mandatory DNA samples^[28], Wi-Fi network monitoring and widespread facial recognition cameras, all connected to integrated data analysis platforms. With the aid of these systems, Chinese authorities have, according to the U.S. State Department, “arbitrarily detained” between 1 and 2 million people^[29].

My research looks at 90 countries around the world^[30] with government types ranging from closed authoritarian to flawed democracies, including Thailand, Turkey, Bangladesh and Kenya. I have found that Chinese companies are exporting AI surveillance technology^[31] to at least 54 of these countries. Frequently, this technology is packaged as part of China’s flagship Belt and Road Initiative^[32], which is funding an extensive network of roads, railways, energy pipelines and telecommunications networks serving 60% of the world’s population^[33] and economies that generate 40% of global GDP.

For instance, Chinese companies like Huawei^[34] and ZTE are constructing “smart cities” in Pakistan^[35], the Philippines^[36] and Kenya^[37], featuring extensive built-in surveillance technology. For example, Huawei has outfitted Bonifacio Global City^[38] in the Philippines with high-definition internet-connected cameras that provide “24/7 intelligent security surveillance^[39] with data analytics to detect crime and help manage traffic.”

Hikvision^[41], Yitu^[42] and SenseTime^[43] are supplying state-of-the-art facial recognition cameras for use in places like Singapore^[44] – which announced the establishment of a surveillance program with 110,000 cameras mounted on lamp posts^[45] around the city-state. Zimbabwe is creating a national image database^[46] that can be used for facial recognition.

However, selling advanced equipment for profit is different than sharing technology with an express geopolitical purpose. These new capabilities may plant the seeds for global surveillance: As governments become increasingly dependent upon Chinese technology to manage their populations and maintain power, they will face greater pressure to align with China’s agenda. But for now it appears that China’s primary motive is to dominate the market for new technologies and make lots of money in the process.

AI and disinformation

In addition to providing surveillance capabilities that are both sweeping and fine-grained, AI can help repressive governments manipulate available information and spread disinformation. These campaigns can be automated or automation-assisted, and deploy hyper-personalized messages^[47] directed at – or against – specific people^[48] or groups.

AI also underpins the technology commonly called “deepfake^[49],” in which algorithms create realistic video and audio forgeries^[50]. Muddying the waters between truth and fiction may become useful in a tight election, when one candidate could create fake videos showing an opponent doing and saying things that never actually happened.

In my view, policymakers in democracies should think carefully about the risks of AI systems to their own societies and to people living under authoritarian regimes around the world. A critical question is how many countries will adopt China’s model of digital surveillance. But it’s not just authoritarian countries feeling the pull. And it’s also not just Chinese companies spreading the technology: Many U.S. companies, Microsoft included, but IBM, Cisco and Thermo Fisher^[51] too, have provided sophisticated capabilities to nasty governments. The misuse of AI is not limited to autocratic states.

References

1. ^{^} Microsoft has teamed up with a Chinese military university (www.ft.com)
2. ^{^} artificial intelligence systems (www.irishtimes.com)
3. ^{^} U.S. senators publicly condemned (www.ft.com)
4. ^{^} National Defense Technology University of China (www.nudt.edu.cn)
5. ^{^} my research shows (scholar.google.com)
6. ^{^} the relationship between citizen and state (doi.org)
7. ^{^} rule of law (theconversation.com)
8. ^{^} these new abilities (qz.com)
9. ^{^} unaccountable institutions and frequent human rights abuses (www.foreignaffairs.com)
10. ^{^} most sophisticated (www.nytimes.com)
11. ^{^} surveillance state (www.engadget.com)
12. ^{^} Xinjiang province (www.theguardian.com)
13. ^{^} presents a chilling model (www.georgesoros.com)
14. ^{^} AP Photo/Ng Han Guan (www.apimages.com)
15. ^{^} Artificial intelligence systems (ai.stanford.edu)
16. ^{^} Many people fail to realize (governanceai.github.io)
17. ^{^} expansive domestic surveillance networks (www.intelligence.senate.gov)
18. ^{^} Arnout de Vries (commons.wikimedia.org)
19. ^{^} meant to predict where crimes will happen (theconversation.com)
20. ^{^} facial recognition (www.nbcnews.com)
21. ^{^} DNA analysis (www.washingtonpost.com)
22. ^{^} data on which those systems are trained (theconversation.com)
23. ^{^} unfair outcomes (theconversation.com)
24. ^{^} falsely determining that African Americans are more likely to commit crimes (www.propublica.org)
25. ^{^} internal security forces process massive amounts of information (www.power3point0.org)
26. ^{^} specific people (www.apnews.com)
27. ^{^} Orwellian (foreignpolicy.com)
28. ^{^} mandatory DNA samples (www.nytimes.com)
29. ^{^} 1 and 2 million people (www.state.gov)
30. ^{^} research looks at 90 countries around the world (papers.ssrn.com)
31. ^{^} exporting AI surveillance technology (carnegieendowment.org)
32. ^{^} Belt and Road Initiative (eng.yidaiyilu.gov.cn)
33. ^{^} serving 60% of the world’s population (www.knightfrank.com)
34. ^{^} Huawei (e.huawei.com)
35. ^{^} Pakistan (www.dawn.com)
36. ^{^} the Philippines (e.huawei.com)
37. ^{^} Kenya (www.chinadaily.com.cn)
38. ^{^} Bonifacio Global City (bgc.com.ph)
39. ^{^} 24/7 intelligent security surveillance (e.huawei.com)
40. ^{^} alveo land/Wikimedia Commons (en.wikipedia.org)
41. ^{^} Hikvision (foreignpolicy.com)
42. ^{^} Yitu (www.scmp.com)
43. ^{^} SenseTime (qz.com)
44. ^{^} Singapore (www.albawaba.com)
45. ^{^} 110,000 cameras mounted on lamp posts (www.reuters.com)
46. ^{^} national image database (foreignpolicy.com)
47. ^{^} hyper-personalized messages (theconversation.com)
48. ^{^} specific people (www.nytimes.com)
49. ^{^} deepfake (www.technologyreview.com)
50. ^{^} realistic video and audio forgeries (theconversation.com)
51. ^{^} IBM, Cisco and Thermo Fisher (www.axios.com)