Programmers need ethics when designing the technologies that influence people's lives

Computing professionals are on the front lines of almost every aspect of the modern world. They’re involved in the response when hackers steal the personal information of hundreds of thousands^[1] of people from a large corporation. Their work can protect – or jeopardize – critical infrastructure like electrical grids^[2] and transportation lines^[3]. And the algorithms they write may determine who gets a job, who is approved for a bank loan^[4] or who gets released on bail^[5].

Technological professionals are the first, and last, lines of defense against the misuse of technology. Nobody else understands the systems as well, and nobody else is in a position to protect specific data elements or ensure the connections between one component and another are appropriate, safe and reliable. As the role of computing continues its decades-long expansion in society, computer scientists are central to what happens next.

That’s why the world’s largest organization of computer scientists and engineers, the Association for Computing Machinery^[6], of which I am president, has issued a new code of ethics for computing professionals^[7]. And it’s why ACM is taking other steps to help technologists engage with ethical questions.

Serving the public interest

A code of ethics is more than just a document on paper. There are hundreds of examples of the core values and standards^[8] to which every member of a field is held – including for organist guilds^[9] and outdoor advertising associations^[10]. The world’s oldest code of ethics^[11] is also its most famous: the Hippocratic oath medical doctors^[12] take, promising to care responsibly for their patients.

I suspect that one reason for the Hippocratic oath’s fame is how personal medical treatment can be, with people’s lives hanging in the balance. It’s important for patients to feel confident their medical caregivers have their interests firmly in mind.

Technology is, in many ways, similarly personal. In modern society computers, software and digital data are everywhere. They’re visible in laptops and smartphones, social media and video conferencing, but they’re also hidden inside the devices that help manage people’s daily lives, from thermostats to timers on coffee makers. New developments in autonomous vehicles, sensor networks and machine learning mean computing will play an even more central role in everyday life in coming years.

A changing profession

As the creators of these technologies, computing professionals have helped usher in the new and richly vibrant rhythms of modern life. But as computers become increasingly interwoven into the fabric of life, we in the profession must personally recommit to serving society through ethical conduct.

ACM’s last code of ethics was adopted in 1992, when many people saw computing work as purely technical. The internet was in its infancy and people were just beginning to understand the value of being able to aggregate and distribute information widely. It would still be years before artificial intelligence and machine learning had applications outside research labs.

Today, technologists’ work can affect the lives and livelihoods of people in ways that may be unintended, even unpredictable. I’m not an ethicist by training, but it’s clear to me that anyone in today’s computing field can benefit from guidance on ethical thinking and behavior.

Updates to the code

ACM’s new ethics code has several important differences from the 1992 version. One has to do with unintended consequences. In the 1970s and 1980s, technologists built software or systems whose effects were limited to specific locations or circumstances. But over the past two decades, it has become clear that as technologies evolve, they can be applied in contexts very different from the original intent.

For example, computer vision research has led to ways of creating 3D models of objects^[14] – and people – based on 2D images, but it was never intended to be used in conjunction with machine learning in surveillance or drone applications^[15]. The old ethics code asked software developers to be sure a program would actually do what they said it would. The new version also exhorts developers to explicitly evaluate their work to identify potentially harmful side effects or potential for misuse.

Another example has to do with human interaction. In 1992, most software was being developed by trained programmers to run operating systems, databases and other basic computing functions. Today, many applications rely on user interfaces to interact directly with a potentially vast number of people^[16]. The updated code of ethics includes more detailed considerations about the needs and sensitivities of very diverse potential users – including discussing discrimination, exclusion and harassment.

More and more software is being developed to run with little or no input or human understanding, producing analytical results to guide decision-making, such as when to approve bank loans. The outputs can have completely unintended social effects, skewed against whole classes of people – like recent cases where data-mining predictions of who would default on a loan showed biases against people who seek longer-term loans^[18] or live in particular areas^[19]. There are also dangers of what are called “false positives,” when a computer links two things that shouldn’t be connected – as when facial recognition software recently matched members of Congress to criminals’ mug shots^[20]. The revised code exhorts technologists to take special care to avoid creating systems with the potential to oppress or disenfranchise whole groups of people.

Living ethics in technology

The code was revised over the course of more than two years, including ACM members and people outside the organization and even outside the computing and technological professions. All these perspectives made the code better. For example, a government-employed weapons designer asked whether that job inherently required violating the code; the wording was changed to clarify that systems must be “consistent with the public good.”

Now that the code is out, there’s more to do. ACM has created a repository for case studies^[21] showing how ethical thinking and the guidelines can be applied in a variety of real-world situations. The group’s “Ask An Ethicist” blog and video series^[22] invites the public to submit scenarios or quandaries as they arise in practice. Word is also underway to develop teaching modules so the concepts can be integrated into computing education from primary school through university.

Feedback has been overwhelmingly positive. My personal favorite was the comment from a young programmer after reading the code: “Now I know what to tell my boss if he asks me to do something like that again.”

The ACM Code of Ethics and Professional Conduct begins with the statement, “Computing professionals’ actions change the world.” We don’t know if our code will last as long as the Hippocratic oath. But it highlights how important it is that the global computing community understands the impact our work has – and takes seriously our obligation to the public good.

References

1. ^{^} steal the personal information of hundreds of thousands (theconversation.com)
2. ^{^} electrical grids (theconversation.com)
3. ^{^} transportation lines (theconversation.com)
4. ^{^} approved for a bank loan (theconversation.com)
5. ^{^} released on bail (theconversation.com)
6. ^{^} Association for Computing Machinery (www.acm.org)
7. ^{^} new code of ethics for computing professionals (ethics.acm.org)
8. ^{^} hundreds of examples of the core values and standards (ethicscodescollection.org)
9. ^{^} organist guilds (www.agohq.org)
10. ^{^} outdoor advertising associations (oaaa.org)
11. ^{^} world’s oldest code of ethics (www.nlm.nih.gov)
12. ^{^} Hippocratic oath medical doctors (www.pbs.org)
13. ^{^} Alexander Raths/Shutterstock.com (www.shutterstock.com)
14. ^{^} creating 3D models of objects (arxiv.org)
15. ^{^} machine learning in surveillance or drone applications (maliciousaireport.com)
16. ^{^} interact directly with a potentially vast number of people (theconversation.com)
17. ^{^} fizkes/Shutterstock.com (www.shutterstock.com)
18. ^{^} biases against people who seek longer-term loans (www.mckinsey.com)
19. ^{^} live in particular areas (sloanreview.mit.edu)
20. ^{^} matched members of Congress to criminals’ mug shots (www.nytimes.com)
21. ^{^} repository for case studies (ethics.acm.org)
22. ^{^} “Ask An Ethicist” blog and video series (ethics.acm.org)