Keywords/Phrases:

Machine Learning: Refers to computers learning from data – spotting trends, making predictions, etc.

Neural Networks: Subsets of machine learning composed of many complex algorithms meant to mimic the way the human brain processes data. Their usage is popular in deep learning software today such as facial recognition.

Training Data: Data used to train a machine learning model to predict certain outcomes given specific inputs.

Introduction:

Artificial Intelligence, or AI, exists across our world more profoundly than most realize, and its scope and impact will rapidly grow in coming years. Beyond unlocking cell phones with facial recognition, AI predicts an online shopper’s tendencies and targets them with specific advertisements, identifies the faces of criminal suspects, and even assists judges in deciding whether or not to grant defendants bail. Therefore, as the implications of AI transition from opening our devices to deciding our civil liberties, it is critical to consider these systems’ imperfections. When Joy Buolamwini, a researcher in the MIT Media Lab, discovered that certain facial recognition systems had trouble detecting her dark-skinned female face but accurately distinguished her while she wore a white mask, she launched on a mission to unveil racial and sexist biases within AI. Her journey is detailed in the Netflix documentary Coded Bias. Unlike Joy, most citizens don’t know of the pertinent danger that biased AI poses, so this article illuminates such hazards while detailing the actions that must be taken to eliminate them.

What is Machine Learning

Machine-learning is the subset of artificial intelligence that uses inputted “training-data” to make predictions about future outcomes. For example, Netflix’s recommendation software utilizes customer’s past selections to predict future movies or TV shows that they may enjoy (“What is Machine Learning?”). Today, a popular way to do this is with neural networks, which examine data similar to how the human brain does so, using thousands, or even millions, of interconnecting nodes. Each node receives a data point (numerical) from one of its connected nodes, and multiplies it by a specific weight (a constant given to that node). Depending on if the result falls below or above a certain threshold, the result will then be sent to another neighboring node or the node will “fire” which means it sends the number outwards to the output values. To some reader’s, this may seem extremely techy, but these ideas have been around since the 1940s, long before Windows and Macintosh (Hardesty). The popularity of machine-learning systems fluctuates across generations, but in the last decade these systems have exploded in popularity with the rise of Big Data, as computers can analyze and spot trends in millions of data points very rapidly, making machine-learning models extremely useful for a large variety of tasks.

The Issue with Machine Learning

Faulty results appear in neural networks when their interconnecting webs of nodes become so large and complex that developers have a difficult time pinpointing exactly what traits that the algorithm identifies to produce its output. Is Netflix’s software making recommendations based on the actor’s you like, genre’s you prefer, or director’s you have previously watched? It’s often unclear. Maybe you can see where this is going. When the justice system utilizes similar algorithms to decide if a defendant should be granted bail or not, by what parameters is the software deciding? Criminal history and employment status, or race and sex? Many civil rights activists would argue that such software bases decisions off of the latter two, which unjustly tramples not just The Fourteenth Amendment but also inherent human rights. This is part of the reason why bias is so dangerous and difficult to pinpoint, because developer’s don’t quite know what within their code creates it. After all, the system is meant to mimic a human brain, so it is impossible to decipher its exact thinking and methodology at all times.

Small-scale illustration of neural network basic functioning. As depicted, there are many hidden layers between the input and output which makes it difficult to precisely spot the source of biases. Sourced from IBM

Preventing Bias

Recently, programmer’s have been able to reduce bias in their machine-learning models by utilizing large and diverse sets of training-data so that their software is accustomed to many different traits and incorporates all into its decision making. Therefore, even if developer’s cannot identify which specific traits the software utilizes to determine outputs, the large quantity of data prevents any singular trait from too heavily influencing the data. Unfortunately, for much of machine learning’s history this usage of diverse training data sets has not been the case, which explains the existence of strong racial and sexist biases that Joy Buolamwini spotted in her 2018 paper, Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification.

Accuracy of Facial Recognition Software by Skin-Tone and Race

Sourced from Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification

History of Machine Learning

Neural networks—the algorithmic approach to neurology that mimics human thinking in computers—was first proven as a concept in 1944 by researchers Warren McCullough and Walter Pitts as they modeled the neuron geometrically. Machine learning in computing, however, became popular in the 1960’s, growing in fame when Robert Nealey, a world-renowned chess player, lost a game of chess to the IBM 7094 computer in 1962 (“What is Machine Learning?”). Prior to 2012, the majority of machine learning models used feature extraction, in which the programmer specifies the exact traits that the model searches for in the input data to determine outputs (Burns and Brush). However, since 2012 a new form of machine-learning has arisen, deep-learning, in which the programmer doesn’t specify any features for the program to focus on, instead letting the program learn freely as a human brain would. While big tech doubted the capabilities of deep-learning for many decades prior to 2010, the development of computers with the strength to rapidly analyze millions of data points encouraged companies like Google to research and develop new forms of artificial intelligence. By 2013, Google’s deep learning software had taught itself the concept of a cat from simply watching youtube videos. Quite im-purr-sive!

Garbage In, Garbage Out

A popular saying within computer science, when you give a program garbage data, it produces garbage results. Deep learning is very good at identifying trends in data, so being an excellent learner it will learn the bias of any dataset given to it. Historically, the tech industry lacks diversity, as 75% of tech workers are male, and 83.3% of tech executives are white (“Diversity In High Tech Statistics”). A similar trend is existent within the data used to train facial recognition software. Therefore, the programs that Boulamwani identified as racially biased were themselves not technically faulty, but they had learned to identify faces from data that skewed towards white males. This left them lacking accuracy when identifying darker skinned and feminine faces because their training hadn’t properly prepared them to do so. Another example is Compas, or Correctional Offender Management Profiling for Alternative Sanctions, the software used by judges to assist in deciding whether or not to grant a patient bail, which analyzes criminal records to predict how likely a defendant is to commit another crime. As black Americans are 5x more likely to be arrested than white Americans, these systems are disproportionately trained on the profiles of black Americans and have come to innacurately identify black Americans as more likely to commit future crimes (Angwin et al.). Even when profiles looked nearly identical between defendants, skin color was enough to change low-risk for a white defendant to high-risk for a black defendant. This has left its creator, Northpointe, subject of numerous allegations of foul play.

As futuristic as AI seems, machine learning makes its decisions based on the data it is trained with—data that is often years or decades old—so in a sense, the output of an AI model is a product of the past. When this past is littered with a lack of diversity in tech as well as racially biased criminal databases, so are the system’s future predictions. The only way to change this is to retrain data with datasets that reflect the true diversity of our world.

Beyond Computing

Depiction of the sources for error within machine learning algorithms

Sourced from the National Institute of Standards and Technology

Many argue that bias within AI branches beyond errors in training these models. While these would be considered Statistical/Computational Biases, other, more profoundly ingrained biases such as Human Biases and Systemic Biases also exist. For example, activists point towards the NYPD’s database of 42,000 “gang affiliates”—with no proven gang affiliation actually required—composed of 99% black and latino citizens (Najibi). Therefore, if “gang affiliation” is considered as a factor in system’s that decide defendant’s bail or identify suspects, race indirectly contributes to the system’s decision. This is an example of how a systemic bias—the racially-biased assumption that black and latinx are more likely to join gangs, contributes to bias within AI that developers may not realize even exists. Human biases and systemic biases are difficult to identify and nearly impossible to repair as they are hidden deep within unconscious parts of our brains, yet we can address them by taking inspiration from big tech’s approach to repairing computational biases. When Gender Shades demonstrated racial bias within facial recognition AI, companies like IBM responded by using diverse datasets to train their AI, which greatly improved accuracy for dark-skinned and female faces. Therefore, to reduce human and systemic biases, diversity must increase in not just the data used to train computers but also the programmer’s that build them. By building software with the opinions, contributions, and unconscious tendencies of diverse programmers, software will become more apt for broad usage worldwide.

In the meantime, one can hold AI developer’s responsible for communicating any potential bias in their programs. As difficult as bias can be to repair, letting the public know of it is easy and a human right that citizens deserve. Research how AI is used in your daily life, and ensure that it’s potential biases are not putting you in danger.

About the Author

AJ Caesar is a rising senior in high school passionate about engineering, environmentalism, and computer science. Recently, after viewing the Netflix documentary Coded Bias he decided to write about bias within AI in an effort to bring attention and improvement to this pertinent issue. Please email iysj.ajcaesar@gmail.com with any questions.

Works Cited

Angwin, Julia, et al. “Machine Bias — ProPublica.” ProPublica, 23 May 2016, https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing. Accessed 12 August 2022.

Arcas, Aguera Y. “Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification.” Proceedings of Machine Learning Research, http://proceedings.mlr.press/v81/buolamwini18a/buolamwini18a.pdf. Accessed 11 August 2022.

Buolamwini, Joy. Gender Shades, http://gendershades.org. Accessed 12 August 2022.

Buranyi, Stephen. “Rise of the racist robots – how AI is learning all our worst impulses.” The Guardian, 8 August 2017, https://www.theguardian.com/inequality/2017/aug/08/rise-of-the-racist-robots-how-ai-is-learning-all-our-worst-impulses. Accessed 13 August 2022.

Burns, Ed, and Kate Brush. “What is Deep Learning and How Does It Work?” TechTarget, https://www.techtarget.com/searchenterpriseai/definition/deep-learning-deep-neural-network. Accessed 12 August 2022.

“Diversity In High Tech Statistics [2022] – Zippia.” Zippia, 14 April 2022, https://www.zippia.com/advice/diversity-in-high-tech-statistics/. Accessed 12 August 2022.

Hardesty, Larry. “Explained: Neural networks.” MIT News, 14 April 2017, https://news.mit.edu/2017/explained-neural-networks-deep-learning-0414. Accessed 12 August 2022.

Najibi, Alex. “Racial Discrimination in Face Recognition Technology.” Science in the News, 24 October 2020, https://sitn.hms.harvard.edu/flash/2020/racial-discrimination-in-face-recognition-technology/. Accessed 11 August 2022.

Onstad, Katrina, and Daniel Ehrenworth. “The AI superstars at Google, Facebook, Apple—they all studied under this guy.” Toronto Life, 29 January 2018, https://torontolife.com/life/ai-superstars-google-facebook-apple-studied-guy/. Accessed 12 August 2022.

Rivero, Nicolás. “Why IBM abandoned its facial recognition program.” Quartz, 10 June 2020, https://qz.com/1866848/why-ibm-abandoned-its-facial-recognition-program/. Accessed 12 August 2022.

Thomas, Mike. “A Brief History of Deep Learning.” Built In, 2 April 2019, https://builtin.com/artificial-intelligence/deep-learning-history. Accessed 12 August 2022.

“What are Neural Networks?” IBM, 17 August 2020, https://www.ibm.com/cloud/learn/neural-networks. Accessed 12 August 2022.“What is Machine Learning?” IBM, 15 July 2020, https://www.ibm.com/cloud/learn/machine-learning. Accessed 11 October 2022.