The ethical risks of biased AI systems can be grouped into two primary categories. The risk of allocative harm and the risk of representational harm. Allocative harm means that opportunities or resources are withheld from certain people or certain groups by the AI system through the bias contained in the system. Let's look at a couple of examples of allocative harm. Our first example would be an algorithm that is used to automate the review of resumes of candidates who apply to certain technical jobs. Upon review of the algorithm, we find that the algorithm is primarily selecting male candidates to be interviewed for the technical jobs rather than female candidates. This would be one example of allocative harm against female candidates who are applying for the role because the algorithm is withholding opportunities from the candidates due to the way it's structured than the bias it contains. A second example would be an algorithm that's used to assign credit limits to applicants applying for a new credit card. When we dive into the algorithm, we find that men and women coming from identical backgrounds with identical characteristics are receiving different credit limits from the algorithm. This is an example that actually occurred a couple of years ago when Apple released its Apple credit card. A large number of people, including one of the co-founders of Apple, Steve Wozniak, charged that the company and the bank that the company is working with on the credit card were using an algorithm that was biased against women applying for the card. They believe that men and women with identical backgrounds, in some cases, married couples who are both applying for the credit card were receiving different credit limits, with men generally receiving a higher credit limit despite having identical characteristics. The second major category of harm that can be caused by bias AI systems is called representational harm. Representational harm means that certain people or groups are being stigmatized or stereotyped by the AI system. AI can either propagate existing stereotypes, which it learns from the way that the model is trained, or even create new stereotypes based on the training it receives. Let's look at an example of representational harm. Suppose we're building a computer vision model to classify images and identify the occupation that the person within the image. We train our model, and we find that the model identifies all females in pictures of a hospital setting as being nurses rather than doctors. This would be one example where our model has learned a historical stereotype, and it's propagating that stereotype in the way that it's creating its predictions. When we build ethical AI systems, we have three primary goals in mind. We want to design systems that are fair, accountable, and transparent. The idea of fairness in AI has roots in the application of anti-discrimination laws to AI systems. One of the challenging things about measuring fairness in AI is that there's no single universally agreed-upon definition of fairness. There are actually many definitions of fairness, and some of these can come in conflict with others. Let's take a look at two primary definitions of fairness which relate to AI systems. Individual fairness and group fairness. The idea of individual fairness means that people who are similar, who have similar attributes or backgrounds should receive similar outcomes from an AI model. For example, if we were developing a model to determine which candidates who apply for graduate school should receive admission to the graduate school, we'd expect that if a man and a woman, both of whom come from identical backgrounds and they have identical characteristics, if both apply, they would either both be accepted or both be rejected by the model. The idea of group fairness holds the different groups should experience similar levels of the positive outcome from a model, or experience similar rates of error within the model. A famous example of a lack of group fairness in an AI system is the facial recognition software developed by many of the major tech companies. It's been proven by researchers over the last few years that facial recognition systems have a significant bias towards certain groups. For example, it's been shown that the rates of error of such systems on faces of black females are much higher than the identification of faces of white males. Accountability means that there's a clear responsibility for the outcomes of an AI model and that users of the system have some recourse if they identify problems or issues with the system in the way that the model is working. Some of the key considerations regarding accountability are, who is responsible for the performance of the system? Who's ensuring that the data is accurate, that the model is functioning well, and that there's no bias contained in the system? On what set of values and laws is the system based? Generally, systems are designed in accordance with the local culture and laws under which they're developed, and so it's important to understand to which set of values and laws should we hold the system accountable? Finally, what recourse do users have? They believe the system is not behaving in accordance with the values and laws. The idea of transparency means that users have visibility into what data is being collected and used by the AI system, and how does the model actually function and generate its outcomes. There are several methods of providing transparency. The first is to use interpretable models in the AI system. Rather than using a complex neural network, for example, it's very difficult to explain how it's achieving its outcomes. We might choose to use a simple linear regression or a decision tree. It makes it very easy to explain to our users how the model functions and how it reaches its outcomes. Another method of providing transparency is communicating the importance of the different features or the attributes which the model uses. We might explain to a user which attributes the model is using and which ways it's assigning to different attributes in order to reach its decisions. We can also use simplified approximations of more complex models to explain to our users how the model predictions are being reached. If we're using a complex model such as a neural network, we may create a system of simplified approximation using a set of business rules or a simpler model, and then use this simpler model or set of rules to be able to explain to our users why the model is reaching the conclusions it reaches. Finally, we could provide what's called counterfactual explanations to our users. This means that we provide insight to our users in the smallest amount of change that would be required in order to change the outcome of our model. For example, if we were creating a model that assigns credit limits to applicants for a new credit card, we might provide an explanation to our user. It says something like, if your credit score were 10 points higher, you would have received a $2,000 higher credit limit. This provides our users some insight into the important characteristics that the model is using to make its decisions and what that user can do to change the decision the model is reaching.
