Explore
For Enterprise
Join for Free
Log In

University of Michigan
Gain new skills in construction management in under 7 months
Earn the Construction Engineering & Management MasterTrack™ Certificate from a top-ranked school.
deeplearning.ai
New! TensorFlow: Data and Deployment Specialization
Join Andrew Ng and Laurence Moroney in the latest Specialization from deeplearning.ai.

浏览 Coursera 的全部课程

浏览
顶级课程
企业版
登录
免费加入

How to Calculate the Area Under the ROC Curve

Loading...

使用 Excel 分析数据

4.2（3,055 个评分） | 270K 名学生已注册

课程 2（共 5 门，从 Excel 到 MySQL：商业分析技术专项课程）

Important: The focus of this course is on math - specifically, data-analysis concepts and methods - not on Excel for its own sake. We use Excel to do our calculations, and all math formulas are given as Excel Spreadsheets, but we do not attempt to cover Excel Macros, Visual Basic, Pivot Tables, or other intermediate-to-advanced Excel functionality. This course will prepare you to design and implement realistic predictive models based on data. In the Final Project (module 6) you will assume the role of a business data analyst for a bank, and develop two different predictive models to determine which applicants for credit cards should be accepted and which rejected. Your first model will focus on minimizing default risk, and your second on maximizing bank profits. The two models should demonstrate to you in a practical, hands-on way the idea that your choice of business metric drives your choice of an optimal model. The second big idea this course seeks to demonstrate is that your data-analysis results cannot and should not aim to eliminate all uncertainty. Your role as a data-analyst is to reduce uncertainty for decision-makers by a financially valuable increment, while quantifying how much uncertainty remains. You will learn to calculate and apply to real-world examples the most important uncertainty measures used in business, including classification error rates, entropy of information, and confidence intervals for linear regression. All the data you need is provided within the course, all assignments are designed to be done in MS Excel, and you will learn enough Excel to complete all assignments. The course will give you enough practice with Excel to become fluent in its most commonly used business functions, and you’ll be ready to learn any other Excel functionality you might need in the future (module 1). The course does not cover Visual Basic or Pivot Tables and you will not need them to complete the assignments. All advanced concepts are demonstrated in individual Excel spreadsheet templates that you can use to answer relevant questions. You will emerge with substantial vocabulary and practical knowledge of how to apply business data analysis methods based on binary classification (module 2), information theory and entropy measures (module 3), and linear regression (module 4 and 5), all using no software tools more complex than Excel.

查看授课大纲

您将学习的技能

Binary Classification, Data Analysis, Microsoft Excel, Linear Regression

审阅

4.2（3,055 个评分）

5 stars
1,731 ratings
4 stars
745 ratings
3 stars
280 ratings
2 stars
154 ratings
1 star
145 ratings

JL

Apr 15, 2018

This course is worth taking by anyone who needs to have a better understanding of how the most efficient and best decisions in the market place are taken using Data management simply in Excel.

RC

Jan 15, 2017

I love this course and the way the material is explained. I plan on taking this course again to improve my understanding of the information because I think it relays very important ideas.

从本节课中

Binary Classification

Separating collections into two categories, such as “buy this stock, don’t but that stock” or “target this customer with a special offer, but not that one” is the ultimate goal of most business data-analysis projects. There is a specialized vocabulary of measures for comparing and optimizing the performance of the algorithms used to classify collections into two groups. You will learn how and why to apply these different metrics, including how to calculate the all-important AUC: the area under the Receiver Operating Characteristic (ROC) Curve.

Introduction to Binary Classification7:43

Bombers and Seagulls: Confusion Matrix8:49

Costs Determine Optimal Threshold4:36

Calculating Positive and Negative Predictive Values5:34

How to Calculate the Area Under the ROC Curve11:44

Binary Classification with More than One Input Variable7:26

教学方

Jana Schaich Borg
Assistant Research Professor
Daniel Egger
Executive in Residence and Director, Center for Quantitative Modeling

脚本

The area under the receiver operating characteristic or ROC curve, also called the AUC, is the most widely used performance metric for binary classification models. Its two great strengths are, first, that AUC results do not change with changes in the incidents of the actual condition, nor is AUC affected by changes in the relative cost of the two different types of binary classification errors, false positives and false negatives. Therefore, when either future incidents or the cost of classification errors or both are unstable or cannot be known, the AUC is generally the best possible performance metric available. The AUC metric ranges from a minimum of one-half to a maximum of one. An AUC of one-half indicates total uncertainty about classification. An AUC of one-half is a result when guessing outcome are random. An AUC of one would be classification with zero errors, a theoretical ideal that no one expects you to meet. A normal good to very good area under the curve is typically in the .65 to .85 range. A common technique in data analysis is to develop a binary classification model by varying certain parameters so as to maximize the AUC on a sample set of data with known outcomes, often called a training set. You will calculate your own area under the curve on a training set and apply this technique for the final course project. This video is a quick tutorial on how to calculate for a given classification model and collection of events with known outcomes, the resulting area under the curve. You should follow along using the review of AUC curve Excel spreadsheet, which I will now turn to. To make an ROC curve, we start with a collection of events. In the example spreadsheet, there are seven events numbered one through seven. In column B, rows 20 through 26. Each event must be assigned a score and an outcome if we are to generate an ROC curve. Our model is a procedure or algorithm that inputs known data about an event and outputs a single score. Here we focus on turning scores, and outcomes, into an ROC curve, so we aren't concerned with how the scores are generated. Our binary outcomes are represented by one, for positive, and zero, for negative. To generate an ROC curve, we need to know what the true outcomes are. And then we need to know how effective our model is at each possible threshold for positive classification. The way that we find this out is that we rank all of our events by score. And you'll notice this has already been done for you, but this procedure only works if you have a strict rank ordering by score of all of your events. And then you're going to proceed down your list, performing classification at each distinct possible threshold. The threshold that we're using, we will call the active threshold. The first threshold that we use is actually a number larger than or higher than the top score. At this threshold, everything is classified as negative. Therefore, there are no false positives, because nothing is positive. And there are no true positives, because nothing is classified positive. Okay, we're going to proceed through our list, and at the very bottom threshold, at a threshold of greater than or equal to -3, every event is classified as positive, and so we will have a total number of true positives, here three, equal to the actual number of positives in our collection. But all of the negative outcomes are classified incorrectly as false positives. What an ROC curve is is a measure of the effectiveness of the model at each possible threshold. In this case, there are eight thresholds from above the highest score to below or equal to the lowest score. We need to convert our total number of false positives and total number of true positives into a false positive rate and a true positive rate. To generate the false positive rate, we're going to divide the number of false positive's at the each particular threshold by the total true number of negatives or four. So let's just pick one particular example. Let's just pick a threshold. That is greater than or equal to one. There are three outcomes, two positive outcomes and a negative outcome. The negative outcome is incorrectly classified as positive, therefore there is one false positive, and the two actually positive outcomes are correctly classified, therefore there are two true positives. The false positive rate is the number of false positives, here one, divided by four, so one divided by four is .25. And the true positive rate is a number of true positive, here two, divided by the total number of positive outcomes, which is three, or two-thirds. This ordered pair one-quarter two-thirds is represented on our ROC curve by the point, .25, .67, here. And, what we're doing when we work our way down from the top, down to the bottom, is we're starting with the point where everything is negative, and we have a false positive rate of 0 and a true positive rate of 0. Then we are proceeding up and to the right, up and to the right. As we work our way down this list, until we arrive at the point 1,1 where everything is classified positive. So we have three over three equals one and four over four equals one. We next will sum the area under these rectangles. We have four rectangles 1, 2, 3, 4. Notice that only the x-axis value or the y-axis value can change between any two points. We never have both changing at the same time, because we are using narrow changes in threshold. So, there only one outcome at a time can be switch from a negative to positive, positive classification. So, only the false positive rate or true positive rate can changes outlooks. So, we sum the area of these rectangles to get an idea of the overall strength of our classification model. So we're going to take the width of each times its height, so the first rectangle has a width of .25 minus 0 and has a height of .67. The second rectangle has a width of .5 minus .25 and it has a height of one. Third rectangle has a width of .75 minus .5, and it has a width of one, and the last rectangle has a width of 1 minus .75, and it has a height of one. And these areas are equal to 0.917, which appears here. And that is how you calculate the area under the ROC curve. I'd like to point out that each threshold has its own confusion matrix, and these confusion matrixes are provided in your spreadsheet. So we have outcomes that do not change. Where we have three true positive outcomes and four actually negative outcomes. But our classifications are shifting. So each time we move down our list, we are shifting one item, Over two from a negative to a positive classification. So next we're going to take the item two, and we're shift it from a negative to a positive classification. And you see that one of our items here will change. Only one can change at a time. Okay? And then we look at the false positive rate and true positive rate. And that is what gives us our points on our ROC Curve. I hope this has been instructive. Good luck in generating your own ROC Curves. I think you'll find that this is one of the most useful tools provided in this course.

探索我们的目录

免费加入并获得个性化推荐、更新和优惠。

Coursera

关于
管理团队
工作机会
目录
证书
MasterTrack™ 证书
学位
企业版
政府版
面向校园

社区

学生
合作伙伴
开发者
Beta 测试人员
专业译员
博客
技术博客

更多

条款
隐私
帮助
内容访问
媒体
联系我们
目录
附属公司

随时随地学习

关注我们

© 2020 Coursera Inc.保留所有权利。