ROC曲线
相关主题
- 1、下载文档前请自行甄别文档内容的完整性，平台不提供额外的编辑、内容补充、找答案等附加服务。
- 2、"仅部分预览"的文档,不可在线预览部分如存在完整性等问题,可反馈申请退款(可完整预览的文档不适用该条件!)。
- 3、如文档侵犯您的权益，请联系客服反馈,我们会尽快为您处理(人工客服工作时间：9:00-18:30)。
``````
Hongjie Wu
contents
01 02
03
Basic concept
Plotting an ROC curve
AUC
Receiver
operating characteristic (ROC) curves are a useful visual tool for comparing two classification models.(比较两个分类模型有用的可视化工具) curve is created by plotting the true positive rate (TPR) against the false positive rate (FPR) at various threshold settings. （ROC曲线显示了真正例和假正例之间的权衡）
probabilistic classifier for each of the 10 tuples in a test set, sorted by decreasing probability order. Column 1 is merely a tuple identification number, which aids in our explanation. Column 2 is the actual class label of the tuple. There are five positive tuples and five negative tuples, thus and . As we examine the known class label of each tuple,we can determine the values of the remaining columns, TP, FP, TN, FN, TPR, and FPR. We start with tuple1, which has the highest probability score, and take that score as our threshold, that is, . Thus, the classifier considers tuple 1 to be positive, and all the other tuples are considered negative. Since the actual class label of tuple 1 is positive, we have a true positive, hence and . Among the remaining nine tuples, which are all classified as negative, five actually are negative (thus, ). The remaining four are all actually positive, thus, . We can therefore compute , while .Thus, we have the point for the ROC curve.
The
true
positive (TP真正例)诊断为有，实际上也有高血压 negative (TN真负例)诊断为没有，实际却没有高血压
eqv. with hit eqv. with correct rejection
true
false
positive (FP假正例)诊断为有，实际上也没有高血压
对于二类问题，
选择阈值t，
使得f(X)>=t的 元组X视为正，
而其它元组视为负
Figure 8.18 Tuples sorted by decreasing score, where the score is the value
returned by a probabilistic classifier.
以从这些点得到凸包。该图现实的对角线，对模型的每个真正例元组，都恰好遇到一个假正例，为了比较，这条直线代表
随机猜测
）.
trapezoid
ቤተ መጻሕፍቲ ባይዱ
method
简单地将每个相邻的点以直线连接，计算连线下方的总面
积。因为每一线段下方都是一个梯形，所以叫梯形法。
优点：简单，所以常用。 缺点：倾向于低估AUC。 AUCH of ROC是机器学习的社群最常使用来比较不同模型优 劣的方法 。然而近来这个做法开始受到质疑，因为有些机 器学习的研究指出， AUC 的噪声太多，并且很常求不出可 信又有效的 AUC 值，使得 AUC 在模型比较时产生的问题比 解释的问题更多 。
negative (FN假负例)诊断为没有，实际却有高血压
eqv. with false alarm, Type I error eqv. with miss, Type II error
false
sensitivity
or true positive rate (TPR)Y轴
eqv. with hit rate, recall
从AUC判断分类器（预测模型）优劣的标准：
AUC = 1，是完美分类器，采用这个预测模型时，存在至少一个阈值 能得出完美预测。绝大多数预测的场合，不存在完美分类器。 0.5 < AUC < 1，优于随机猜测。这个分类器（模型）妥善设定阈值 的话，能有预测价值。 AUC = 0.5，跟随机猜测一样（例：丢铜板），模型没有预测价值。 AUC < 0.5，比随机猜测还差；但只要总是反预测而行，就优于随机 猜测。
在所有实际为阳性的样本中，被正确地判断为阳性之比率
fall-out
or false positive rate (FPR)X轴
在所有实际为阴性的样本中，被错误地判断为阳性之比率
Figure
8.18
shows the probability value (column 3) returned by a
ROC AUC
To
assess the accuracy of a model, we can measure the area under the curve.（为了评估模型的准确率，可以测量曲线下方的面积）
Several
software packages are able to perform such calculation.（一些软件包可以用来进行这些计算）
Thank
you!!
(元组按递减得分排序，其中得分是概率分类器的返回值)
There are many methods to obtain a curve out of these points, the most common of which is to use a convex hull. The plot also shows a diagonal line where for every true positive of such a model, we are just as likely to encounter a false positive. For comparison, this line represents random guessing（许多方法可
Hongjie Wu
contents
01 02
03
Basic concept
Plotting an ROC curve
AUC
Receiver
operating characteristic (ROC) curves are a useful visual tool for comparing two classification models.(比较两个分类模型有用的可视化工具) curve is created by plotting the true positive rate (TPR) against the false positive rate (FPR) at various threshold settings. （ROC曲线显示了真正例和假正例之间的权衡）
probabilistic classifier for each of the 10 tuples in a test set, sorted by decreasing probability order. Column 1 is merely a tuple identification number, which aids in our explanation. Column 2 is the actual class label of the tuple. There are five positive tuples and five negative tuples, thus and . As we examine the known class label of each tuple,we can determine the values of the remaining columns, TP, FP, TN, FN, TPR, and FPR. We start with tuple1, which has the highest probability score, and take that score as our threshold, that is, . Thus, the classifier considers tuple 1 to be positive, and all the other tuples are considered negative. Since the actual class label of tuple 1 is positive, we have a true positive, hence and . Among the remaining nine tuples, which are all classified as negative, five actually are negative (thus, ). The remaining four are all actually positive, thus, . We can therefore compute , while .Thus, we have the point for the ROC curve.
The
true
positive (TP真正例)诊断为有，实际上也有高血压 negative (TN真负例)诊断为没有，实际却没有高血压
eqv. with hit eqv. with correct rejection
true
false
positive (FP假正例)诊断为有，实际上也没有高血压
对于二类问题，
选择阈值t，
使得f(X)>=t的 元组X视为正，
而其它元组视为负
Figure 8.18 Tuples sorted by decreasing score, where the score is the value
returned by a probabilistic classifier.
以从这些点得到凸包。该图现实的对角线，对模型的每个真正例元组，都恰好遇到一个假正例，为了比较，这条直线代表
随机猜测
）.
trapezoid
ቤተ መጻሕፍቲ ባይዱ
method
简单地将每个相邻的点以直线连接，计算连线下方的总面
积。因为每一线段下方都是一个梯形，所以叫梯形法。
优点：简单，所以常用。 缺点：倾向于低估AUC。 AUCH of ROC是机器学习的社群最常使用来比较不同模型优 劣的方法 。然而近来这个做法开始受到质疑，因为有些机 器学习的研究指出， AUC 的噪声太多，并且很常求不出可 信又有效的 AUC 值，使得 AUC 在模型比较时产生的问题比 解释的问题更多 。
negative (FN假负例)诊断为没有，实际却有高血压
eqv. with false alarm, Type I error eqv. with miss, Type II error
false
sensitivity
or true positive rate (TPR)Y轴
eqv. with hit rate, recall
从AUC判断分类器（预测模型）优劣的标准：
AUC = 1，是完美分类器，采用这个预测模型时，存在至少一个阈值 能得出完美预测。绝大多数预测的场合，不存在完美分类器。 0.5 < AUC < 1，优于随机猜测。这个分类器（模型）妥善设定阈值 的话，能有预测价值。 AUC = 0.5，跟随机猜测一样（例：丢铜板），模型没有预测价值。 AUC < 0.5，比随机猜测还差；但只要总是反预测而行，就优于随机 猜测。
在所有实际为阳性的样本中，被正确地判断为阳性之比率
fall-out
or false positive rate (FPR)X轴
在所有实际为阴性的样本中，被错误地判断为阳性之比率
Figure
8.18
shows the probability value (column 3) returned by a
ROC AUC
To
assess the accuracy of a model, we can measure the area under the curve.（为了评估模型的准确率，可以测量曲线下方的面积）
Several
software packages are able to perform such calculation.（一些软件包可以用来进行这些计算）
Thank
you!!
(元组按递减得分排序，其中得分是概率分类器的返回值)
There are many methods to obtain a curve out of these points, the most common of which is to use a convex hull. The plot also shows a diagonal line where for every true positive of such a model, we are just as likely to encounter a false positive. For comparison, this line represents random guessing（许多方法可