在机器学习中,常常需要对学习算法性能进行评估,自然需要建立一些评估准则。
参考http://www2.cs.uregina.ca/~dbd/cs831/notes/confusion_matrix/confusion_matrix.html中的描述,一个比较简单的描述是混淆矩阵(confusionmatrix)。它是这样定义的:
The entries in the confusion matrix have the following meaning inthe context of our study:
Predicted | |||
Negative | Positive | ||
Actual | Negative | a | b |
Positive | c | d |
Several standard terms have been defined for the 2 classmatrix:
[1]
[2]
[3]
[4]
[5]
[6]
The accuracy determined using equation 1 may not be an adequateperformance measure when the number of negative cases is muchgreater than the number of positive cases (Kubat et al., 1998).Suppose there are 1000 cases, 995 of which are negative cases and 5of which are positive cases. If the system classifies them all asnegative, the accuracy would be 99.5%, even though the classifiermissed all positive cases.
上一段话的意思是:使用accuracy评估分类器,效果可能不那么好,特别是负样本数目占较大比例时。
Other performance measures account for this byincludingTPin aproduct: for example,geometricmean(g-mean) (Kubat et al., 1998),as defined in equations 7 and 8,andF-Measure(Lewis andGale, 1994), as defined in equation 9.
[7]
[8]
[9]
In equation 9,bhas a value from 0to infinity and is used to control the weight assignedtoTPandP.Any classifier evaluated using equations 7, 8 or 9 will have ameasure value of 0, if all positive cases are classifiedincorrectly.