SAS软件应用之一致性检验kappa
一致性检验kappa讲课文档
本章小节
❖ 对两法测定结果一致部分进行检验，看一致 部分是否是由偶然因素影响的结果，它叫做 “一致性检验”，也称Kappa检验。说明两 种方法测定结果的实际一致率与随机一致率 之间的差别是否具有显著性意义。
第十六页，共18页。
本章小节
❖ SAS过程中也是用FREQ过程进行一致性检 验，只需要在TABLES语句中添加agree选项 即可输出Kappa值，但是要进行一致性检验， 需要编写其它程序语句。双向有序属性相同 的R×C表中的两分类变量皆为有序且属性相 同。实际上是2×2配对设计的扩展，此时宜 用一致性检验（或称Kappa检验）。
第三页，共18页。
Kappa检验
❖ Kappa是评价一致性的测量值。检验是否沿对角线格子 中的计数（接收比率一样的零件）与那些仅是偶然的期 望不同。设Po =对角线单元中观测值的总和，Pe=对角 线单元中期望值的总和。则Kappa =（Po - Pe）/（1 Pe）。Kappa是测量而不是检验。其大小用一个渐进和 标准误差构成的t统计量决定。一个通用的经验法则是 Kappa大于0.75表示好的一致性（Kappa最大为1）； 小于0.4表示一致性差。Kappa不考虑评价人间的意见 不一致性的程度，只考虑他们一致与否。
第八页，共18页。
KAPPA值的计算及检验
❖ 在诊断试验的研究中，数据资料多为双向有 序的列联表资料，即两个变量都是有序变量， 而且属性相同。属性相同分为三种情况，一 种情况是属性、分级水平数和分级水平都完 全相同。如甲医生和乙医生都把病人的检查 结果分为1、2、3、4四个等级。此时可直接 作Kappa检验。当这两个变量都只有2个水平 时，就成为配对设计的四格表资料，可使用 配对χ2检验，即McNemar检验。
一致性检验
kappa值的计算及检验
• 对两法测定结果一致部分进行检验，看一致部分是否是由偶然因 素影响的结果，它叫做一致性检 验，也称Kappa检验。说明两种 方法测定结果的实际一致率与随机一致率之间的差别是否具有显 著性意义。需要计算反映两法一致性程度高低的系数， 叫做 Kappa统计量。具体公式如下：
Kappa=(P0-Pe)/(1-Pe) ,P0=(a+d)/n ,Pe=［(a+b)(a+c)+(c+d)(b+d)］/n² • P0为实际一致率，Pe为理论一致率。
kappa值的计算及检验
• Kappa是一个统计量，也有抽样误差，其渐进标准误（ASE）。由 于U=Kappa/ASE近似服从标准正态分布，故可借助正态分布理论。 H0：Kappa=0, H1：Kappa≠0.如果 拒绝HO认为两种方法具有较高 的ppa值判断一致性的建议参考标准为： • Kappa=+1，说明两次判断的结果完全一致； • Kappa=-1，说明两次判断的结果完全不一致； • Kappa=0,说明两次判断的结果是机遇造成； • Kappa<0,说明一致程度比机遇造成的还差，两次检查结果很不一致，但在
实际应用中无意义； • Kappa>0,此时说明有意义，Kappa愈大，说明一致性愈好； • Kappa≧0.75，说明已经取得相当满意的一致程度； • Kappa〈0.4，说明一致程度不够理想；
Kappa检验
Kappa是评价一致性的测量值，检验是否沿对角线格子中的计数 （接收比率一样的零件）与那些仅是偶然的期望不同。设Po=对角 线单元中观测值的总和，Pe=对角线单元中期望值的总和。则 Kappa= （Po-Pe）/（1=Pe）。Kappa是测量而不是检验其大小用一 个渐进和标准误差构成的t统计量决定。 一个通用的经验法则是 Kappa大于0.75表示好的一 致性（Kappa最大为1）；小于0.4表示一 致性差、 Kappa不考虑评价人间的意见不一致性的程度，只考虑他 们一致与否。
配对卡方检验及Kappa检验(一致性检验)
一、配对卡方检验把每一份样本平均分成两份，分别用两种方法进行化验，比较此两种化验方法的结果（两类计数资料）是否有本质的不同；或者分别采用甲、乙两种方法对同一批病人进行检查，比较此两种检查方法的结果（两类计数资料）是否有本质的不同，此时要用配对卡方检验。
操作方法：单击【Statistics钮】，在弹出的Statistics对话框中选择McNemanr复选框，进行McNemanr检验。
即配对卡方检验，只能针对方形表格进行。
不能给出卡方值，只能给出P值。
二、一致性检验（Kappa检验）诊断试验的一致性检验经常用在下列两种情况中：一种是评价待评价的诊断实验方法与金标准的一致性；另一种是评价两种化验方法对同一个样本（化验对象）的化验结果的一致性或两个医务工作者对同一组病人的诊断结论的一致性或同一医务工作者对同一组病人前后进行两次观察作出的诊断的一致性等等。
Kappa值即内部一致性系数(inter-rater,coefficient of internal consistency)，是作为评价判断的一致性程度的重要指标。
取值在0～1之间。
Kappa≥两者一致性较好；>Kappa≥两者一致性一般；Kappa<两者一致性较差。
操作方法：单击【Statistics钮】，在弹出的Statistics对话框中选择Kappa 复选框。
计算Kappa值。
如果选择Risk复选框，则计算OR值（比数比）和RR值（相对危险度）。
病例对照研究（case control study）是主要用于探索病因的一种流行病学方法。
它是以某人群内一组患有某种病的人（称为病例）和同一人群内未患这种病但在与患病有关的某些已知因素方面和病例组相似的人（称为对照）作为研究对象；调查他们过去对某个或某些可疑病因（即研究因子）的暴露有无和（或）暴露程度（剂量）；通过对两组暴露史的比较，推断研究因子作为病因的可能性：如果病例组有暴露史者或严重暴露者的比例在统计学上显著高于对照组，则可认为这种暴露与患病存在统计学联系，有可能是因果联系。
Kappa系数：一种衡量评估者间一致性的常用方法
Kappa系数：一种衡量评估者间一致性的常用方法Biostatistics in psychiatry (25)?Kappa coefficient: a popular measure of rater agreementWan TANG 1*, Jun HU 2, Hui ZHANG 3, Pan WU 4, Hua HE 1,51 Department of Biostatistics and Computational Biology, University of Rochester, Rochester, NY , United States 2College of Basic Science and Information Engineering, Yunnan Agricultural University, Kunming, Yunnan Province, China 3Department of Biostatistics, St. Ju de Children’s Research Hospital, Memphis, TN, United States 4Value Institute, Christiana Care Health System, Newark, DE, United States 5Center of Excellence for Suicide Prevention, Canandaigua VA Medical Center Canandaigua, NY , United States *correspondence:***********************.eduA full-text Chinese translation of this article will be available at /cn on March 25, 2015.Summary: In mental health and psychosocial studies it is often necessary to report on the between-rater agreement of measures used in the study. This paper discusses the concept of agreement, highlighting its fundamental difference from correlation. Several examples demonstrate how to compute the kappa coefficient – a popular statistic for measuring agreement – both by hand and by using statistical software packages such as SAS and SPSS. Real study data are used to illustrate how to use and interpret this coefficient in clinical research and practice. Thearticle concludes with a discussion of the limitations of the coefficient. Keywords: interrater agreement; kappa coefficient; weighted kappa; correlation [Shanghai Arch Psychiatry . 2015; 27(1): 62-67. doi: 10.11919/j.issn.1002-0829.215010]1. IntroductionFor most physical illnesses such as high blood pressure and tuberculosis, definitive diagnoses can be made using medical devices such as a sphygmomanometer for blood pressure or an X-ray for tuberculosis. However, there are no error-free gold standard physical indicators of mental disorders, so the diagnosis and severity of mental disorders typically depends on the use of instruments (questionnaires) that attempt to measure latent multi-faceted constructs. For example, psychiatric diagnoses are often based on criteria specified in the Fourth edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-IV)[1], published by the American Psychiatric Association. But different clinicians may have different opinions about the presence or absence of the specific symptoms required to determine the presence of a diagnosis, so there is typically no perfect agreement between evaluators. In this situation, statistical methods are needed to address variability in clinicians’ ratings.Cohen’s kappa is a widely used index for assessing agreement between raters.[2] Although similar in appearance, agreement is a fundamentally different concept from correlation. To illustrate, consider an instrument with six items and suppose that two raters’ ratings of the six items on a single subject are (3,5), (4,6), (5,7), (6,8), (7,9) and (8,10). Although the scores of the two raters are quite different, the Pearson correlationcoefficient for the two scores is 1, indicating perfectcorrelation. The paradox occurs because there is a bias in the scoring that results in a consistent difference of 2 points in the scores of the two raters for all 6 items in the instrument. Thus, although perfectly correlated (precision), there is quite poor agreement between the two raters. The kappa index, the most popular measure of rate rs’ agreement, resolves this problem by assessing both the bias and the precision between raters’ ratings.In addition to its applications to psychiatric diagnosis, the concept of agreement is also widely applied to assess the utility of diagnostic and screening tests. Diagnostic tests provide information about a patient’s condition that clinicians’ often use when making decisions about the management of patients. Early detection of disease or of important changes in the clinical status of patients often leads to less suffering and quicker recovery, but false negative and false positive screening results can result in delayed treatment or in inappropriate treatment. Thus when a new diagnostic or screening test is developed, it is critical to assess its accuracy by comparing test results with those from a gold or reference standard. When assessing such tests, it is incorrect to measure the correlation of the results of the test and the gold standard, the correct procedure is to assess the agreement of the test results with the gold standard.2. ProblemsConsider an instrument with a binary outcome, with ‘1’ representing the presence of depression and ‘0’ representing the absence of depression. Suppose two independent raters apply the instrument to a random sample of n subjects. Let and denote the ratings on the n subjects by the two raters for i =1,2,...,n . We are interested in the degree of agreement between the two raters. Since the ratings are on the same scale of two levels for both raters, the data can be summarized in a 2×2 contingency table.To illustrate, Table 1 shows the results of a study assessing the prevalence of depression among 200 patients treated in a primary care setting using two methods to determine the presence of depression;[3] one based on information provided by the individual (i.e., proband) and the other based on information provided by another informant (e.g., the subject’s family member or close friend) about the proband. Intuitively, we may think that the proportion of cases in which the two ratings are the same (in this example, 34.5% [(19+50)/200]) would be a reasonable measure of agreement. But the problem with this proportion is that it is almost always positive, even when the rating by the two methods is completely random and independent of each other. So the proportion of overall agreement does not indicate whether or not two raters or two methods of rating are in agreement.by chance. This chance agreement must be removed in order to provide a valid measure of agreement. Coh en’s kappa coefficient is used to assess the level ofagreement beyond chance agreement.depressed 5065115total11684200For example, suppose that two raters with no training or experience about depression randomly decide whether or not each of the 200 patients has depression. Assume that one rater makes a positive diagnosis (i.e., considers depression present) 80% of the time and the other gives a positive diagnosis 90% of the time. Based on the assumption that their diagnoses are made independently from each other, Table 2 represents the joint distribution of their ratings. The proportion that the two raters give the same diagnosis is 74% (i.e., 0.72+0.02), suggesting that the two raters are doing a good job of diagnosing the presence of depression. But this level of agreement is purely by chance, it does not reflect the actual degree of agreement between the two raters. This hypothetical example shows that the proportion of cases in which two raters give the sameratings on an instrument is inflated by the agreementnegative 0.180.02 0.20total0.900.101.003. Kappa for 2×2 tablesConsider a hypothetical example of two raters giving ratings for n subjects on a binary scale, with ‘1’ representing a positive result (e.g., the presence of a diagnosis) a nd ‘0’ representing a negative result (e.g., the absence of a diagnosis). The results could be reported in a 2x2 contingency table as shown in Table 3. By convention, the results of the first rater are traditionally shown in the rows (x values) and the results of the second rater are shown in the columns (y values). Thus, n ij in the table denotes the number of subjects who receive the rating of i from the first rater and the rating j from the second rater. Let Pr(A) denote the probability of event A; then p ij =Pr(x=i,y=j) represent the proportion of all cases that receive the rating of i from the first rater and the rating j from the second rater, p i+=Pr(x=i) represents the marginal distribution of the first rater’s ratings, and p +j =Pr(y=j) represents themarginal distribution of the second rater’s ratings.11101+0 (negative)n 01n 00n 0+totaln +1n +0nIf the two raters give their ratings independently according to their marginal distributions, the probability that a subject is rated 0 (negative) by chance by both raters is the product of the marginal probabilities p 0+ and p +0. Likewise, the probability of a subject being rated 1 (positive) by chance by both raters is theproduct of the marginal probabilities p 1+ and p +1. The sum of these two probabilities (p 1+*p +1 + p 0+*p +0) is the agreement by chance, that is, the source of inflation discussed earlier.After excluding this source of inflation from the total proportion of cases in which the two raters give identical ratings (p 11 + p 00), we arrive at the agreement corrected for chance agreement, (p 11+p 00 –(p 1+*p +1 + p 0+*p +0)). In 1960 Cohen [1] recommended normalizing this chance-adjusted agreement as the Kappa coefficient (K ):(1)This normalization process produces kappa coefficients that vary between ?1 and 1, depending on the degree of agreement or disagreement beyond chance. If the two raters completely agree with each other, then p 11+p 00=1 and K =1. Conversely, if the kappa coefficient is 1, then the two raters agree completely. On the other hand, if the raters rate the subjects in a completely random fashion, then the agreement is completely due to chance, so p 11=p 1+*p +1 and p 00=p 0+*p +0 do (p 11+p 00 – (p 1+*p +1 + p 0+*p +0))=0 and the kappa coefficient is also 0. In general, when rater agreement exceeds chance agreement the kappa coefficient is positive, and when raters disagree more than they agree the kappa coefficient is negative. The magnitude of kappa indicates the degree of agreement or disagreement.The kappa coefficient can be estimated by substituting sample proportions for the probabilities shown in equation (1). When the number of ratings given by each rater (i.e., the samplesize) is large, the kappa coefficient approximately follows a normal distribution. This asymptotic distribution can be estimated using delta methods based on the asymptotic distributions of the various sample proportions.[4] Based on the asymptotic distribution, calculations of confidence intervals and hypothesis tests can be performed. For a sample with 100 or more ratings, this generally provides a good approximation. However, it may not work well for small sample sizes, in which case exact methods may be applied to provide more accurate inference.[4]Example 1. Assessing the agreement between the diagnosis of depression based on information provided by the proband compared to the diagnosis based on information provided by other informants (Table 1), the Kappa coefficient is computed as follows:The asymptotic standard error of kappa is estimated as 0.063. This gives a 95% confidence interval of κ, (0.2026, 0.4497). The positive kappa indicates some degree of agreement about the diagnosis of depression between diagnoses based on information provided by the proband versus diagnoses based on information provided by other informants. However, the level of agreement, though statistically significant, is relatively weak.In most applications, there is usually more interest in the magnitude of kappa than in the statistical significance of kappa. When the sample is relatively large (as in this example), a low kappa which represents relatively weak agreement can, nevertheless, be statistically significant (that is, significantly greater than 0). The degree of beyond-chance agreement has been classified in different ways by different authors who arbitrarily assigned each category to specific cutoff levels of Kappa. For example, Landis and Koch [5] proposed that a kappa in the range of 0.21–0.40 be considered ‘fair’ agreement, kappa=0.41–0.60 be c onsidered ‘moderate’ agreement, kappa=0.61–0.80 be considered ‘substantial’ agreement, and kappa >0.81 be considered ‘almost perfect’ agreement.4. Kappa for categorical variables with multiple levels The kappa coefficient for a binary rating scale can be generalized to cases in which there are more than two levels in the rating scale. Suppose there are k nominal categories in the rating scale. For simplicity and without loss of generality, denote the rating levels by 1,2,...,k . The ratings from the two raters can be summarized in a k ×k contingency table, as shown in Table 4. In the table, n ij , p ij , p i+, and p +j have the same interpretations as in the 2x2 contingency table (above) but the range of the scale is extended to i,j =1,…,k. As in the binary example, we first compute the agreement by chance, (the sum of the products of the k marginal probabilities, ∑ p i+*p +i for i =1,…,k ), and subtract this chance agreement from the total observed agreement (the sum of the diagonal probabilities, ∑ p ii fo r i =1,...,k ) before estimating the normalized agreement beyond chance:(2)11121k 1+2n 21n 22...n 2k n 2+..................k n k1n k2...n kk nk+totaln +1n +2...n +knAs in the case of binary scales, the kappa coefficient varies between ?1 and 1, depending on the extent of agreement or disagreement. If the two raters completely agree with each other (∑ p ii =1, for i =1,…,k ), then the kappa coefficient is equal to 1. If the raters rate the subjects at random, then the total agreement is equal chance agreement (∑ p ii =∑ p i+*p +i , for i =1,…,k ) so thekappa coefficient is 0. In general, the kappa coefficient is positive if there is agreement or negative if there is disagreement, with the magnitude of kappa indicating the degree of such agreement or disagreement between the raters. The kappa index in equation (2) is estimated by replacing the probabilities with their corresponding sample proportions. As in the case of binary scales, we can use asymptotic theory and exact methods to assess confidence intervals and make inferences.5. Kappa for ordinal or ranked variablesThe definition of the kappa coefficient in equation (2) assumes that the rating categories are treated as independent categories. If, however, the rated categories are ordered or ranked (for example, a Likert scale with categories such as ‘strongly disagree’, ‘disagree’, ‘neutral’, ‘agree’, and ‘strongly agree’), then a weighted kappa coefficient is computed that takes into consideration the different levels of disagreement between categories. For example, if one rater‘strongly disagrees’ and another ‘strongly agrees’ this must be considered a greater level of disagreement than when one rater ‘agrees’ and another ‘strongly agrees’.The first step in computing a weighted kappa is to assign weights representing the different levels of agreement for each cell in the KxK contingency table. The weights in the diagonal cells are all 1 (i.e., w ii =1, for all i ), and the weights in the off-diagonal cells range from 0 to <1 (i.e., 0<="" degrees="" disagreement="" equation="" for="" generate="" i="" ij="" kappa="" of="" or="" p="" ranked="" that="" the="" then="" these="" to="" varying="" weighted="" weights="" ≠j=""> The weighted kappa is computed by replacing the probabilities with their respective sample proportions, p ij , p i+, and p +i . If w ij =0 for all i ≠j , the weighted kappa coefficient K w reduces to the standard kappa in equation (2). Note that for binary rating scales, there is no weighted version of kappa, since κ remains the same regardless of the weights used. Again, we can use asymptotic theory and exact methods to estimate confidence intervals and make inferences.In theory, any weights satisfying the two defining conditions (i.e., weights in diagonal cells=1 and weights in off-diagonal cells >0 and <1) may be used. In practice, however, additional constraints are often imposed to make the weights more interpretable and meaningful. For example, since the degree of disagreement (agreement) is often a function of the difference between the i th and j th rating categories, weights are typically set to reflect adjacency between rating categories, such as by w ij =f (i-j ), where f is some decreasing function satisfying three conditions: (a) 0<="" larger="" on="" p="" these="" to="" used="" weights="">weights of pairs of categories that are closer to each other and smaller weights (i.e., closer to 0) are used for weights of pairs of categories that are more distant from each other.Two such weighting systems based on column scores are commonly employed. Suppose the column scores are ordered, say C 1≤C 2…≤C r and assigned values of 0,1,…r. Then, the Cicchetti–Allison weight and the Fleiss–Cohen weight in each cell of the KxK contingency table are computed as follows: Cicchetti-Allison weights: Fleiss-Cohen weights:Example 2. If depression is categorized into three ranked levels as shown in Table 5, the agreement of the classification based on information provided by the probands with the classification based on information provided by other informants can be estimated using the unweighted kappa coefficient as follows:Applying the Cicchetti-Allison weights (shown in Table 5) to the unweighted formula generates a weighed kappa: Applying the Fleiss-Cohen weights (shown in Table 5) involves replacing the 0.5 weight in the above equation with 0.75 and results in a K w of 0.4482. Thus the weighted kappa coefficients have larger absolute values than the unweighted kappa coefficients. The overall result indicates only fair to moderate agreement between the two methods of classifying the level of depression. As seen in Table 5, the low agreement is partly due to the fact that a large number of subjects classified as minor depression based on information from the proband were not identified using information from other informants.6. Statistical SoftwareSeveral statistical software packages including SAS, SPSS, and STATA can compute kappa coefficients. But agreement dataconceptually result in square tables with entries in all cells, so most software packages will not compute kappa if the agreement table is non-square, which can occur if one or both raters do not use all the rating categories when rating subjects because ofbiases or small samples.In some special circumstances the software pack-ages will compute incorrect kappa coefficients if a square agreement table is generated despite the failure of both raters to use all rating categories. For example, suppose a scale for rater agreement has three categories, A, B, and C. If one rater only uses categories B and C, and the other only uses categories A and B, this could result in a square agreement table such as that shown in Table 6. This is a square table, but the rating categories in the rows are completely different from those represented by the column. Clearly, kappa values generated using this table would not provide the desired assessment of rater agreement. To deal with this problem the analyst must add zero counts for the rating categories not endorsed by the raters to create a square tablewith the right rating categories, as shown in Table 7.6.1 SASI n SAS, one may use PROC FREQ and specify thecorresponding two-way table with the “AGREE” option.B 51419total211637B 051419C 0000total211637Here are the sample codes for Example 2 using PROC FREQ:PROC FREQ DATA = (the data set for the depression diagnosis study);TABLE (variable on result using proband) * (variable on result using other informants)/ AGREE;RUN;PROC FREQ uses Cicchetti-Allison weights by default. One can specify (WT=FC) with the AGREE option to request weighted kappa coefficients based on Fleiss-Cohen weights. It is important to check the order of the levels and weights used in computing weighted kappa. SAS calculates weights for weighted kappa based on unformatted values; if the variable of interest is not coded this way, one can either recode the variable or use a format statement and sp ecify the “ORDER = FORMATTED” option. Also note that data for contingency tables are often recorded as aggreg ated data. For example, 10 subjects with the rating ‘A’ from the first rater and the rating ‘B’ from the second rater may be combined into one observation with a frequency variable of value 10. In such cases a weight statement “weight (the frequency variab le);” may be applied to specify the frequency variable.6.2 SPSSIn SPSS, kappa coefficients can be only be computed when there are only two levels in the rating scale so it is not possible to compute weighted kappa coefficients. For a two-level rating scale such as that described in Example 1, one may use the following syntax to compute the kappa coefficient:CROSSTABS/TABLES=(variable on result using proband) BY (variable on result using other informants)/STATISTICS=KAPPA.An alternatively easier approach is to select appropriate options in the SPSS menu:1. Click on Analyze, then Descriptive Statistics, then Crosstabs.2. Choose the variables for the row and column variables in the pop-up window for the crosstab.3. Click on Statistics and select the kappa checkbox.4. Click Continue or OK to generate the output for the kappa coefficient.7. DiscussionIn this paper we introduced the use of Cohen’s kappa coefficient to assess between-rater agreement, which has the desirable property of correcting for chance agreement. We focused on cross-sectional studies for two raters, but extensions to longitudinal studies with missing values and to studies that use more than two raters are also available.[6] Cohen’s kappa generally works well, but in some specific situations it may not accurately reflect the true level of agreement between raters.[7]. For example, when both raters report a very high prevalence of the condition of interest (as in the hypothetical example shown in Table 2), some of the overlap in their diagnoses may reflect their common knowledge about the disease in the population being rated. This should be considered ‘true’ agreement, but it is attributed to chance agreement (i.e., kappa=0). Despite such limitations, the kappa coefficient is an informative measure of agreement in most circumstances that is widely used in clinical research.Cohen’s kappa can only be applied to categorical ratings. When ratings are on a continuous scale, Lin’s concordance correlation coefficient[8] is an appropriate measure of agreement between two raters,[8] and the intraclass correlation coefficients[9] is an appropriate measure of agreement between multiple raters.Conflict of interestThe authors declare no conflict of interest.FundingNone.概述：在精神卫生和社会心理学研究中，常常需要报告研究使用某一评估方法的评估者间的一致性。
第22章 一致性检验kappa PPT课件
KAPPA值的计算及检验
Kappa是一个统计量，也有抽样误差，其渐
进标准误（ASE）。由于u＝Kappa/ASE近 似服从标准正态分布，故可借助正态分布理 论。H0：Kappa＝0，H1：Kappa≠0。如果 拒绝H0认为两种方法具有较高的一致性。
KAPPA值的计算及检性相同的分级水平数相同，但分级水平不全相同。 如甲医生和乙医生都把病人的检查结果分为四个等级，但甲 医生的分级为1、2、3、4，而乙医生的分级为2、3、4、5。 在这种情况下，由于列联表的行数和列数仍然是一致的，即 列联表仍为方表，所以也可计算出相应的Kappa统计量。第 三种是属性相同，但分级水平数和分级水平不全相同。这种 情况就是我们所说的列联表的行列数不一致。由于收集上来 的数据不能轻易删除掉，所以我们考虑添加行或列使联表成 为方表。如行数为n，例数为n-1，则我们只需要添加第n列， 在第n行第n列的格点中添加权值0001，而第n行的其它格点 均设为0，就可以命名其成为方表，并计算Kappa统计量了。 由于权值系数很小，所以不会影响Kappa值的计算结果。
KAPPA值的计算及检验
对两法测定结果一致部分进行检验，看一致部分是 否是由偶然因素影响的结果，它叫做“一致性检 验”，也称Kappa检验。说明两种方法测定结果的 实际一致率与随机一致率之间的差别是否具有显著 性意义。需要计算反映两法一致性程度高低的系数， 叫做Kappa统计量。具体公式如下： Po Pe ad (a b)(a c) (c d )(b d ) Kappa , Po , Pe 1 Pe n n2
二分类资料一致性分析
前面我们已经介绍四格表资料的2检验，本
节需要介绍的是Kappa检验。那么Kappa检 验与配对2检验有什么区别呢？Kappa检验 重在检验两者的一致性，配对2检验重在检 验两者间的差异。对同一样本数据，这两种 检验可能给出矛盾的结论。主要原因是两者 对所提供的有统计学意义的结论要求非常严 格所致。
一致性检验kappa科恩
二分类资料一致性分析
前面我们已经介绍四格表资料的2检验，本节需要介绍 的是Kappa检验。那么Kappa检验与配对2检验有什 么区别呢？Kappa检验重在检验两者的一致性，配对2 检验重在检验两者间的差异。对同一样本数据，这两种 检验可能给出矛盾的结论。主要原因是两者对所提供的 有统计学意义的结论要求非常严格所致。
KAPPA值的计算及检验
对两法测定结果一致部分进行检验，看一致部分是否是 由偶然因素影响的结果，它叫做“一致性检验”，也称
Kappa检验。说明两种方法测定结果的实际一致率与随 机一致率之间的差别是否具有显著性意义。需要计算反
映两法一致性程度高低的系数，叫做Kappa统计量。具 体公式如下：
Kappa
Kappa检验
Kappa是评价一致性的测量值。检验是否沿对角线格子 中的计数（接收比率一样的零件）与那些仅是偶然的期 望不同。设Po =对角线单元中观测值的总和，Pe=对角 线单元中期望值的总和。则Kappa =（Po - Pe）/（1 Pe）。Kappa是测量而不是检验。其大小用一个渐进和 标准误差构成的t统计量决定。一个通用的经验法则是 Kappa大于0.75表示好的一致性（Kappa最大为1）； 小于0.4表示一致性差。Kappa不考虑评价人间的意见 不一致性的程度，只考虑他们一致与否。
一致性检验 kappa
学习目标
熟悉Kappa值的判断标准； 掌握Kappa值的计算以及检验方法； 掌握二分类资料和有序分类资料的一致性分析；
Kappa系数在一致性评价中的应用研究
象2009,3(4)
空气质量预报是复杂的系统工程,也是环境科学研究的热点和难点所在.通过文献综述分析了现有研究的不足,指出现有的研究没有考虑由于偶然性和随机性导致的一致性.基于权重Kappa统计值的方法,在剔除了由于偶然性和随机性造成的一致性的基础上,对3种常用的空气质量预报方法的预测结果的一
(0.925±0.028)、第三掌骨(0.703±0.050)、第五掌骨(0.789±0.042)、第一近节指骨(0.964±0.021)、第三近节指骨(0.941±0.026)、第五近节指骨(0.887±0.034)、第三中节指骨(0.917±0.031)、第五中节指骨(0.919±0.030)、第一远节指骨(0.772±0.050)、第三远节指骨(0.883±0.040)、第五远节指骨(0.856±0.041).对13个Kappa值进行u检验,P值均小于0.05.②总体一致性:Kappa=0.776,标准误为0.044(u=16.128,P<0.05).结论 骨龄测定中单个骨骺核观察者间人工评级的一致性较高.实际工作中,应通过定期培训,提高评价观察者之间的一致性,修正诊断结果,提高诊断准确性.
致性进行了衡量,有利于提高对不同模型预测结果的差异性的认识,对进一步提高空气质量预报的准确率有一定的意义.
10.期刊论文华琳.阎岩.张建关于对诊断一致性Kappa系统的探讨-数理医药学杂志2006,19(5)
appa系数进行了分析和说明,由于Kappa系数仅适用于行数和列数相等的方表,针对Kappa检验的这一局限性,给出了行数和列数不一致时使用SPSS软件实现Kappa检验的方法.
[课件]SPSS一致性检验
![[课件]SPSS一致性检验](https://img.taocdn.com/s3/m/fffb3e3dc5da50e2524d7fd5.png)
分层因素在几个组之间的分布不均，既可
能削弱了原本存在的行变量与列变量之间的关系，
也可能使得原本不存在关系的两个变量的关系呈
现统计学显著性。
14.6分层卡方检验
例2：期末临近，各学院学生对授课老师网上评价逐渐开始。教育技术学 班长对同学们关于以下3位专业课老师周玉霞老师、邓鹏老师、和学 仁老师的网上评价进行了调查，其中一项指标是在课后是否经常向老 师寻求帮助，希望分析寻求帮助与性别间有无联系。
14.5.1 Kappa一致性检验
例1：云南师范大学信息学院教育技术学理学和教育学专业准备在明年 暑假做一个关于云南省乡级中学多媒体技术在教学中的使用情况调查，但 是对中学具体情况不太了解。于是，请周老师推荐了20所备选中学，理 学班长和教育学班长对这20所中学根据需要做了一个评价，把它们评为 好、中、差三个等级，以便确定应对哪些中学进行更进一步落实，那么理 学班长和教育学班长的评价结果是否一致？
Kappa≥0.75 两者一致性较好 0.75＞Kappa≥0.4 一致性一般 Kappa＜0.4 一致性较差 Kappa=0 两者完全无关 Kappa=1 两者完全一致
14.5.2配对卡方检验
McNemar配对卡方检验
14.6分层卡方检验
分层卡方检验是把对象分解成不同的层次，
每层分别研究行变量与列变量的相关。
14.6分层卡方检验
14.6分层卡方检验
14.6分层卡方检验
Cochran's and Mantel-Haenszel 卡方检验
14.6分层卡方检验
Hale Waihona Puke THANKS FOR YOUR LISTENING！
PPT模板下载：/moban/
SPSS一致性检验&配对卡 方检验&分层卡方检验
SAS软件应用之一致性检验kappa
Kappa检验
对于用Kappa值判断一致性的建议参考标准为： Kappa =+1，说明两次判断的结果完全一致； Kappa =-1，说明两次判断的结果完全不一致； Kappa =0，说明两次判断的结果是机遇造成； Kappa<0，说明一致程度比机遇造成的还差，两次检查结果 很不一致，但在实际应用中无意义； Kappa>0，此时说明有意义，Kappa愈大，说明一致性愈好； Kappa≥0.75，说明已经取得相当满意的一致程度； Kappa<0.4，说明一致程度不够理想；
KAPPA值的计算及检验
对两法测定结果一致部分进行检验，看一致部分是 否是由偶然因素影响的结果，它叫做“一致性检 验”，也称Kappa检验。说明两种方法测定结果的 实际一致率与随机一致率之间的差别是否具有显著 性意义。需要计算反映两法一致性程度高低的系数， 叫做Kappa统计量。具体公式如下： Po Pe ad (a b)(a c) (c d )(b d ) Kappa , Po , Pe 1 Pe n n2
有序分类资料一致性分析
单向有序R×C表
有两种形式。一种是R×C 表中的分组变量是有序的，而指标变量是无 序的。此种单向有序R×C表资料可用行×列 表资料的2检验进行分析。另一种情况是 R×C表中的分组变量是无序的，而指标变量 是有序的，此种单向有序R×C表资料宜用秩 和检验进行分析。
P0为实际一致率，Pe为理论一致率。
KAPPA值的计算及检验
Kappa是一个统计量，也有抽样误差，其渐
进标准误（ASE）。由于u＝Kappa/ASE近 似服从标准正态分布，故可借助正态分布理 论。H0：Kappa＝0，H1：Kappa≠0。如果拒 绝H0认为两种方法具有较高的一致性。
第22章 一致性检验KAPPA【SAS从入门到精通】
KAPPA值的计算及检验
• 对两法测定结果一致部分进行检验，看一致部分 是否是由偶然因素影响的结果，它叫做“一致性 检验”，也称Kappa检验。说明两种方法测定结 果的实际一致率与随机一致率之间的差别是否具 有显著性意义。需要计算反映两法一致性程度高 低的系数，叫做Kappa统计量。具体公式如下：
KAPPA值的计算及检验
• 另一方面，如果两个变量中有一个变量是金标准， 那么我们不但能分析出检验结果的一致性，还可 以计算出敏感度、特异度、误诊率和漏诊率等指 标。如果有不同的诊断分界点，还可以绘制出 ROC曲线。 • 诊断试验的评价在医学研究中具有十分重要的意 义，目前大多数文献都使用Kappa统计量来检验 结果的一致性。所以本研究主要是对Kappa系数 作一个探讨和分析。诊断试验评价的统计学方法 还会随着更多问题的提出和解决而不断得到发展、 修正和扩展。
有序分类资料一致性分析
• 单向有序R×C表 有两种形式。一种是R×C表中的分组变量是有 序的，而指标变量是无序的。此种单向有序R×C表资料可用行× 列表资料的2检验进行分析。另一种情况是R×C表中的分组变量 是无序的，而指标变量是有序的，此种单向有序R×C表资料宜用 秩和检验进行分析。
有序分类资料一致性分析
Kappa检验
• Kappa是评价一致性的测量值。检验是否沿对角 线格子中的计数（接收比率一样的零件）与那些 仅是偶然的期望不同。设Po =对角线单元中观测 值的总和，Pe=对角线单元中期望值的总和。则 Kappa =（Po - Pe）/（1 - Pe）。Kappa是测量而 不是检验。其大小用一个渐进和标准误差构成的 t统计量决定。一个通用的经验法则是Kappa大于 0.75表示好的一致性（Kappa最大为1）；小于0.4 表示一致性差。Kappa不考虑评价人间的意见不 一致性的程度，只考虑他们一致与否。
一致性检验之Kappa、ICC、kendall协调系数的差别
一致性检验之Kappa、ICC、kendall协调系数的差别展开全文一致性检验的目的在于比较不同方法得到的结果是否具有一致性。
检验一致性的方法有很多比如：Kappa检验、ICC 组内相关系数、Kendall W协调系数等。
每种方法的功能侧重，数据要求都略有不同：一致性检验Kappa系数检验，适用于两次数据（方法）之间比较一致性，比如两位医生的诊断是否一致，两位裁判的评分标准是否一致等。
ICC组内相关系数检验，用于分析多次数据的一致性情况，功能上与Kappa系数基本一致。
ICC分析定量或定类数据均可；但是Kappa一致性系数通常要求数据是定类数据。
Kendall W协调系数，是分析多个数据之间关联性的方法，适用于定量数据，尤其是定序等级数据。
进一步说明（1）Kappa检验Kappa检验分为简单Kappa检验和加权Kappa检验，两者的区别主要在于：如果研究数据是绝对的定类数据（比如阴性、阳性），此时使用简单Kappa系数；如果数据为等级式定类数据（比如轻度，中度，重度；也或者不同意，中立，同意）；此时可使用加权（线性）Kappa 系数。
应用举例两个医生分别对于50个病例进行MRI检查（MRI检查诊断共分三个等级，分别是轻度，中度和重度），对比两名医生检查结果诊断的一致性水平。
（1表示轻度，2表示中度，3表示重度）使用路径：SPSSAU→医学实验→KappaKappa系数结果表根据上表可知，两位医生对于MRI检查诊断结论具有较强（Kappa值=0.644）的一致性。
（2）ICC组内相关系数ICC组内相关系数可用于研究评价一致性，评价信度，测量复测信度（重测信度）等。
相对于Kappa系数，ICC组内相关系数的适用范围更广，适用于定量或者定类数据，而且可针对双样本或者多样本进行分析一致性。
但ICC的分析相对较为复杂，通常需要从三个方面进行分析并且选择最优的ICC模型；分别是模型选择，计算类型和度量标准。
- 1、下载文档前请自行甄别文档内容的完整性，平台不提供额外的编辑、内容补充、找答案等附加服务。
- 2、"仅部分预览"的文档,不可在线预览部分如存在完整性等问题,可反馈申请退款(可完整预览的文档不适用该条件!)。
- 3、如文档侵犯您的权益，请联系客服反馈,我们会尽快为您处理(人工客服工作时间：9:00-18:30)。
有序分类资料一致性分析
❖ R×C表可以分为双向无序、单向有序、双向 有序属性相同和双向有序属性不同4类。
❖ 双向无序R×C表 R×C表中两个分类变量皆 为无序分类变量，对于该类资料：①若研究 目的为多个样本率（或构成比）的比较，可 用行×列表资料的2检验；②若研究目的为 分析两个分类变量之间有无关联性以及关系 的密切程度时，可以用行×列表资料的2检 验以及Pearson列联系数进行分析。
KAPPA值的计算及检验
❖ 对两法测定结果一致部分进行检验，看一致部分是 否是由偶然因素影响的结果，它叫做“一致性检 验”，也称Kappa检验。说明两种方法测定结果的 实际一致率与随机一致率之间的差别是否具有显著 性意义。需要计算反映两法一致性程度高低的系数， 叫做Kappa统计量。具体公式如下： K a P 1 o P p P e e,P o p a n a d ,P e ( a b )a (c ) n 2 ( c d )b (d )
二分类资料一致性分析
❖ 前面我们已经介绍四格表资料的2检验，本 节需要介绍的是Kappa检验。那么Kappa检 验与配对2检验有什么区别呢？Kappa检验 重在检验两者的一致性，配对2检验重在检 验两者间的差异。对同一样本数据，这两种 检验可能给出矛盾的结论。主要原因是两者 对所提供的有统计学意义的结论要求非常严 格所致。
KAPPA值的计算及检验
❖ 另一方面，如果两个变量中有一个变量是金标准， 那么我们不但能分析出检验结果的一致性，还可以 计算出敏感度、特异度、误诊率和漏诊率等指标。 如果有不同的诊断分界点，还可以绘制出ROC曲线。
❖ 诊断试验的评价在医学研究中具有十分重要的意义， 目前大多数文献都使用Kappa统计量来检验结果的 一致性。所以本研究主要是对Kappa系数作一个探 讨和分析。诊断试验评价的统计学方法还会随着更 多问题的提出和解决而不断得到发展、修正和扩展。
有序分类资料一致性分析
❖ 单向有序R×C表 有两种形式。一种是R×C 表中的分组变量是有序的，而指标变量是无 序的。此种单向有序R×C表资料可用行×列 表资料的2检验进行分析。另一种情况是 R×C表中的分组变量是无序的，而指标变量 是有序的，此种单向有序R×C表资料宜用秩 和检验进行分析。
有序分类资料一致性分析
❖ 双向有序属性不同的R×C表 R×C表中的两分类变 量皆为有序且属性不相同。对于该类资料，需要分 析两有序分类变量间是否存在线性变化趋势，宜用 有序分组资料的线性趋势检验。
❖ 双向有序属性相同的R×C表中的两分类变量皆为有 序且属性相同。实际上是2×2配对设计的扩展，此 时宜用一致性检验（或称Kappa检验）。
K数相同，但分级水平不全相同。 如甲医生和乙医生都把病人的检查结果分为四个等级，但甲 医生的分级为1、2、3、4，而乙医生的分级为2、3、4、5。 在这种情况下，由于列联表的行数和列数仍然是一致的，即 列联表仍为方表，所以也可计算出相应的Kappa统计量。第 三种是属性相同，但分级水平数和分级水平不全相同。这种 情况就是我们所说的列联表的行列数不一致。由于收集上来 的数据不能轻易删除掉，所以我们考虑添加行或列使联表成 为方表。如行数为n，例数为n-1，则我们只需要添加第n列， 在第n行第n列的格点中添加权值0001，而第n行的其它格点 均设为0，就可以命名其成为方表，并计算Kappa统计量了。 由于权值系数很小，所以不会影响Kappa值的计算结果。
❖ P0为实际一致率，Pe为理论一致率。
KAPPA值的计算及检验
❖ Kappa是一个统计量，也有抽样误差，其渐 进标准误（ASE）。由于u＝Kappa/ASE近 似服从标准正态分布，故可借助正态分布理 论。H0：Kappa＝0，H1：Kappa≠0。如果拒 绝H0认为两种方法具有较高的一致性。
KAPPA值的计算及检验
❖ 在诊断试验的研究中，数据资料多为双向有 序的列联表资料，即两个变量都是有序变量， 而且属性相同。属性相同分为三种情况，一 种情况是属性、分级水平数和分级水平都完 全相同。如甲医生和乙医生都把病人的检查 结果分为1、2、3、4四个等级。此时可直接 作Kappa检验。当这两个变量都只有2个水平 时，就成为配对设计的四格表资料，可使用 配对χ2检验，即McNemar检验。
Kappa检验
❖ 对于用Kappa值判断一致性的建议参考标准为： ❖ Kappa =+1，说明两次判断的结果完全一致； ❖ Kappa =-1，说明两次判断的结果完全不一致； ❖ Kappa =0，说明两次判断的结果是机遇造成； ❖ Kappa<0，说明一致程度比机遇造成的还差，两次检查结果
很不一致，但在实际应用中无意义； ❖ Kappa>0，此时说明有意义，Kappa愈大，说明一致性愈好； ❖ Kappa≥0.75，说明已经取得相当满意的一致程度； ❖ Kappa<0.4，说明一致程度不够理想；
第22章 一致性检验kappa
Kappa检验
❖ Kappa是评价一致性的测量值。检验是否沿对角线 格子中的计数（接收比率一样的零件）与那些仅是 偶然的期望不同。设Po =对角线单元中观测值的总 和，Pe=对角线单元中期望值的总和。则Kappa = （Po - Pe）/（1 - Pe）。Kappa是测量而不是检验。 其大小用一个渐进和标准误差构成的t统计量决定。 一个通用的经验法则是Kappa大于0.75表示好的一 致性（Kappa最大为1）；小于0.4表示一致性差。 Kappa不考虑评价人间的意见不一致性的程度，只 考虑他们一致与否。
❖ 所以，对于双向有序且属性相同的数据，我们可以 采用Kappa检验判断其一致性。
本章小节
❖ 1960年Cohen等提出用Kappa值作为评价判断的一 致性程度的指标。实践证明，它是一个描述诊断的 一致性较为理想的指标，因此在临床试验中得到广 泛的应用。Kappa是评价一致性的测量值。检验是 否沿对角线格子中的计数（接收比率一样的零件） 与那些仅是偶然的期望不同。设Po =对角线单元中 观测值的总和，Pe=对角线单元中期望值的总和。 则Kappa =（Po - Pe）/（1 - Pe）。Kappa是测量 而不是检验。其大小用一个渐进和标准误差构成的t 统计量决定。一个通用的经验法则是Kappa大于 0.75表示好的一致性（Kappa最大为1）；小于0.4 表示一致性差。