Measures of Interobserver Agreement

Content: INTRODUCTIONRELIABILITY FOR CONTINUOUS SCALE MEASUREMENTS Model for Reliability StudiesInference Procedures on the Index of Reliability for Case (1)Analysis of Method - Comparison StudiesComparing Reliability Coefficients MEASURES OF 2x2 ASSOCIATION AND AGREEMENT OF CROSS CLASSIFIED DATAIntroductionIndices of Adjusted Agreement Cohen's Kappa =Chance Corrected Measure of AgreementIntraclass KappaThe 2x2 Kappa in the Context of AssociationStratified Kappa Conceptual issues COEFFICIENTS OF AGREEMENT FOR MULTIPLE RATERS AND MULTIPLE CATEGORIESIntroductionMultiple Categories and Two RatersAgreement for Multiple Raters and Dichotomous ClassificationMultiple Raters and Multiple CategoriesTesting the Homogeneity of Kappa Statistic from Independent StudiesASSESSING AGREEMENT FROM DEPENDENTIntroductionDependent Dichotomous Assessments Adjusting for CovariatesLikelihood Based ApproachEstimating Equations ApproachLoglinear and Association ModelsAppendix I: Joint probability distribution of repeated dichotomous assessmentsAppendix II: Correlation between estimated  kappas SAMPLE SIZE REQUIREMENTS FOR THE DESIGN OF RELIABILITY STUDYIntroductionThe Case of Continuous MeasurementsThe Non-Normal CaseCost ImplicationsThe Case of Dichotomous AssessmentsBibliography