3.11.10. statsmodels.stats.inter_rater¶
Inter Rater Agreement
3.11.10.1. contains¶
fleiss_kappa cohens_kappa
- aggregate_raters:
- helper function to get data into fleiss_kappa format
- to_table:
- helper function to create contingency table, can be used for cohens_kappa
Created on Thu Dec 06 22:57:56 2012 Author: Josef Perktold License: BSD-3
3.11.10.2. References¶
- Wikipedia: kappa’s initially based on these two pages
- http://en.wikipedia.org/wiki/Fleiss%27_kappa http://en.wikipedia.org/wiki/Cohen’s_kappa
SAS-Manual : formulas for cohens_kappa, especially variances see also R package irr
3.11.10.3. TODO¶
standard errors and hypothesis tests for fleiss_kappa other statistics and tests,
in R package irr, SAS has more
- inconsistent internal naming, changed variable names as I added more
- functionality
- convenience functions to create required data format from raw data
- DONE
3.11.10.4. Functions¶
aggregate_raters(data[, n_cat]) |
convert raw data with shape (subject, rater) to (subject, cat_counts) |
cohens_kappa(table[, weights, ...]) |
Compute Cohen’s kappa with variance and equal-zero test |
fleiss_kappa(table) |
Fleiss’ kappa multi-rater agreement measure |
to_table(data[, bins]) |
convert raw data with shape (subject, rater) to (rater1, rater2) |