compare {CMA} | R Documentation |
Classifiers can be evaluated separately using the method
evaluation
. Normally, several classifiers
are used for the same dataset and their performance is
compared. This comparison procedure is essentially facilitated by
this method.
For S4 method information, s. compare-methods
compare(clresultlist, measure = c("misclassification", "sensitivity", "specifity", "average probability", "brier score", "auc"), aggfun = mean, plot = FALSE, ...)
clresultlist |
A list of lists (!) of objects of class cloutput or clvarseloutput .
Each inner list is usually returned by classification .
Additionally, the different list elements of the outer list
should have been created by different classifiers, s.
also example below. |
measure |
A character vector containing one or more of the elements listed below.
By default, all measures are computed, using evaluation
with scheme = "iterationwise" .
Note that "sensitivity", "specifity", "auc" cannot be computed
for the multiclass case.
|
aggfun |
Function that determines how performance among different iterations are aggregared.
Default is mean , other possible choices are quantiles. |
plot |
Should the performance of different classifiers be visualized by a joint boxplot ?
Default is FALSE . |
... |
Further arguments passed to boxplot in the case that plot = TRUE . |
A data.frame
with rows corresponding to the compared classifiers
and columns to the performance measures, aggregated by aggfun
, s. above.
If more than one measure is computed and plot = TRUE
, one separate
plot is created for each of them.
Martin Slawski martin.slawski@campus.lmu.de
Anne-Laure Boulesteix http://www.slcmsr.net/boulesteix
Dudoit, S., Fridlyand, J., Speed, T. P. (2002)
Comparison of discrimination methods for the classification of tumors using gene expression data.
Journal of the American Statistical Association 97, 77-87
## Not run: ### compare the performance of several discriminant analysis methods ### for the Khan dataset: data(khan) khanX <- as.matrix(khan[,-1]) khanY <- khan[,1] set.seed(27611) fiveCV10iter <- GenerateLearningsets(y=khanY, method = "CV", fold = 5, niter = 2, strat = TRUE) ### candidate methods: DLDA, LDA, QDA, pls_LDA, sclda class_dlda <- classification(X = khanX, y=khanY, learningsets = fiveCV10iter, classifier = dldaCMA) ### peform GeneSlection for LDA, FDA, QDA (using F-Tests): genesel_da <- GeneSelection(X=khanX, y=khanY, learningsets = fiveCV10iter, method = "f.test") ### class_lda <- classification(X = khanX, y=khanY, learningsets = fiveCV10iter, classifier = ldaCMA, genesel= genesel_da, nbgene = 10) class_qda <- classification(X = khanX, y=khanY, learningsets = fiveCV10iter, classifier = qdaCMA, genesel = genesel_da, nbgene = 2) ### We now make a comparison concerning the performance (sev. measures): ### first, collect in a list: dalike <- list(class_dlda, class_lda, class_qda) ### use pre-defined compare function: comparison <- compare(dalike, plot = TRUE, measure = c("misclassification", "brier score", "average probability")) print(comparison) ## End(Not run)