What is the best statistical method to compare a measurement method with a criterion method? I know that there is some controversy concerning the use of Bland-Altman plot and I would rather use Intra-Class Coefficient correlation. Unfortunately, it seems there are different types of ICC and I am a bit confused about which one to select.
ICC is appropriate for reliability (test-retest reproducbility) but is never appropriate for validity (comparing methods). Use the usual (Pearson) correlation. Better still, use the standard error of the estimate, which represents the noise in the prediction of the criterion. Compare the magnitude of the noise with the smallest clinically important difference (the signal). In deriving the error, make the criterion the Y value and the other measure the X value. For more info, read my papers on How to Interpret Changes in an Athletic Performance Test and on Bias in Bland-Altman but not Regression Validity Analyses in the current issue of Sportscience.
This query reminds me that I have been forgetting to send this list a copy of the statistical queries I receive personally and my replies. Let me know if you have strong feelings either way about resuming that service.
Will