At the ACSM annual meeting earlier this month there was a stimulating colloquium on validity, presented by Greg Atkinson and Alan Batterham. One of the issues raised was that a Bland-Altman plot for comparison of two measures (a plot of each subject's difference score vs mean score for the two measures) may show proportional bias when there isn't really any in reality. 

It happens when one measure is predicted or calibrated from another.  For example, if someone does a study to predict VO2max from a shuttle run, derives the prediction equation, then compares predicted and actual VO2max using a Bland-Altman plot, the scatter of points shows a positive trend that increases for decreasing correlation between the measures.  So what?  I think it means the Bland-Altman plot is misleading, because it suggests some sort of systematic difference in the measures.  But there is none, surely?  There's nothing wrong with deriving an equation to predict a criterion measure from a practical measure, then using the predicted values.  But according to the Bland-Altman plot, there is something wrong.

You can also get proportional bias when the measures really do differ in scale, but how would you know whether proportional bias apparent in a plot is due to such real differences or is just an artefact of prediction or calibration?

I have devised a speadsheet to illustrate the effects.  To download the spreadsheet, click on http://sportsci.org/resource/stats/Regression_vs_BlandAltman.xls .  You may want to resave it out of the browser window into a new purely Excel window before playing with it.  It will take you a while to figure out what's going on.  Hopefully the reward will be worth the effort.  Note that every time you do something to a cell, or every time you save the file, the random number generator updates all the cells to new values.

I have other reasons for not using the Bland-Altman approach, but to some extent these are philosophical reasons related to viewing validity as calibration.  The current problem of artefactual proportional bias is perhaps more serious.  Comments, anyone?  Send them to the list please.  We haven't had a good discussion for some time.

Will

Will G Hopkins, PhD FACSM
Work +64 9 917 9793, Fax +64 9 917 9960
Home +64 9 376 0198, Cell +64 27 427 2518
Health Science/Sport and Recreation
Auckland University of Technology
Private Bag 92006, Auckland 1020, New Zealand
will@...
Statistics: http://newstats.org
Sportscience: http://sportsci.org
---------------------------------
Be creative: break rules.