My impression of the medical sciences is that effect sizes is a poor
alternative to using minimal important differences. Effect sizes were
used much more often in the past, and now people try to think about
MID. But it is difficult. So many people continue to use effect
sizes....


Ian Shrier MD, PhD, Dip Sport Med, FACSM
Associate Professor, Dep't of Fam Med, McGill University
Past-President, Canadian Academy of Sport Medicine
Check out: www.casm-acms.org
SKYPE name: ian.shrier

Centre for Clinical Epidemiology and Community Studies
SMBD-Jewish General Hospital
3755 Cote Ste-Catherine Rd
Montreal, Qc H3T 1E2
Tel: 514-340-7563
Fax: 514-340-7564





On 1-Sep-09, at 6:02 PM, Rowlands, David wrote:

> First up, on the review, while we can't say from the information
> provided whether the science was not up to scratch, the reviewer's
> comments are totally out of kilter.
>
> Second, I hope to help here on the issue many scientists strike when
> trying to adapt magnitude based inference to physiological data.
> After a few years of getting my head around it, I endorse Will's
> comment that standardisation (via effect size) of a physiological,
> psychological or other mechanistic or related measures provides a
> statistically valid effect threshold. Like with the estimates of the
> smallest effects on performance, the effect size encompasses the
> variability of the measure and the magnitude of the outcome, which
> relate in totality to the relevance and utility of the measure in
> the real world.
>
> I look to clinical medicine for comparison where you read about
> smallest clinical effects, which appear to come from observations in
> practice = sampling from the population. From my recent but limited
> research experience in clinical exercise science and sports med, the
> smallest effect size comes out at about a similar magnitude to the
> subjectively estimated smallest clinical effect. This is probably
> not surprising because for any effect to matter, it will likely have
> to score consistently above the within-subject CV for the measure
> which makes up a sizeable component of the sample variance along
> with individual variability. It is likely similar patterns for
> physiological outcomes will emerge once researchers put their mind
> to it, drop significance testing, and adopt magnitude-based
> evaluation approaches that provide inference closer to the
> biological response (=more likely to get the science right).
> Consideration of statistical power and assessment via probabilities
> need also
> to be placed near the top of the priority list.
>
> I think we have to trust the life-time work by people like Jacob
> Cohen on this, whose views and opinions were developed through
> rigorous investigation, scenario modeling, and peer-review, and
> should therefore be respected and seriously considered as best
> practice in modern scientific analysis and inference.
>
> Three of the greatest constraints to progress are: 1) statistical
> analytical
> skills (=stats teaching) and user friendly tools (=market demand);
> 2) the attention (and ignorance?) and lack of discipline (not
> insisting their own journal guidelines are met) on the matter by
> journal editors; 3) magnitude-based probabilistic inference requires
> a greater investment in time than hypothesis testing, and in this
> error of governance by accountancy the short-term solution rules.
>
> David
>
>
>
>
> ------------------------------------
>
> Post messages to sportscience@yahoogroups.com. To (un)subscribe,
> send any message to sportscience-(un)subscribe@yahoogroups.com. View
> all messages at http://groups.yahoo.com/group/sportscience/.Yahoo!
> Groups Links
>
>
>