We've all been victims of bad reviewing, but I would like to share a
particularly egregious example that someone has just shared with me, from a
paper rejected by the Scandinavian Journal of Medicine and Science in
Sports. The author has given me permission to send this one to the list:

"The statistical evaluation is too comprehensive and should be restricted to
only include the two-way repeated measures ANOVA and perhaps a student
T-test to compare mean values. Having conclusions like a 'possible' higher
pulmonary oxygen uptake does not make sense."

The author was using the approach of magnitude-based inferences that Alan
Batterham and I have been promoting and that is one of the key issues in the
article on progressive statistics in the January issue of MSSE. The extent
to which this reviewer's comment was a factor in the rejection is unclear,
but my guess is that it was important.

Repeated-measures ANOVA is definitely past its use-by date: it is plagued by
the missing-value problem, it gives the wrong p values with some designs,
and it does not properly account for different sources of variation and
error. I'm a great fan of the t statistic, but it needs to be used to make
probabilistic statements about true values, not to test a null hypothesis.
And I find it particularly disappointing that a reviewer considers that the
use of "possible" in an inference about an outcome does not make sense.

I will forward this message to Michael Kjaer, the editor of SJMSS, in the
hope that he will encourage some upskilling with this reviewer and any
others with similar old-fashioned ideas.

Hopkins WG, Marshall SW, Batterham AM, Hanin J (2009). Progressive
statistics for studies in sports medicine and exercise science.
Medicine and Science in Sports and Exercise 41, 3-12

Will
Will G Hopkins, PhD FACSM
Contact info: http://sportsci.org/will
Sportscience: http://sportsci.org
Statistics: http://newstats.org
Be creative: break rules.