First up, on the review, while we can't say from the information provided
whether the science was not up to scratch, the reviewer's comments are totally
out of kilter.

Second, I hope to help here on the issue many scientists strike when trying to
adapt magnitude based inference to physiological data. After a few years of
getting my head around it, I endorse Will's comment that standardisation (via
effect size) of a physiological, psychological or other mechanistic or related
measures provides a statistically valid effect threshold. Like with the
estimates of the smallest effects on performance, the effect size encompasses
the variability of the measure and the magnitude of the outcome, which relate in
totality to the relevance and utility of the measure in the real world.

I look to clinical medicine for comparison where you read about smallest
clinical effects, which appear to come from observations in practice = sampling
from the population. From my recent but limited research experience in clinical
exercise science and sports med, the smallest effect size comes out at about a
similar magnitude to the subjectively estimated smallest clinical effect. This
is probably not surprising because for any effect to matter, it will likely have
to score consistently above the within-subject CV for the measure which makes up
a sizeable component of the sample variance along with individual variability.
It is likely similar patterns for physiological outcomes will emerge once
researchers put their mind to it, drop significance testing, and adopt
magnitude-based evaluation approaches that provide inference closer to the
biological response (=more likely to get the science right). Consideration of
statistical power and assessment via probabilities need also to be placed near
the top of the priority list.

I think we have to trust the life-time work by people like Jacob Cohen on this,
whose views and opinions were developed through rigorous investigation, scenario
modeling, and peer-review, and should therefore be respected and seriously
considered as best practice in modern scientific analysis and inference.

Three of the greatest constraints to progress are: 1) statistical analytical
skills (=stats teaching) and user friendly tools (=market demand); 2) the
attention (and ignorance?) and lack of discipline (not insisting their own
journal guidelines are met) on the matter by journal editors; 3) magnitude-based
probabilistic inference requires a greater investment in time than hypothesis
testing, and in this error of governance by accountancy the short-term solution
rules.

David