Posted by Larry Hoover on April 22, 2008, at 20:32:56
In reply to Re: debate » Larry Hoover, posted by llurpsienoodle on April 22, 2008, at 14:30:01
> "Among other things, these applications have revealed that the misuse of ordinal scaled data can produce erroneous data and drive inaccurate conclusions. Consequently, concerns must be raised over the accuracy of the results of the meta-regression performed by Kirsch et al, given they have undertaken sophisticated mathematical operations on data which do not support such activities. Moreover, it is worth noting that even the calculation of a mean, a standard deviation, and a change score are invalid on ordinal data, given that these all assume equal interval scaling."
>
>
> no no no! tell me it ain't so-- the authors conducted parametric statistical tests on ordinal statistics? (can you hear me retching?)Your timing is superb. I have been mulling this issue over for some weeks now, ever since I first looked at the subject paper. It just had never occurred to me to question the fact that it is standard practise to treat Hamilton or Beck scores as interval data. It is routine to read about mean improvement, standard deviations, confidence intervals......rather than median scores or quintile ranks.
But yes, Kirsch et al did multiple regressions on ordinal data.
I've been thinking that perhaps the most valid of the invalid statistics might be the number reaching a threshold, usually a 50% reduction in scores, which would be a test-retest measure unique to each subject. This threshold would serve to create a binary outcome measure, which percentage or count might then be contrasted in the experimental groups?
> Once upon a time I was a "peer" and reviewed some truly outrageous papers, a couple of iffy ones, and one good one. There was also the paper that my advisor said "must come from lab X... very esteemed lab...I'm sure it's great..." leaving the peon llurpsie little option but to overlook significant editorial issues and infer meaning and conclusions where none were written. barf.
I did literature reviews for a few years.....sifting through mounds of garbage to find the gems. I hear you, loud and clear. Although my analyses were post peer-review, I often wondered how some works had made it to print. Well over half were simply worthless.
> Maybe I have the energy to read this entire thread, now that I've interrupted it.
>
> llI would welcome your commentary. And you're most certainly not interrupting.
Lar
poster:Larry Hoover
thread:823248
URL: http://www.dr-bob.org/babble/20080420/msgs/824878.html