Posted by used2b on April 11, 2005, at 12:03:05
In reply to Re: empirical evidence, posted by Dr. Bob on April 11, 2005, at 10:12:27
> > Are you suggesting that, to the best of your scientific knowledge, you consider an individuals' analysis of personal Babbleometer data to be reliable evidence of the efficacy of this forum?
>
> I'm suggesting that it's one way to track how you do over time and that it may give you a helpful perspective on how things are going and a better idea of what helps and what doesn't.
>
> You might also consider sharing the data with your health care professionals for their input.
>
> Bob
I suppose we could also share with a health care professional information we get from a psychic or a palm reader. But none of it would comprise empirical evidence by accepted scientific or medical standards.BBmeter can only record self-reported changes in an individual, though the instrument is flawed even for that purpose. It cannot determine if those changes are a result of participating in PB, nor whether the changes are durable.
Evidence that one individual alone reported beneficial changes that correlate to participation in PB would not be reliable evidence. We would need to review a data set for a statistically significant sample of participants, selected randomly from the group. Otherwise, our evidence might be tainted by those who self-select to report improvement as an unconscious means of re-enforcing a placebo effect.
Then, from that random group, we would need test results collected at pre-determined times, and not at times the subjects choose to voluntarily self-report. Again, self-selection of reporting time could allow results to reflect subjects' choice, intended or not, to report only when they are feeling better or worse.
Then, even if we had data from randomly selected subjects, collected according to a pre-determined schedule, at most we would perhaps collect evidence of a correlation between subjects' conditions and participation in this forum. We would have no evidence that any changes reflected in aggregate data were caused by PB.
Then our evidence would need to be compared with other sources of data - especially adverse event reports. It is doubtful there is any way of reliably tracking individuals who have engaged in self-harm after participating in the site, so even if we have adverse event data, we have to question whether it is complete.
Once we have a set of reports from the babblemeter instrument, and some indication of the prevalence of adverse events, we would still have no way of determining causation, or whether the group is selecting individuals who might best benefit while engaging in practices that trigger unreported deterioration or self-harm in others. And even if we did have a relible instrument, administered dispassionately, before I accept the results as reliable scienctific evidence, I would want to see results you report reviewed by your professional peers.
Surely you recognize that many of the people who visit this site are informed about standards of scientific and medical evidence. While some individuals might gain interesting information from casual use of your test instrument, I feel somewhat put down that I would be expected to accept such an instrument as a source of empirical evidence about the likelihood I will enjoy any benefit from participating in this forum. The suggestion presumes I am not as smart as those who publish medical journals, and that I would do as well to rely on substandard evidence in selecting best approaches for my own care.
poster:used2b
thread:478418
URL: http://www.dr-bob.org/babble/social/20050408/msgs/482794.html