The Blue Cross / Blue Shield Association's
Technology Evaluation Center Report
On Biofeedback in the Treatment of Urinary Incontinence

CENSORED
By Blue Cross / Blue Shield!

On February 24, 1998, after closing, we received the following FAX from BC/BS TEC:

Dear Dr. Perry:

As we discussed and you agreed, please cease unauthorized use of the copyrighted TEC Executive Summary on Biofeedback for Urinary Incontinence by removing it from your website.

Thank you.

Sincerely,

Sarah L. Bleeks, Director
Business Development

cc: N. Aronson
     B. Cheng

On February 25, 1998, we sent the following acknowledgement:

February 25, 1998

Note: Previously, the entire original text is presented, with critical comments interspersed and a commentary at the end.  Now we only include "brief quotations" (112 words, by our count) in the context of this critical review, as permitted under the fair-use doctrine of copyright law.

An InContiNet Critical Review:

Blue Cross / Blue Shield Association
Technology Evaluation Center Report

Biofeedback in the Treatment
of Adult Urinary Incontinence

Executive Summary

The TEC Summary begins by asserting that bladder training and pelvic muscle exercises are "the standard behavioral treatments" (4 words quoted) for the common forms of incontinence, and indicates that the present report will investigate whether the addition of biofeedback results in any improvement over the standard treatments.   This is, of course, a decidedly "short view" of history, since pelvic muscle exercises did not become popular until Arnold Kegel, MD, introduced biofeedback, using the Kegel Perineometer, in 1948. [See The Kegel Perineometer: Biofeedback 20 years before its time.]

It wasn't until the 1970s that bladder training was introduced, and pelvic muscle exercises without biofeedback became popular among nurses -- only after Kegel's perineometer ceased to be readily available.  In spite of defining these non-biofeedback behavioral methods as the standard, there is no mention of how effective they are.  [Readers should be aware that in the 1980s and 1990s, BC/BS defined the non-biofeedback teaching of Kegel Exercises as a standard nursing procedure, and therefore, not a separately reimbursable activity.]

The TEC report then turns to assert their (own) standards for the evaluation of any technology, and to rate biofeedback against those standards.  First:

"1. The technology must have final approval from the appropriate government regulatory bodies. " (13)

The report offers a brief description of biofeedback, mentioning both manometric and electromyographic varieties, and correctly notes that behavioral techniques are not subject to FDA regulation, but falsely states that the instruments themselves "may or may not be subject to FDA regulation".(9)  In fact, all biofeedback devices used in the health field are carefully regulated, and subject to intense and very expensive scrutiny with respect to health and safety issues.  In fact, the biofeedback technology used for incontinence has all the final approvals necessary from all regulatory agencies.

"2. The scientific evidence must permit conclusions concerning the effect of the technology on health outcomes." (16)

The summary report states that only six "controlled trials" (2) could be identified, in which pelvic muscle exercises alone were used as control groups to compare with "behavioral treatments plus biofeedback" (4).  They also complain that the studies were "relatively small" (2) with less than 300 subjects, except for one we recognize as Burns, which began with 135 subjects.  (The remaining five studies, therefore, enrolled an average of 33 subjects each.)

We are especially concerned about the use of the term "trials" instead of the more appropriate "studies".  "Trials" are used in drug research, as in "clinical trials".  The terms "studies" or "experiments" are more commonly used in behavioral research.  This vocabulary clearly reveals the bias of the BC/BS TEC assessment toward drug studies.  

We also see that only "controlled trials" were considered in the review.   This is appropriate (and necessary) in drug testing, but is considered inappropriate in most behavioral research, where simpler designs are usually used.  For an extended discussion of this issue, see Shellenberger and Green, From The Ghost in the Box to Successful Biofeedback Training (Health Psychology Publications, Greeley, CO) 1986.  For a brief discussion, see the critical comments at the end of this review.

It is difficult to fully understand the TEC assessment of the six studies, since they have repeatedly refused to reveal which six studies out of the 36 listed in their references were selected.  They claim that in four of the 6 studies there was no statistically significant difference between PME and PME+biofeedback, although in 3 of the 4 there were nonsignficant trends that supported biofeedback over plain PME.  Clearly Burns et al, which had 135 subjects at the start, has to be one of these three; Burns found a trend (61% vs. 54% symptom improvement) that did not reach statistical significance. 

The one study (out of 6) that did not show a trend towards biofeedback is probably Burton, Pearce, Burgio et al (1988). Although this study has often been cited as proof that biofeedback is not superior, most people do not read closely enough to notice that Burton et al did not compare PME vs. PME+biofeedback, but compared other behavioral interventions to PME+biofeedback.  It will be interesting to see, if the six studies are ever publicly identified, if Burton was mistakenly included by TEC.

The TEC summary is difficult to understand and contains contradictory statements.  On the one hand, we are told that four studies did not find statistically significant differences -- by simple subtraction, that leaves two studies that presumably did find statistically significant differences.  Yet the text states that "[the other] two studies reported a benefit for PME plus biofeedback greater than that achieved with PME alone, although in 1 of these 2 studies, no tests of statistical significance were performed." (30)    If that last claim is true, then it would mean that five, not four, studies actually had trends favoring biofeedback, and in one of them the trend was statistically significant.

And, if we consider only those studies that actually compared PME to PME+biofeedback, we find that all five of the studies found biofeedback to be superior to plain PME, and in either one or two of them [depending on which TEC sentence we believe] this superiority was statistically significant.  Clearly TEC's own findings do not support their own summary statement, that "The controlled trials report conflicting results."  (6) The results are, in fact, amazingly consistent.

Interestingly, the TEC report refers to the studies in which the trends did not reach statistical significance as having produced "negative"(1) results.  This use of completely incorrect terminology causes us to wonder about the statistical sophistication of the so-called expert panel that produced the report. 

The TEC report also complains that few of the studies had sufficient subjects to detect group differences (which is both true and obvious), and mentions that only [Burns] explicitly addressed that issue.  Of course the real problem with Burns is that she did such a poor job of using biofeedback to teach exercises that she would have needed thousands of subjects to achieve statistical significance.  Burns' 61% symptom reduction rate is tied in last place as one of the worst biofeedback outcomes ever reported in the literature.  [See comparison.htm for details.]

The TEC summary concludes: "Based on the above, biofeedback for the treatment of adult urinary incontinence does not meet the Blue Cross and Blue Shield Association Technology Evaluation Center (TEC) criteria." (27) Based on the above, we conclude, on the contrary, that TECs pompous conclusion is simply bogus. 

The fundamental problem with TEC is their attempt to apply drug-research methods and criteria to the study of behavioral interventions.  This mistake is discussed at length in the monograph "The Ghost in the Box", to which we now turn.

A Commentary on the TEC Review

In 1986 Robert Shellenberger, PhD and Judith Alyce Green, PhD [daughter of Elmer Green, widely regarded as one of the founders of biofeedback, who worked at the Menninger Clinic for many years] published a monumental monograph entitled "From the Ghost in the Box to Successful Biofeedback Training".   Their primary theme was that there is no "ghost" or agent inside the biofeedback "box" that produces any "change" in the patient; the patient must change himself.  The box doesn't "do" anything to the patient. 

This is in marked contrast to the mechanism of pharmacological agents, for which it is postulated that the "drug" inside the capsule will produce certain "specific effects" on the patient, with total disregard for whether or not the patient "knows" he is taking the drug, or "believes in" its efficacy.   But in biofeedback, the "box" only provides certain information to the patient, who may or may not use that information to effect a change in his own biological functioning.

Controlled trials are necessary in drug research to establish that any change resulting from the use of the drug did not result from some other source, such as patient expectations or other patient behavior.  Placebos are used to see if these other variables induce changes similar to those expected for the drug.  A statistical procedure, Analysis of Variance (ANOVA), is used to mathematically separate out the effects attributable to the test drug from all other effects; the latter are called "error variance" and discarded. 

But in biofeedback research ANOVA cannot be used, they argue, since the theory does not expect any effect to result from the biofeedback box itself.  If there is any change in the patient's behavior, it is because the patient received, processed,   and understood information (about his own self-regulation) from the biofeedback box, and used that information in a deliberate way to alter his internal and/or external behavior.  The authors use the analogy of the track runner's stopwatch to explain.   The runner's objective is, of course, to run faster, but stopwatch does NOT "make" the runner run faster; it only provides him with information so that he can decide how hard to push himself.  In contrast, the "pill" does make the heart run faster (or slower), regardless of what the runner decides to do. 

The authors (and others) use the term "training" to differentiate it from "treatment", which is more appropriate for drugs and other external interventions.  The concept of "control groups" is meaningless in a biofeedback context.  Consider, for example, an Olympic runner.  To win the race, it is not necessary to prove that the runner did better than his neighbor, a couch potato.  The winner simply runs faster than the other runners.  Similarly, when the objective is to eliminate the use of absorbent products and reduce urinary leaks, it is only necessary to compare the diaper budget "before" and "after" the intervention. 

In behavioral research, it is quite common to use within-subjects designs; the "ABA" or "ABAB" sequence is widely used.  In trial "A", we record baseline behavior; in "B" we apply an intervention; and in the second "A" we observe the behavior when the intervention is withdrawn.  The efficacy of PME is clearly shown when incontinent patients stop practicing their exercises for a few days -- and the leakage returns.  

Similarly, the superiority of PME+biofeedback is clearly shown, not by comparison to an untreated population, but by the greater reduction in "leaks", in a shorter period of time, than accomplished by plain PME. 

If this difference between drug research and behavioral research had been understood by the BC/BS committee, they would not have confined themselves to a scant half-dozen studies (opps, "trials") in which "control groups" were used.  They would have seen that, regardless of the particular instruments used, PME+biofeedback produces better results than PME alone. 

There are, however, important distinctions between the various models that include biofeedback in one form or another.  In general, as shown in detail in the paper at http://www.incontinet.com/comparison.htm, studies which used 100% home trainers -- with or without clinical instruments -- achieved symptom reduction rates over 90%, while studies that used clinical instruments alone -- without home trainers -- achieved rates in the 80-90% range.  Generally, studies of PME alone report results in the 50-60% symptom reduction range. 

Thus the conclusion, that biofeedback does not add to the effectiveness of BT and PME, is only an artifact of the limited selection of drug-model studies which were reviewed by the panel.  Had the panel included experts on the evaluation of behavioral interventions they would not have restricted their inquiry to the small number of projects they did -- and they would have come to very different conclusions about the value of biofeedback. 

John D. Perry, PhD
Psychologist (ret.)
January 24, 1998

Copyright 1998 by InContiNet.com    Click Here to return to our Home Page.
Check http://www.incontinet.com/bcbstec.htm for the latest version of this document.
This document last revised: Wednesday, May 04, 2005 12:46:36 AM