The importance of critical watching

A critical review of Healing Cancer from the Inside Out, a DVD

At risk of sounding like I never give anything that I don’t agree with a fair hearing, I have to say that, as in my ‘review’ of the 80/10/10 Diet, this is something that I just couldn’t finish.

This DVD is about alternative approaches to treating cancer. Now, once again, I’m no one-eyed fan of the mainstream approach, and I’ve seen at least one instance of someone seeming to suffer more from the treatment than they did from the disease. I’m also confident that there are a lot of people making a lot of money from cancer. But from what little I did see of this production, the use of statistics in this DVD pretty much takes the cake for the applicability of the characterisation “lies, damned lies, and statistics.”

It has to be said at the outset that this DVD could be used as a “how to spot propaganda” example. The ‘scary’ emotive background music and on-screen text, overlaid on scrolling academic journal articles, reeks of someone trying to push an agenda. I have to admit that I was inclined to excuse some of this as ‘cultural’; having seen American talk shows I (reasonably?) assumed that there was something in the American psyche that makes exaggerated drama appealing. Also, the fact that a lot of people watching the DVD would probably have been recently diagnosed with cancer, or perhaps a family member of such a person, led me to think, naively, that I should expect an even-handed approach.

Anyway, to the details…

The DVD talks about how we’re losing the war on cancer, how long term survival rates have hardly budged since the 70s, etc. Even more so than in the 80/10/10 Diet, the DVD is an assault on mainstream western medicine.

“Untreated patients live longer than treated patients”. That was one claim spread across an impressive looking list of references. Seven articles were cited to support this claim. I checked the first three:

1. JAMA, 1992, 257, p. 2191
2. Lancet, 1991, August, p. 901
3. NEJM, 1986, May 8, p. 1226

(1) The JAMA article, as cited, doesn’t exist – as far as I can see it should have been 267, not 257. This would make the correct citation JAMA 1992;267(16):2191-2196 – same year, same starting page, and 267 instead of 257. The article is called “High 10-Year Survival Rate in Patients With Early, Untreated Prostatic Cancer”.

Now, I should say up-front that I haven’t been able to access the full text, but the abstract states that the patients had “initially untreated prostatic cancer” (emphasis mine) but then goes on to list interventions: “Patients with tumor progression were hormonally treated (orchiectomy or estrogens) if they had symptoms.” Orchiectomy is the surgical removal of the testicles (which produce the hormone testosterone, which ‘feeds’ prostate cancer). That doesn’t sound like “untreated patients” to me (as per the DVD’s claim). From the abstract I assume that the trial was designed to see what would happen if the cancer was left to run its course, but they were still doing interventions if the tumours progressed. The abstract does conclude with a “low disease-specific mortality rate” and suggests further studies should compare therapies for early prostate cancer with an untreated group. It also notes that of the total 223 patients, 124 had died by the end of the study, and of those 124 deaths, only 19 were attributable to prostate cancer – the remaining 105 deaths were from other causes. That’s certainly eye-opening and food for thought, but it doesn’t anywhere say “untreated patients live longer than treated patients”, which is the claim that this study is cited to support.

(2) There are several “Lancet” journals, and I’ve been unable to find this article in any of the ones I looked in. Once again though, if we change ‘August’ to ‘April’, there’s a match… The Lancet 1991 April 13 pp. 901-902 (same starting page again). The article is called “Medical oncology in the 1990s”. It’s actually not a study, it’s an editorial (though presumably a well informed one). It refers to chemotherapy (not treatment as a whole – the article is all about chemo) not providing benefit, but it doesn’t say anywhere that “untreated patients live longer than treated patients”, which is the claim for which this reference is cited as support.

(3) I actually found this one without having to imagine what mistakes they may have made in the reference. Once again I was unable to access the full text, but the abstract states “We assessed the overall progress against cancer during the years 1950 to 1982…. we are losing the war against cancer”. It’s certainly not a glowing appraisal of the success of western medicine in treating cancer, but it does acknowledge “…extension of the productive years of life”. The abstract obviously contradicts the notion that untreated patients live longer than treated patients.

While this is interesting and at times eye-opening reading, it’s not impressive support for such an inflammatory claim.

Then there’s this one:

…and on it goes, with the higher percentages left until last. This table is presented in full screen glory, white text on a black background. The narrator states that these figures come from a review of all randomised clinical trials performed in over a decade which reported an increase in 5 year survival rate due to the use of chemotherapy. He goes on to explain that in medicine a success rate of over 30% is required before a drug is considered better than a placebo.

I looked up this study and was surprised to find that the numbers line up. Overall, in Australia, across all cancer types, the final percentage was 2.3%

Wow – this was one case where the DVD was actually backed up by the studies by which it claimed to be backed up.

But then I read it properly in the cold light of day. The study is not really even about the effectiveness of chemo. The study is about the overfunding and over-availability of chemo compared to the number of people likely to benefit from it. Now, that may sound like one and the same thing, and the nuanced difference here did do my head in for a while as I attempted to grapple with what was being said (and I misunderstood it myself a few times… I think I’ve got it right  now…).

So, how is the effectiveness of chemo different from the overfunding of chemo compared to the number of people likely to benefit from it…

Firstly, the figures in the table were determined by multiplying:

1. the total number of malignancies of a particular type in the population at large with
2. the number of people in the subgroup of (1) which are known to benefit from chemo, expressed as a percentage of (1), and then with
3. the increase in number of people who survive for 5 years conferred on that subgroup (2) as a result of chemo, expressed as a percentage of that subgroup.

Number 2 is the kicker here. It’s known that chemo will do next to nothing (or nothing) for many types and stages of cancer. Let’s imagine we have 5000 people with cancer of the big toe, and that we already know that chemo will only work if the cancer is on the left big toe and your first name is Jane. On that basis, of that 5000 people, there may be a subgroup of 10 people for whom we’d expect chemo to be effective. If chemo on this subgroup is shown to increase the 5 year survival rate by 10%, then the final number of people to benefit is 1 (since 10% of 10 is 1). And 1 is 0.02% of 5000. So, on this basis, chemo is only likely increase 5 year survival rate of big toe cancer sufferers by 0.02%.

Notice that the small percentage is largely due to the small size of the subgroup who are known to benefit from chemo.The study is not saying that with regard to increasing 5 year survival, ‘chemo has a 0.02% effectiveness when used‘, it’s saying, ‘chemo has a 0.02% effectiveness across all cases of that malignancy because we know it won’t work in most cases’.

Here’s a real example from the study. The study’s figures for colon cancer in Australia are:

7243  x  35%  x  5% = 128 people (1.8%)

(incidence of colon cancer in the entire population) x (the percentage of these people in a subgroup which is known to benefit from chemo) x (increase in 5-year survival rate from chemotherapy) = total

So the study is saying that in 7243 incidents of colon cancer in the population, only 128 more people (1.8%) are likely to survive to 5 years as a result of chemo. It does not mean that chemo is 1.8% effective, or that chemo provides a 1.8% chance of increasing five year survival rate when recommended as treatment. Indeed, the study notes “Surgery is the only established curative treatment for colon cancer, with chemotherapy used as adjuvant treatment”;  the study actually takes the limited usefulness of chemo as a starting point.

Why, then, is the guy on the DVD giving us these figures, and then telling us about 30% success rate being required for a better-than-placebo assessment?  Why contrast a requirement for success with a figure that doesn’t represent success rates?  An analogy:

BOSS: “Hey guys, you were supposed to convert 30% of your leads into real sales, and you haven’t; 7000 cars drove past this store in the last month, and you’ve only made 130 sales!”

SALES TEAM: “Uh, yeah, but of those 7000 cars, only a fraction actually stopped to do some shopping in this town…”

This, to my mind, is an appalling abuse of statistics. I reckon if someone in mainstream medicine produced a DVD with such a lax application of statistics for public consumption, they’d be shut down. If there was evidence that every person with cancer is put onto chemo (i.e. that every one of the 7000 car drivers went into the shop), then the low figures that are implied to be a ‘success rate’ would be accurate, but there’s certainly no indication that this is the case.

Now, that’s not to say that the actual effectiveness figures for chemo are actually glowing. If the DVD wanted to look at the effectiveness of chemo, there was plenty to work with, without exaggerating its case. It should have been looking at the effectiveness figures for the subgroups in which benefits are known. Looking at colon cancer again, this would mean 5% (as I said, not glowing, but better than 1.8%).

And that’s the point of the study – the question the study raises is ‘why does research for chemo receive so much funding, and why are the drugs so available, when chemo will benefit so few people?’ So I don’t know why the creators of the DVD felt they should paint an even worse picture than the study itself paints.

Maybe I’m unusual, but I can’t see the point of subjecting myself to propaganda in the name of enlightening myself against a different form of propaganda. I probably am unusual because I find this kind of misappropriation of truth, or, being more generous, a general “glossing over” of selective key points, to be quite personally affronting. And I’m also affronted for the thousands of other people who watch it and don’t bother looking into references because they trust that someone taking the higher moral ground would not overstate their case.

Looking at the articles themselves was far more informative than watching the scary graphics and the horror movie suspense music that accompanies the DVD.

Discussion from the study includes:

The best example of the ‘over-selling’ of chemotherapy is in breast cancer, where chemotherapy was introduced as the example of the new cure for solid malignancies. In Australia, in 1998, only 4638 of the 10661 women with newly diagnosed breast cancer were eligible for adjuvant chemotherapy (44% of total). From our calculations, only 164 women (3.5%) actually had a survival benefit from adjuvant chemotherapy. In other words, on average, 29 women had to be treated for one additional woman to survive more than 5 years…

 

…This, of course, leads to a discussion of the cost implications of cytotoxic chemotherapy. Although this is a separate issue, we note that the cost of cytotoxic drugs provided by the Pharmaceutical Benefits Scheme in Australia increased from $67M for the year ended 30 June 2000 to $101.3M for the year ended 30 June 2001. The 51% increase in total drug cost was due to a 17% increase in the number of prescriptions and a 29% increase in average prescription price….

 

…However, despite the early claims of chemotherapy as the panacea for curing all cancers, the impact of cytotoxic chemotherapy is limited to small subgroups of patients and mostly occurs in the less common malignancies.

Even so, any new chemotherapy drug is still promoted as a major breakthrough in the fight against cancer, only to be later rejected without the fanfare that accompanied its arrival.

If I’m ever told I need chemo, I’ll be doing a bit of research. But I won’t be taking anything the “Rave Diet” crowd (producers of the DVD) says at face value either.

If anyone knows of any genuinely even-handed discussions of these issues, I’d love to hear of them.