Ben Goldacre - Bad Science

This strain of MRSA had never been found in any patient in the UK, and it had only ever been seen rarely in Australia. There is very little chance that this was found wild in the UK: in all likelihood it was a contaminant, from the media work ChemSol had done for Australian tabloids. On the other six plates, where Malyszewicz thought he had MRSA, there were mostly just bacilli, a common but completely different group of bacteria. MRSA looks like a ball. Bacilli look like a rod. You can tell the difference between them using 100x magnification – the ‘Edu Science Microscope Set’ at Toys’ ’Us for £9.99 will do the job very well (if you buy one, with the straightest face in the world I recommend looking at your sperm: it’s quite a soulful moment).

We can forgive journalists for not following the science detail. We can forgive them for being investigative newshounds, perhaps, even though they were repeatedly told – by perfectly normal microbiologists, not men in black – that Chemsol’s results were improbable, and probably impossible. But was there anything else, something more concrete, that would have suggested to these journalists that their favourite lab was providing inaccurate results?

Perhaps yes, when they visited Malyszewicz’s laboratory, which had none of the accreditation which you would expect for any normal lab. On just one occasion the government’s Inspector of Microbiology was permitted to inspect it. The report from this visit describes the Chemsol laboratory as ‘a freestanding, single storey wooden building, approximately 6m × 2m in the back garden’. It was a garden shed. They go on to describe ‘benching of a good household quality (not to microbiology laboratory standards)’. It was a garden shed with kitchen fittings.

And we should also mention in passing that Malyszewicz had a commercial interest: ‘Worried about MRSA? The perfect gift for a friend or relative in hospital. Show them how much you care for their health by giving a Combact Antimicrobial Hospital Pack. Making sure they come out fighting fit.’ It turned out that most of Chemsol’s money came from selling disinfectants for MRSA, often with bizarre promotional material.

How did the papers respond to the concerns, raised by senior microbiologists all over the country, that this man was providing bogus results? In July 2004, two days after Malyszewicz allowed these two real microbiologists in to examine his garden shed, the Sunday Mirror wrote a long, vitriolic piece about them: ‘Health Secretary John Reid was accused last night of trying to gag Britain’s leading expert on the killer bug MRSA.’ Britain’s leading expert who has no microbiology qualifications, runs his operation from a shed in the garden, mispronounces the names of common bacteria, and demonstrably doesn’t understand the most basic aspects of microbiology. ‘Dr Chris Malyszewicz has pioneered a new method of testing for levels of MRSA and other bacteria,’ it went on. ‘They asked me a lot of questions about my procedures and academic background,’ said Dr Malyszewicz. ‘It was an outrageous attempt to discredit and silence him,’ said Tony Field, chairman of the national MRSA support group, who inevitably regarded Dr Malyszewicz as a hero, as did many who had suffered at the hands of this bacterium.

The accompanying editorial in the Sunday Mirror heroically managed to knit three all-time classic bogus science stories together, into one stirring eulogy:

Whistle-blowers appear to bring out the very worst in this Government.

NO WAY TO TREAT A DEDICATED DOCTOR

First, Frankenstein foods expert Arpad Puzstai felt Labour’s wrath when he dared to raise the alarm over genetically-modified crops. Then Dr Andrew Wakefield suffered the same fate when he suggested a link between the single-jab MMR vaccine and autism. Now it’s the turn of Dr Chris Malyszewicz, who has publicly exposed alarmingly high rates of the killer bug MRSA in NHS hospitals.

Dr Chris Malyszewicz should get a medal for his work. Instead he tells the Sunday Mirror how Health Secretary John Reid sent two senior advisers to his home to ‘silence him’.

The Sunday Mirror was not alone. When the Evening Standard published an article based on Malyszewicz’s results (‘Killer Bugs Widespread in Horrifying Hospital Study’), two senior consultant microbiologists from UCH, Dr Geoff Ridgway and Dr Peter Wilson, wrote to the paper pointing out the problems with Malyszewicz’s methods. The Evening Standard didn’t bother to reply.

Two months later it ran another story using Malyszewicz’s bogus results. That time Dr Vanya Gant, another UCH consultant microbiologist, wrote to the paper. This time the Standard did deign to reply:

We stand by the accuracy and integrity of our articles. The research was carried out by a competent person using current testing media. Chris Malyszewicz … is a fully trained microbiologist with eighteen years’ experience … We believe the test media used … were sufficient to detect the presence of pathogenic type MRSA.

What you are seeing here is a tabloid journalist telling a department of world-class research microbiologists that they are mistaken about microbiology. This is an excellent example of a phenomenon described in one of my favourite psychology papers: ‘Unskilled and Unaware of It: How Difficulties in Recognizing One’s Own Incompetence Lead to Inflated Self-Assessments’, by Justin Kruger and David Dunning. They noted that people who are incompetent suffer a dual burden: not only are they incompetent, but they may also be too incompetent to assay their own incompetence, because the skills which underlie an ability to make a correct judgement are the same as the skills required to recognise a correct judgement.

As has been noted, surveys repeatedly show that a majority of us consider ourselves to be above average at various skills, including leadership, getting on with other people, and expressing ourselves. More than that, previous studies had already found that unskilled readers are less able to rate their own text comprehension, bad drivers are poor at predicting their own performance on a reaction-time test, poorly performing students are worse at predicting test performance, and most chillingly, socially incompetent boys are essentially unaware of their repeated faux pas .

Perceived logical reasoning ability and test performance as a function of actual test performance

Kruger and Dunning brought this evidence together, but also did a series of new experiments themselves, looking at skills in domains like humour and logical reasoning. Their findings were twofold: people who performed particularly poorly relative to their peers were unaware of their own incompetence; but more than that, they were also less able to recognize competence in others , because this, too, relied on ‘meta-cognition’, or knowledge about the skill.

That was a pop-psych distraction. There is also a second, more general point to be made here. Journalists frequently flatter themselves with the fantasy that they are unveiling vast conspiracies, that the entire medical establishment has joined hands to suppress an awful truth. In reality I would guess that the 150,000 doctors in the UK could barely agree on second-line management of hypertension, but no matter: this fantasy was the structure of the MMR story, and the MRSA swab story, and many others, but it was a similar grandiosity that drove many of the earlier examples in this book where a journalist concluded that they knew best, including ‘cocaine use doubles in the playground’.