Bad hospitals: a convenient myth

There’s no such thing as a bad hospital.

When I made that statement a couple of weeks ago, it was met with some derision. That’s hardly surprising, particularly in the light of yesterday’s news that the Mid Staffordshire NHS Trust is to be wound up: as well as Cannock, it runs Stafford Hospital which has become a healthcare horror story, the hospital we all love to hate. That’s all of us except, strangely enough, the population of Stafford itself, who have been running an increasingly active campaign in support of their hospital.

Stafford Hospital: notoriously run by Mid Staffs NHS Trust
now about to be dissolved.

The scandal broke back in March 2009. A report gave a catalogue of shameful failures of care. But, more specifically, the press focused on the finding that between 2005 and 2008, between 400 and 1200 more people died there than might have been expected. Now that really set the alarm bells ringing. Up to 1200 excess deaths? A place of healing seemed to have turned into a killing field. No other judgement about the hospital could possibly have been as devastating.

So it’s curious that when the Francis report into the shortcomings of Mid Staffs was published in February 2013, it made no mention of this devastating statistic.

The figure came from Sir Brian Jarman, a professor at Imperial College London and one of the founders of healthcare information service provider Dr Foster Intelligence (DFI). It was based on DFI’s vaunted ‘hospital standardised mortality ratio’, the flagship of the indicators the company has developed to evaluate hospital care.

Why wasn’t it included in the final report?

Because it is extremely difficult to interpret in practical terms. It seems to be saying that there were up to 1200 avoidable deaths in the hospital, but it decidedly does not mean that. Studies since have shown that a great many of the ‘excess deaths’ reflected factors entirely unrelated to care quality, such as including in the hospital’s mortality figures patients who were already dead on arrival, or not making allowance for the terminal patients who were there for palliative care only and were simply being nursed to a humane death with no hope of recovery.

A 2009 study of 50 questionable deaths in the hospital revealed that just one of them might have been preventable.

So the Francis report detailed many completely genuine instances of poor care but dropped all mention of the mortality figures that caused so much noise in the first place. The failings identified led to departures and even, recently, to two nurses from the particularly criticised Accident and Emergency area being struck off. The new management has made major efforts to correct shortcomings, efforts vindicated when the Care Quality Commission (the body that monitors hospital performance) officially declared that all its concerns had been addressed.

Dr Foster has also given the hospital a clean bill of health.

Oddly enough, though, that  last statement doesn’t suprise me. It’s not the first time DFI has given a favourable judgement on Stafford. Hunting through my old posts recently, I came across one from December 2009. It mentioned that Dr Foster had published a new study and again created quite a stir by identifying the ‘dirty dozen’ of ‘worst’ hospitals in England.

In that study, Dr Foster used fifteen metrics to evaluate hospitals. All of them were tightly defined, even in some cases narrowed to a single operation. For example, one concerned a particular technique for gall bladder removal. So a hospital whose General Surgeons used that specific technique well would score highly on that measure; with only fifteen being taken into account, that would probably have a major impact on its overall standing too.

Equally, a hospital with general surgeons who were not particularly competent in that approach might find its rating seriously depressed. And yet General Surgery is just one of the 25-30 specialties in even a small hospital like Stafford. The gall bladder indicator tells you nothing about any of the other specialties.

More generally, just fifteen metrics are far too few to give a clear view of performance across all specialities, or by the whole body of staff, which at Stafford is nearly 3000 strong.

That didn’t stop DFI publishing its performance ratings of hospitals based on fifteen indicators. Poor old Stafford, I can imagine you saying, being castigated again. But have no fear: it wasn’t. Far from it. On that terribly narrow range of DFI indicators, Mid Staffs, far from being one of the dirty dozen, was categorised ninth out of 146 around the country.

And that was based on data up to March 2009, the very period for which Stafford Hospital was being crucified for appalling performance.

Excellent confirmation, if any were necessary, that trying to define good or bad hospitals is an impossible task, especially if you use a highly limited range of indicators to assess them.

In fact, to get a real idea of performance across the whole range of services within a hospital would take many dozen indicators, far more than any organisation such as DFI, other companies or even the health service have the means to measure. And even if the full range were available, how could they be combined into a single value reflecting whether the hospital as a whole is ‘good’ or ‘bad’? How does one weight poor performance in cancer care (very serious) against excellence in delivering babies (huge volume)? How long must patients wait in Emergency services to cancel out the benefit of having an enviable record in avoiding hospital acquired infections?

No-one can answer those questions. Which is why I maintain there’s no such thing as a good or bad hospital: there’s no way of saying that one hospital, taken as a whole, is either particularly bad or particularly good. It’s much more likely that it will be under-performing in some areas and doing well in others.

That’s why when the dust settled, the people of Stafford became upset that in the aftermath of the original scandal, the hospital’s Accident and Emergency services were cut down to daytime only (8:00 a.m. till 10:00 p.m.) At any other times, people requiring emergency hospital care travel to Stoke (20 minutes by car) or Wolverhampton (25 minutes). Alternatively, they hang on for the local service to open and join the massive queues that now form at the entrance: Stafford, like most places around the country, has seen a big increase in Accident and Emergency attendances.

As a result of yesterday’s decision, Cannock hospital is now to be brought under the management of Wolverhampton, and Stafford hospital under Stoke’s. Will that make things any better? Only if management by executives remote from the place where services are being delivered is likely to lead to better performance than was achieved by local control.

Why have we reached this position?

Because life’s a lot simpler if we can identify a whipping boy and vent our animosity against it.

Because Stafford Hospital is synonymous with bad quality and there is mileage for politicians in running it down, and no mileage at all in defending it.

Because, in short, there’s nothing easier or more likely to generate good press coverage, than to attack a ‘bad hospital’.

Even though we all ought to know there’s no such thing.