Saturday, 19 September 2009

A sorry saga - the crumbling cookie

This week saw the publication of a story that "more than half of all Britons have been injured by biscuits."

The story received plenty of coverage in the British national press, and if true, is surely one of the UK’s most pressing Occupational Health and Safety concerns.

Australian’s are culturally similar to Britons, so I asked around the office and found that no one around here had ever been injured by a biscuit. So I thought back to my 18 months in the UK and to my recollection I was never injured by a cookie, and I certainly don’t remember a horribly outrageous and dangerous biscuit-eating culture.

Is biscuit-eating really an extreme sport in the UK?

It's quite hard to know where to start with this article, but it is almost the perfect example of "how not to report science in the newspaper."

We’ll start with the actual reporting before dealing with the science behind the story. The early claim that "more than half of all Britons have been injured by biscuits" is diluted a little in the following paragraphs, in which it becomes "an estimated 25 million adults have been injured while eating during a tea or coffee." Injured while eating during a coffee break covers a lot more than being injured simply by eating a biscuit. But a quick look at the original "scientific report" (see later) mentions nothing about 25 million adults in the UK (roughly half the population) being injured in such a way, nor is it possible that anything in the report could be extrapolated to such a number. The company that conducted this research, Mindlab International, states that their study says nothing about the proportion of people being injured in this way and that they have no idea where the number came from.

The article then goes on to say that "at least 500" people have been hospitalised because of their injuries. This is a slight difference to 25 million, and indeed 500 out of 64 million people in the UK is such a small number that it is hardly worth reporting. You can see where the number 500 comes from by looking at The Home Accident Surveillance System 2002 report.

The next thing to do is look at what constitutes a biscuit-related injury. My favourite hidden biscuit dangers, according to The Telegraph, are:

  • flying fragments,
  • people poking themselves in the eye with a biscuit,
  • falling off a chair reaching for the tin,
  • "one man even ended up stuck in wet concrete after wading in to pick up a stray biscuit,"
  • choking on crumbs,
  • being bitten by a pet or 'other wild animal' whilst trying to get their biscuit.
Heavy hitting stuff!

The article then goes on to rate the dangerous biscuits, using the Biscuit Injury Threat Evaluation Index (BITE Index, get it??). Of course there is no explanation of how these numbers were obtained and what exactly they mean:

Custard Cream 5.64
Cookie 4.34
Choc Biscuit Bar (eg: Rocky) 4.12
Wafer 3.74
Rich Tea 3.45
etc. etc.

It’s at this point you start to wonder why any of this was reported at all. Well, the study was commissioned by Rocky, a type of biscuit made by Fox’s. You’ll note how they are the only example of a biscuit given in the above list – all the rest are generic biscuit types.

What we have here is a perfect example of a company trying to get some publicity by commissioning a pseudo-science study and putting out a press release. Because "science" and "maths" are used in the article, an impression of credibility is given to the story. This is precisely why people say things like "you can prove anything with statistics" and think anyone with an interest in science is either mad, a boffin or has no connection to the real world. It’s ludicrous that this kind of story can make it into the UK national press, and it’s dangerous because it not only gives people the wrong impression of science, but some people might actually believe it! And whatever happened to fact-checking journalists? Are we so short on news that this gets reported as such?

Ben Goldacre has a nice article on how companies use "news" like this to get around advertising rules. It makes me sad that advertising, which at least most of the time we know is advertising, has rules against misinformation but that news articles don’t.

Now to take a look at the actual science behind the story, and it’s about as good as the reporting. Needless to say that the survey data used to create the mathematical BITE formula is not publicly available, but this is no surprise. It should not surprise anyone that the report is not peer reviewed or published anywhere credible, but let’s not set our hopes too high here. You can find the report on the Rocky site. Let’s also ignore the bit that says the research is independent, and skip to the fact that:

"The study, supervised by Mindlab International Ltd scientists, involved ten students aged 16 -18 to investigate the physical properties of biscuits."

So this report was done by high-school students?

Here is the final BITE equation:


Pretty isn’t it? Looks complicated, it must be correct. One of the weird things about this report is that every equation looks to have been scanned in from somewhere else – not that this influences how we analyse it, but it doesn’t look very professional.

There are two parts to this equation: biscuit dependent and consumer dependent factors. Biscuit dependent means the factors that are derived from biscuit behaviour, and consumer dependent means the factors found through survey data – these are not given or explained.

Six possible injuries from biscuit-eating have been named:
  • Eye/ear/trachea irritation caused by crumbs.
  • Scalding – due to splashes caused when a piece of dunked biscuit falls into hot liquid.
  • Back Injury, hernia, muscular problems from picking up dropped biscuit pieces.
  • TMJ (temporomandiular joint) syndrome to jaw by frequent biscuit chewing.
  • Workplace injury due to being distracted by the sound of biscuits being broken.
  • Dental Damage due to biting on a hard biscuit or something within the biscuit, such as a nut or piece of hard chocolate.

Have a read over them again… TMJ (temporomandiular joint) syndrome to jaw by frequent biscuit chewing....

Going on a bear hunt does an excellent review of the whole sorry saga, including a look at the role of Mindlab International – the blog lists its "favourite plausible fudge factors, false assumptions and downright crazinesses" - check them out for a good laugh.

These are my favourite two:
  • A breaking biscuit has a noise level of between 5 - 15dB. This is apparently loud enough to cause injury through distraction in the workplace. A typical whisper is 20dB - rustling leaves are around 10dB. I would estimate that my generally quiet workplace operates with about 35dB of background noise, and if I have my ipod on, then that's probably about 60dB (louder when I turn on angry music and read reports like this one).
  • The report models a crumb getting in your ear as equally likely as getting in your eye. This seems a bit like the famous Seinfeld spitting incident – the crumb would have to rise upwards from where the biscuit is broken, then curve back around into the ear. That’s one magic luggie crumb.
I love their statement:

For obvious safety reasons, laboratory work did not involve trials in which vulnerable body parts were repeatedly impacted by biscuit crumbs!

Careful of those deadly crumbs!

Private funding of scientific research is very important – plenty of good, independent science is done this way. But this particular example is just ridiculous! Mindlab is part of the Sussex University Innovation Centre and so presumably should be above this kind of nonsense. They say:

Our unique research methodology combines hard neuroscience and sophisticated mathematical analysis to provide reliable and timely answers of immediate practical benefit.

Given that the survey data was given to them by the company that commissioned them to produce the research, that rather spurious analysis was done by high school students, and that there is absolutely no practical importance in this work, I think they pretty much failed their own mission statement.

This kind of thing is fine for generating an interactive tool for a biscuit website – indeed it is fun – but let’s not dress this up as science or news.