Obvious vs Surprising

I wrote a draft for this post about six months ago and never got around to finishing it off and posting it, but some recent stories in The Independent newspaper has triggered my ‘irritated by the media’ switch so I’m back on the case. I’ll get to the stories themselves in due course, but the thrust of this post is the way in which the media often characterises scientists and plays on incredibly simple sterotypes of both the people and thier research for cheap headlines. While I am happy to admit that there is generally something (often quite a lot) in the origin of a stereotype (and certainly at the risk of sounding like a classic NIMBY / Daily Mail or Telegraph reader / political correctness nazi) one wonders if they would be allowed to do the same kind of characterisation of a race or religion. The constant caricatures and dismissals of researchers as boffins, egg-heads and nerds or crass puns like ‘sexperts’ is hardly promoting a friendly attitude or one that dispalys condifence or respectability in research.

Apparently much research comes under the manner of “scientists study the obvious” and “boffins overturn common knowledge, it’s really interesting”. These are so predictable and common you can almost read the title of a research paper and fill in the blanks of the headline before it’s published.

Either you are wasting time and money testing something ‘everyone knows’ or you are apparently brilliant for testing something everyone knows and getting an odd answer. It never seems to occur to them that the important thing here is the testing not the result but the testing, and that rather by definition the common knowledge can’t be too great given how often it gets overturned. Either this is worth doing, or it is not, it is not ONLY worth doing when you know you’ll get an interesting answer, though the way these are reproted you might be tempted to think so.

Often there are excellent reasons for retesting something even as well established as say germ theory as you might find something new, a new test can be attempted, new organisms or new strains can be tested, old datsets examined or revised, dodgy datasets revisited and more. As we have seen recently, simply the application of the headline writer and subeditor can distort a perfectly worthy and interesting, even important piece of research, if otherwise well reported. Which brings me back to The Independent.

The first link takes you to, and I only wish I was kidding, and article entitled “From the Universiy of the Bleeding Obvious“. Hmmm, so no up-front bias there then. The other two I have included you will see came in at around the same time, and while I admit not exactly polar opposites, do have the ring of challenging the status quo by re-examining things that people are well aware of and finding them not to be true. They are here and here. Aside from the obvious hypocricy of this position – basically bending the titles to fit the stereotypes and giving readers a prejudged bias in reading them, the actual content itself of the ‘obvious’ makes it clear that what they report and what was done are simply not the same.

Anyone with a passing knowledge in how science is done can see that either a) there might be perfectly good reasons for doing at least some of these studies, or b) so much is left out that it is impossible to say if was a good idea or not. I suggest you read the article first, but it is so simplistic to pick them apart it’s barely worth it (only the first is given in any detail). I could of course be wrong and thse might be terrible peices of research that should never have been published (there are enough of them) but given how science works, I can’t help but think that my hypotheses are largely correct.

1. Straight men like seeing women in bikinis. Well yes, obviously. But what is clear from the report is that the researchers are measuring not just the fact of this, but the magnitude. And doing it by seeing which parts of the brain are affected. So we could also say that researchers use something with a known outcome (men like sexy women) to actually get a measure of how this works in the brain and in what way, and how this might affect behaviour. But obviously that’s not very insteesting and we can’t have a dig at the researchers for it. Researchers who, I might add, I very much doubt thought their work would appear under this headline when they agreed to give out quotes on their research. The actual details are quite good as to what was done and the surprising results they got, if you read that far of course and have not just read the first few lines and given up.

2. Fake orgasms are different to real ones. Yet in the THREE LINES this gets they point out that this was doen with brain scans to see whaich part of the brain responded and how. In other words, real and interesting science was done and they even manage to report on that, while dismissing it. It’s impressive in its own way.

3. The rhythm method is unrelaible. And more interestingly, women can get preganant at ANY point in their cycle. That is news. Again, report the interesting and important point while saying it was silly. Or to be more accurate, making up a heqdline that I doubt was even vaguely assocaited with the original research to create a stick to beat them with. Nice.

4. Men like women in red. Read their words then get them to write out correlation is not causation until the end of time. Also point out that maybe men *now* like red becuase of the past assocaition and not just as a general evolutionary consequence which is (I would suspect) what they were testing. Even which sades of red were chosen would be illuminating.

5. The fitter you are the greater your life expectancy. Tested in service personnel only. So just maybe they were *also* looking at other factors like length of service and combat tours and stress or injuries to see what else affected them. In short, you pull out one factor that gave a predicatable result out of who knows how many comparisons and hammered them for collecting their data well and analysing it thoroughly. Pure class guys.

6. Living by roads increases asthma. Probably does, but may not, at least not significantly compared to other things (like living near power stations or railways) and again, what else was being tested and how were these people compared? How bad was the asthma? Did it correlate with age or other allergies or illnesses or pet ownership?

7. Hurrying causes your attention to wander. Does it? Many people might say they have no problem focusing on a task when in a hurry, or are still receptive to outside effects, so n fact this probably contradicts the worldview of many people. And again, magnitude and not just absolutes are obviously involved.

8. Giving up smoking helps your lungs. Well yes, again, obviously. Except, maybe not. Maybe the damage might have been so great that stopping made no difference. Isn’t that worth testing? Finding out just how much improvement was gained after how much time is also useful if say, you need to operate on a recent ex-smoker and don’t know if they can take the treatment.

9. Binge drinking causes loss of balance. The minimal space this is given still manages to imply that the loss of balance might well be in day-to-day life and not just when drunk. Also again, if they are measuring when people lose corordination, and how much, over how long a period of time, this is useful data. Are people safe to driave say, 12 hours later or not? I’d like to know that actually.

So there you go, crass reporting that even in their microcosm of tiny reports manages to manipulate the data, the results and the quotes towards an easy target – the researchers themsleves, rather than actually, you know, reporting on the actual science. Judging by the comments on the article, they succeeded admirably. And this is one of the better newspapers for science reporting. They at least *have* a dedicated science section. Oh dear.

7 Responses to “Obvious vs Surprising”

  1. 1 Christopher Taylor 01/04/2009 at 9:13 am

    The fitter you are the greater your life expectancy. Tested in service personnel only.

    I recall hearing somewhere that studies on humans often use members of the military as subjects, because members of the same military unit will be eating much the same food, following much the same routine, etc. This cuts down on the rampant variables that a such a persistent problem in studying an organism that you can’t just lock up in a laboratory with total control over its environment.

    Also, may I point out that exercising is, after all, a process of deliberately submitting the body to short-term damage in order to increase one’s long-term ability to handle that damage. So I think it’s a perfectly valid question as to whether the risks vs. benefits are always correlative, or whether there’s a point where the harm caused by the short-term damage outweighs the long-term benefits (think of the injury rate among professional sports players, for instance).

  2. 2 David Hone 01/04/2009 at 9:25 am

    Another good point that I missed. Which of course really just reinforces the more general point I am making. It’s easy to pick on a study if you cherry pick from the results, or don’t explain what or how the researchers were testing. I suspect I could do something simialr to just about any major scientific study with a bit of thought.

  3. 3 Dave Godfrey 01/04/2009 at 4:17 pm

    I wouldn’t be surprised if half of those studies were repetitions of previous work with better control procedures, different statistical techniques to account for confounding variables, or any number of other tweaks.

    Journalists point and laugh, but I bet most of those trials were pretty cheap to run. They may not even have gathered new data, and just found a different way to process existing results.

  4. 4 Mike Taylor 01/04/2009 at 4:43 pm

    Is it just me, or is there something really funny about the caption to the picture in the third linked story? “Jerry Lee Lewis, the rock ‘n’ roll star infamously married his 13-year-old first cousin once removed, 10 years his junior, in 1957. His popularity initially fell but recovered when he began performing country and western music.”

  5. 5 David Hone 01/04/2009 at 4:49 pm

    Hi Dave, yes another good point about trail design and certainly the low costs of data mining.

    Mike, errr, yes, that *is* funny, but *obviously* this is a case of correaltion and not cause. Probably.

  6. 6 Jerry D. Harris 06/04/2009 at 10:32 pm

    Apparently much research comes under the manner of “scientists study the obvious” and “boffins overturn common knowledge, it’s really interesting”. These are so predictable and common you can almost read the title of a research paper and fill in the blanks of the headline before it’s published.

    …reminds me of this

  7. 7 David Hone 07/04/2009 at 8:03 am

    Yep, I can go with that Jerry.

Comments are currently closed.

@Dave_Hone on Twitter

Enter your email address to follow this blog and receive notifications of new posts by email.

Join 553 other followers

%d bloggers like this: