OK, fine, I’ll talk about that pterosaur / albatross paper

The title here might give you the impression that I’ve been deluged with e-mails about this paper, and although I was away when it came out and thus missed most of the online discussion I‘ve not really spoken to any of my colleagues about it and not had a single request to talk about it. The title here more refers to my own mental battle as I have been determined not to comment on it on the blog and save anything I’d want to say for a formal publication (if I ever do one).

I have made a conscientious effort on the Musings to try to avoid criticising academic colleagues, papers or organisations directly (not that I have always succeeded mind) and I’ll try to keep that up here. That might sound odd given that there is only really one paper I could be referring to here and thus any comments are going to be fairly obviously related to it, but what I will do is hold it up as another example of the failure of peer review.

For those who don’t know a recent paper came out positing that some pterosaurs could perhaps not have been able to fly on albatross-like wings as they would simply not have been able to generate sufficient power to keep aloft if the winds were ever low and would be limited to more vulture-like broad soaring wings (more here). OK, so how is this a failure of review? Well the short version is that little of what was said about pterosaurs was new, interesting or especially accurate (in my opinion, and if you hop over to the DML you will see a ton of other people with the same views – I will add at this point that the albatross stuff was great research. Really. Innovative, exciting and informative).

There are seven authors of this paper and one assumes at least two referees and an editor went over the paper. I would bet a sizeable sum of money that no-one who knows any serious amount about pterosaurs saw this paper before publication (or if they did, their likely comments and objections were overruled or ignored). So in a paper that explicitly discusses pterosaur flight in the title, abstract and discussion and represents a significant chunk of the writing and certainly the impact of the paper, no pterosaur expert had any contribution to it at all. Now I will suggest that the authors should at least have contacted someone for a little advice (no need to add extra authors to the paper if they don’t want to, advice is freely available and there are plenty of people they could have spoken to) and that is an error on their part. Worse though is that clearly the journal decided that no pterosaur expert needed see the paper to offer their knowledge, expertise and experience to this topic despite the significant role it has in the paper.

Peer review is supposed to correct for just this kind of thing. The authors are flying a little blind here and treading on territory well outside their range of normal knowledge / experience and could do with some help. There’s nothing wrong with that, but like switching from a plane to a helicopter, you might want a guiding hand or a leaf through the instruction book before you start. They should have sought some, but if not the journal can help provide that. If not then the next paper I write on theropods I might as well stick in a few tangential lines about ducks if I can find some link between them and trust that I don’t need to read up on ducks, ask and an expert about ducks, and the journal will only ask dinosaur experts to judge the duck-based content. That way I can publish stuff on ducks and get it into the peer-reviewed literature when rather by definition the stuff on ducks has not been reviewed by ‘peers’. The editors should not allow this kind of thing to pass ‘unseen’ in my opinion and should have it taken out, or suggest that an expert referee see it, or ask the authors to seek additional help / collaborators none of which I think are unreasonable, or difficult to achieve. It should not represent a big investment in time or effort for any of these groups and indeed I would suggest that it represents a fundamental part of the criterion of good science and good review.

If the editors did get a pterosaur expert to act as a referee I would be very surprised if he had let it through in the state that it was published in. Perhaps one did read it and raised a number of points which were not changed in which case the issue is still one of editorial control. Why did the editors not think that a group of people who work on extant birds could make a good fist of pterosaur anatomy and ecology without any background or assistance? What is the point of taking papers for review if they are not subjected to the kind of rigorous assessment that is required? It’s silly and pointless and means that good papers by good authors will make basic mistakes that others have to come along and clean up later on. It’s a waste of their time and a waste of our time.

Discussion of this general theme with colleagues suggest that some think this kind of thing is reasonable since we, as researchers, should be free to speculate and try to link diverse research branches together and that a few mistakes in the literature is a good thing if it comes as a minor expense with novel ideas and new lines of research. I take the point, certainly, but there are a few things I would reply to this line.

First of all, it can take just a few frivolous lines of nonsense in a paper to cause another researcher to waste a ton of time formally correcting it in the literature. In fact there are short frivolous papers out there that really clog up some areas of research with nonsense and can require Herculean efforts to correct when they should never have been published in the first place and would not have been with decent and pertinent review. Second in this day and age it is easy to the point of ridiculousness to contact an academic expert or two in just about any aspect of research at all. Even ten years ago had I wanted to say much about ducks I would struggle to find much literature or communicate with an expert. Now I can find hundreds of papers online in minutes and send an e-mail or two to world authorities asking for help in a matter of moments. I can potentially save myself and others lots of work and confusion by taking the time to get a bit of help and advice and it’s very, very easy to get. Thirdly the concept I think shows a fundamental bit of disrespect to other academics – why spend years working on ducks if a theropod expert to trample over your evidence and theories with no obvious recourse to actually speaking to anyone or reading relevant literature which you went to the trouble of generating for him to read if he ever felt the need?

I agree that we do not want to stifle creativity and innovation, but there is a difference between reasoned speculation based on even a small amount of knowledge and research and wild speculation based on well, not very much at all. There is no place, I feel, for papers or significant parts of papers where the authors cannot demonstrate they have dealt with the issue properly AND it has not been approved by an authority on the subject. It’ll make this more confusing, not clearer, and will add errors, not innovation.

5 Responses to “OK, fine, I’ll talk about that pterosaur / albatross paper”

  1. 1 Andy 01/06/2009 at 1:40 pm

    Unfortunately, this sort of phenomenon (claims on topics for which the authors are underqualified) is rather common in all fields. . .and, I don’t know that editors necessarily do enough to nip this in the bud. Dinosaurian functional morphology (not that it’s any better or worse than other fields, just that it’s one with which I’m familiar) is pretty bad in this way. I see a lot of tendency to just cite other VP workers, rather than going to the “real” neontological literature. . .and of course the converse is true for neontologists. Ah well. . .may this cycle be broken within our lifetime!

  2. 2 David Hone 01/06/2009 at 4:19 pm

    I agree entirely. There is noting wrong with a *little* speculation that will tie things together and inspire more research, but when there is lots of literature and expertise readily available on a topic then this kind of thing is unnecessary and unfortunate. Historically science has been a bunch of separate branches and we need to start ting them together with ever increasing frequency to do more and better science.

  3. 3 Manabu Sakamoto 02/06/2009 at 7:16 pm

    Well, if you want neontological data support for dinosaur functional morphology, I’m preparing a manuscript on just that in biting mechanics…

  4. 4 David Hone 02/06/2009 at 10:09 pm

    Well it’d be a good start!

  1. 1 Pterosaurs flew, who knew? « Dave Hone's Archosaur Musings Trackback on 15/11/2010 at 6:51 pm
Comments are currently closed.

@Dave_Hone on Twitter

Enter your email address to follow this blog and receive notifications of new posts by email.

Join 553 other followers

%d bloggers like this: