or…Some thoughts on Personal Responsibility and the Peer Review Process
Eric Steig
Ryan O’Donnell made a series of serious of allegations against me at ClimateAudit, in the context of our friendly dispute about whether his new paper in the Journal of Climate supports or ‘refutes’ my own results, published in Nature.
To his credit, Ryan has offered to retract these allegations, now that he is a little better acquainted with the facts. However, it is still important, I think, to set the record straight from my point of view. There were such a great number of claims about my “dishonesty,” “duplicity” and [implied] stupidity, all of which are untrue, that it really isn’t worth trying to respond in any detail. Just responding to the main two ought to suffice to make the point.
“Eric recommends that we replace our TTLS results with the ridge regression ones (which required a major rewrite of both the paper and the SI) and then agrees with us that the iRidge results are likely to be better . . . and promptly attempts to turn his own recommendation against us.”
“[in his RealClimate post]…he tries to … misrepresent the Mann article to support his claim [about the iridge routine] when he already knew otherwise. How do I know he knew otherwise? Because I told him so in the review response.”
While it is quite possible that O’Donnell believes both of these claims, they are both false, as it is rather easy to demonstrate.
First, I never suggested to the authors that they use ‘iridge’. This was an innovation of O’Donnell and his co-authors, and I merely stated that it ‘seems’ reasonable. As O’Donnell’s co-authors are fond of pointing out, I am not a statistician, and I did not try to argue with them on this point. I did, however, note that previously published work had shown this method to be problematic:
“The use of the ‘iridge’ procedure makes sense to me, and I suspect it really does give the best results. But O’Donnell et al. do not address the issue with this procedure raised by Mann et al., 2008, which Steig et al. cite as being the reason for using ttls in the regem algorithm. The reason given in Mann et al., is not computational efficiency — as O’Donnell et al state — but rather a bias that results when extrapolating (‘reconstruction’) rather than infilling is done.
Second, I was the reviewer of the first three drafts of O’Donnell et al submission. However, I did not the review draft four, which was the published one. , and which is markedly different from draft 3 [note correction: it has been pointed out that it’s not really very different; in other words, my criticisms of draft 3 were ignored]. Nor was I ever shown their response to my comments on draft 3, so I did not in fact ‘already know’ what O’Donnell claims I did. It appears that the editor was swayed by the arguments that I was not a helpful reviewer. In other words, even if one believes that I was “bullying” them into showing particular results, they still had the last word (as any author should).
The fact of the matter is that my reviews of O’Donnell’s paper were on balance quite positive. I wrote in the confidential comments to the editor in my very first review that
I emphasize that I think that a fundamentally reworked version of this manuscript could potentially provide a useful scientific contribution, and many of the points made do indeed have scientific merit. Indeed, the authors have done a very thorough analysis, and are to be congratulated on this.
In my second review, I wrote that “O’Donnell et al. have substantially improved their manuscript … and clarified a series of items that led to some confusion on my part.”
With respect to O’Donnell’s lengthy discussion of the technical aspects of the difference between our papers, I’m not complaining. It is possible to have a disagreement — or even to be wrong — about the technical aspects of a paper without being ‘duplicitous’. The dependence of any analysis on the technical aspects of the methodology are completely legitimate subjects of discussion, and it is important to be clear about what does and what does not depend on those choices. People who want to see what the data are saying about the real world will focus on the similarities, people who are focussed on proving people wrong will focus on the differences. This is how O’Donnell and I can (legitimately) disagree about what their results mean.
The reality is that editors, not reviewers, make decisions about what is acceptable and what is not. Any comments I made as a reviewer of O’Donnell et al.’s work would have been weighed against what other reviewers said (and obviously were, since the main criticism I had of the paper was not ever addressed), not to mention the responses of the authors themselves. And the decision about what content eventually winds up published is still ultimately up to the authors. If the authors feel that they are being bullied into presenting their results in a particular way (as is the allegation here), then they have the choice to withdraw the paper and submit it elsewhere, or complain to the editor. But once they have signed off on the paper, it is their paper, and blaming someone else — reviewer or editor — for its content is simply passing the buck.
It’s perhaps also worth pointing out that the *main* criticism I had of O’Donnell’s paper was never addressed. If you’re interested in this detail, it has to do with the choice of the parameter ‘k_gnd’, which I wrote about in my last post. In my very first review, I pointed out that as shown in their Table S3, using k_gnd = 7,
“results in estimates of the missing data in West Antarctica stations that is further from climatology (which would result, for example, from an artificial negative trend) than using lower values of k_gnd.”
Mysteriously, this table is now absent in the final paper (which I was not given a chance to review).
Some months ago, O’Donnell cordially (though quite inappropriately) asked me if I was one of the reviewers, and also promised not to reveal it publicly if I didn’t want him to. I told him I was, but that I would prefer this not be public since the ‘opportunity for abuse’ was simply too great. Talk about prescience!
Many of my colleagues have warned me many times not to trust the good intentions of O’Donnell, Condon, and McIntyre. I have ignored them, evidently to my peril. But you know what has given me the most pause? The fact that a number of my colleagues and many otherwise intelligent-seeming people still seem to treat these guys as legitimate, honest commenters, whose words have equal weight with, say, those of Susan Solomon or J. Michael Wallace, or, for that matter, Gavin Schmidt or Mike Mann or myself. As a reporter wrote to me today “it’s simply impossible for a lay observer to make a judgement on his/her own.” Really?!
Perhaps there is a silver lining here. Perhaps the utter silliness of the shrill accusations that O’Donnell made against me — based on a version of the facts, in his head, that are demonstrably and unequivocally false, coupled with the fact that he then retracted them (or at least has promised to do so), will help more people see what the steadily growing list of other scientists who’ve been accused by McIntyre and his associates of plagiarism, dishonesty, data manipulation, fraud, deceit, and duplicity have been telling me for years: these people are willing to say anything, regardless of the cost to others’ reputations and to the progress of legitimate science, to advance their paranoid worldview.
I’ve even got a name for the clarity this affair would seem to offer: O’Donnellgate.
Sadly, attacking climate scientists by mis-quoting and mis-representing private correspondences or confidential materials appears now to be the primary modus operandi of climate change deniers. To those that still don’t get this — and who continue to believe that these people can be trusted to present their scientific results honestly, and who continue to speculate that their may be truth in the allegations made over the years against Mike Mann, Ben Santer, Phil Jones, Stephen Schneider, Andrew Weaver, Kevin Trenberth, Keith Briffa, Gavin Schmidt, Darrell Kaufmann, and many many others, just because they ‘read it on a blog somewhere’ — I’d be happy to share with you some of the more, err, ‘colorful’, emails I’ve gotten from O’Donnell and his coauthors.
If you still don’t get it, then I have a suggestion for a classic short story you should read. It’s called, The Lottery, by Shirley Jackson.
25 Steve Bloom: Revkin: Did you not see the Shell Oil advertisement beside Revkin’s blog in the recent past? Fossil fuel influence is rather pervasive in journalism. Couple that with the fact that science and math courses are not required in journalism school. I doubt that Revkin has a choice.
Gavin: What is \ridge regression\? What is \iridge\?
Eric Steig: Your response in 42 is correct, but I thought you already knew who McIntyre is.
Why am I commenting on this subject? Oh, because the denialists make hay out of nonsense. I hope to not have soap operas to comment on in the future. Woops, not RC’s fault. You had to do it. It will probably get worse.
I hope Eric isn’t serious about refusing to review a paper submitted by this crew (#42). I think one of their goals with their sort of public behavior is to scare off legitimate reviewers, which isn’t good for the peer review process.
[Response: True, but of course if it gets around that no one legitimate will review their work, then it won’t get taken seriously at all. But that’s not my goal. I simply don’t want anything to do with these people. In fact, I may not bother with a rebuttal to Journal of Climate, because in a couple years temperatures in West Antarctica will probably have reached such an extreme that none of our ‘reconstructions’ will matter. In fact that may have already happened — see e.g. Record warming in the South Pacific and western Antarctica associated with the strong central‐Pacific El Niño in 2009–10. –eric]
Oh, and, MrPete … RyanO promised Eric that he wouldn’t divulge the fact that Eric was the first reviewer, after inappropriately trying to pierce the anon review process by e-mailing Eric directly.
After so promising, he publicly “outed” Eric anyway.
I’m very interested in your moral defense of this crap.
Notice: I’m done with this conversation. You can ask all you want about what I ‘really’ meant when I said that O’Donnell’s idea of using ‘iridge’ was a good idea or not, or whether I agree with O’Donnell’s latest commentaries, or whether I’m embarrassed that O’Donnell et al. discovered significant warming in West Antarctica before (oh, wait, after we did), or what I think about Steve McIntyre defining West Antarctica as ‘any place not warming’ and the West Antarctic Ice Divide ice core site is on the Antarctica Peninsula, or whether I think Steve McIntyre actually read the paper he is a co-author on, but I don’t intend to waste my time answering.
Back to science.
None of this to imply that further reasonable discussion of peer review or Antarctic climate isn’t welcome. It very much is. I just won’t be participating much for the forseeable future.
Best wishes to all.
Eric
Eric: I don’t blame you, one bit.
These people aren’t interested in the search for knowledge, only in trying to ruin your reputation and discredit you.
Just keep plugging away and publishing.
My guess is that without your review, they would’ve never gotten published in the first place.
Dr. Steig,
I’m sorry you had to deal with this. This “noise” is precisely what drives the skeptics, so I think it’s a good idea to stray away from it. Keep up the good work
Prof. Steig #54 I don’t really have anything to add to the science, and I don’t really know much about peer-review. I do know something about public vilification. You are under no obligation to expose yourself to it. The work you do is already enough; if you decide to cease or curtail your participation in the wider debate I can hardly blame you. However, your contributions will be missed.
leaving apart the unpleasant aspect of personal attacks (on both sides), I suggest everybody to do an interesting thought experiment : imagine that this debate (the scientific one I mean) would not deal with Antarctica , but for instance the Martian polar caps- just about correct mathematical procedure to reconstruct the map of temperatures from the satellite and some landers data for instance. Obviously the debate would then have been confined among a very small set of scientists, but it may have happened that some of them complain about the behavior of some other ones, may be in the unrefereed letter “News from mars” , with a harsh answer from the other scientist in ” Martian express” (“News from mars” is published by a former PhD student of the first one, and “Martian express by a coauthor of many papers from the second one). Well, things like that happen even in science.
The question is : would anybody here have a clear idea of who is right and who is wrong, both concerning the scientific method of reconstruction, and the personal attitude ? would have it not concerned climate science, would it be that clear ?
[Response: This is a nice thought experiment. I can in all honesty tell you that I think it would be clear that O’Donnell made several real improvements. I said as much in my reviews, and I’ll say it again here: globally, for the satellite era, O’Donnell’s solution may be more accurate. However, the issue here is that his does not appear to be the better solution in the particular area of Mars that we all think is important, nor in extrapolating into the data-poor past. That was the point of my previous post: O’Donnell is demonstrably less correct where it matters, and the question is why. (How O’Donnell turned this from a purely scientific question to an attack on my integrity is completely baffling.)–eric]
In the previous thread, I asked Eric to show his reviewer comments, but O’Donnell made that an unnecessary request. After reading those comments, I already determined that O’Donnell’s claims of duplicity on the iridge issue was absolutely not present in the reviewer comments versus the RC piece. It’s fun to see Eric making the same point (makes me feel a bit sane). However, I am also a seasoned reviewer (and author), and thus likely better capable of judging this author-reviewer play. Many here probably have the same experience (or more).
We should therefore not blame the journalists too easily for not understanding some of the discussion, and unfortunately several of those parts of the discussion that go to the heart of the matter. For them this whole affair has probably become “he said – she said”, in which both are right…and both are wrong. It will then depend on who they trust most. And let’s be honest, for quite some people here and at CA, the same will be true.
[Response: This is a thoughtful comment, and I agree. The problem with the journalists is that they really ought to learn from O’Donnellgate that it’s possible to discern who can be trusted and who cannot. They don’t seem to have any trouble distinguishing, say, Barak Obama and Osama bin Laden; or EarthFirst! vs. the Sierra Club; note that in these examples, most people have issues with both sides, but have no trouble making a clear distinction about who’s view to trust more.
Oop, there I go back to responding! Hard to resist when the comments are thoughtful.–eric]
Thank you for clearing the air, Eric. Those of us who have been in the scientific trenches know that peer review is not a dinner party. You put a lot of effort into encouraging RyanO and crew to publish and to make their submission better. Too bad they had they soil the sandbox.
Quote, Steig, “I am not a statistician”.
You do know that such an admission will forever be held against you.
@48 BPW. When I see “BPW” asking the same kinds of questions of O’Donnell that BPW is asking here — questions not present in the first 250 comments on the relevant thread at CA — I’ll stop questioning your intellectual honesty.
Re “[Response: Actually, I meant *privately* with people I reason to trust will not to make them public.–eric]”
Are these emails on the UWS server?
“Calling all hackers!”
Yes, I’m kidding, honestly.
I’m sure I wouldn’t be as tacit or as principled as you in keeping them private, given recent circumstances. Perhaps John Nielsen-Gammon should/would like to view them.
Eric, have yourself and Ryan O’Donnell ever met face to face? I hate to seem impertinent, and apologies if you see it that way as it’s not my intention, but I saw a possible positive outcome from the initial discussions between the two of you here at RC. I can’t help but feel that just the two of you meeting up (and **ONLY** the two of you) would be a positive step, rather than everything happening via blogs and emails which are notorious for being easily misinterpreted without the ability to quickly refine or clarify a point face to face.
@ Ryan O’Donnell, in case you’re reading this, same goes to you.
The way it’s turned is too bizarre. It doesn’t add up at all.
(You guys don’t have to post this here at RC if you don’t want to, it’s just a suggestion. Don’t take it the wrong way.)
Sorry, in my previous post (in moderation) “Neilsen-Gammon” should have read “Anthony Broccoli”, though on reflection…
dhogaza and others,
a) Please don’t infer anything further than what I said, which is that discussion in terms of the journal’s policy seems off base. I have no time to get into the actual issue, which is why I said nothing about what I really think. I’ve not fully read/understood the details on this; I would love to. (Hint: even friends who agree often find it hard to communicate well about complex topics; here we have “opponents.” Looks like plenty of room for more humility on all sides.)
I’ll add the following, made as brief as I could, to share a bit of the real-world story behind the story. This is as good a platform as any, if our august host is willing (smile)…
b) I only took the time to post those links because I have the (very small) responsibility of maintaining the *.info site, which as you might guess has been getting quite a bit of traffic. I was distracted enough by the hullabaloo that I noticed the need for a “diff” of revs 3 and 4 which is something useful I can easily contribute. Beyond that (and even for that) I don’t have time to engage…
c) I don’t know how serious you are about wanting to see something published beyond the coring data itself (which of course has been fully placed online for quite some time.) I would love to have time to prepare and publish an article covering the coring data and more (I also stopped by LTRR a couple of times while in the area, took photos and would love to promote the need for funding to address other behind the scenes aspects of dendro science.) Something tells me nobody’s going to cover the cost of my time, which means this all has to fit around other realities…
Only problem is, my real world life has been affected a bitby my own medical challenges, and my (only) coworker who has complications of kidney failure and an (associated) quad-bypass heart surgery. That’s why I’m not posting much lately, anywhere. BTW I’ve not had time to ask but Steve McI has been quiet lately too; I suspect his RW life is a bit distracting as well.
Bottom line: I’m not asking for sympathy, just urging even more patience with the “citizen science” world. You cannot imagine how frustrating it is to have incomplete self-funded work, particularly when I’m intimately aware that several dendro researchers all died in their 50’s with incomplete work!
Having completed a few groundbreaking volunteer projects (there’s a reason I’m an expert in GIS, group e-communication, etc) with little or no outside funding, I’ve learned it can take 10x as long (literally!) to get it done as a dedicated volunteer, and many people burn out along the way. I’ve had the gift of not burning out so far… we still get there, just takes a lot longer.
I apologize in advance for my likely unseemly lack of further immediate responsiveness.
While not wanting to dismiss the emotional impact of being accused in this way, I do wonder how much notice the rest of the world take of these disputes? In continuing the debate through further rebuttals, I think you risk allowing the deniers to control the scientific agenda. My own feeling (for what little it is worth as one who thankfully doesn’t have to bear the emotional burden) is to let time, the community and the data decide, and move on to something more worthwhile.
One thought: Has O’Donnel ever published before? I remember for my early publications I took referees comments and feedback from other scientists very personally. It’s taken years to grasp at an emotional level that critique is not attack.
It’s quite clear that there is some confusion amongst the skeptics about publication. From their slightly paranod comments, they seem to think that journals are obliged to publish whatever is submitted.
And many thanks to Eric for taking the time to ‘engage’ in a meaningful way, and given the extreme level of provocation employed, for keeping his cool in response to what was very much a biting-the-hand-that-fed-you situation.
Eric,
from all this, it seems that it would be good to have many more ground stations in Antarctica. Any chance that this might happen?
[Response: Well, since O’Donnell have shown it makes no difference or not what is happening on the ground at Byrd, I would say, no, no need!
Seriously though, the problem is maintenance. I think more grounds stations would be fantastic, and in fact I’m working on a project figuring out where they should go, as is a group at Madison (I think). The idea here is if we can demonstrate (or not) that a new station in a particular location would contribution to *weather* forecasting (for Antarctic logistics – airplane flights), then it might happen. I won’t hold my breath however!
What actually would be more useful for climate, is a lot of borehole temperature records. There’s a perfect PHD proposal project.–eric]
Indeed. From climategate to this, the dirty tricks campaign has been gaining momentum. Simply spreading doubt is no longer enough for McI, et al. Attacks on the scientific process itself are now in play.
…and these are the same
jokerspeople whining about “civility” and demanding respect. Feh.Eric, thank you for the enormous amount of work you put in to your original study and to reviewing O’Donnell et al’s paper. Your frustration is patent in this post and your comments but I am sure a lot of people recognise your courage and commitment to excellence through all of this. Don’t let the vitriol thrown at you stop you from working in this area further.
Question: At what point do physical and biological changes in the region render this sort of debate moot? Having just finished “Fraser’s Penguins,” I’m struck by what these indicators seem to be telling us not only about the past five centuries or so, but also the seemingly accelerating warmup.
Marco (59),
Very well said.
“Who do you trust” remains the big question for most people trying to make sense out of such debates. Which is a perfect opportunity to pitch my post on that subject again ;-)
http://ourchangingclimate.wordpress.com/2009/02/08/who-to-believe/
(more applicable to the general rather than to the specific case though)
Is there a year-round station operating at Patriot Hills? Not sure if they ares till flying int there…but that is a site that is visited frequently during the summer and it would be relatively easy to stay on top of maintenance. The problem though really obtaining long term records….that is why using the satellite data was such a novel and cool approach…but apparently a bit of a you-know-what to implement.
Anyone got ideas for an AWS that can somehow keep itself above the snow pack? ;)
I don’t know whether to be proud or ashamed at the credibility I’d initially lent the O’Donnell allegations. Just another day in the age of the internet, the meme, the memory hole and their contributions to the ignominious demise of objective truth I guess. Science (not to mention scientists tethered to dogged consciences and quaint notions of sincerity) may profit less here than gluttonous buffoons like the Koch’s. Perhaps natural selection is not intelligent.
Some aspects of what happened seem to be a combination of not having experience in nor understanding the peer review process, pretty thin skins and then the poisonous atmosphere that has developed. I doubt that O’Donnell’s et al paper failing to disprove the Antartic warming helped either.
If you don’t know the process and have little experience with it and you combine that with a thin skin, then the critique process is likely to feel like an attack on you versus a hard examination of the science and a push to improve it.
Throw in that all around you are claiming that the “other side” is evil, lying blah blah blah then you may well rush to judgement and claim it is happening to you too (never mind if it really ever happened at all to the others claiming that it did). then in your put upon state you could easily post on a blog stuff that you never should have said. And in your ire violate promises that you made.
So what was an interesting, promising development where some good science was actually done becomes another casuality of the poison. I have to admit how much of what happened is due to the fact that the study did not disprove, but instead reinforced something that went against the claims of the camp contributed.
Or one could evaluate the technical arguments for yourself to see if people are telling the truth.
Is there no chance to sort matters out with Ryan O’Donnell? What I had seen of him so far, made him come across as a pretty sensible and honest fellow. I cannot help but think that his mind got poisoned in the toxic atmosphere of the auditors.
I was pleasantly surprised when I saw people suggest that Eric Steig and Ryan O’Donnell should work together on a paper, synthesizing their strong points. Finally a breakthrough in the stalemate between climate scientists and knowledgeable skeptics, or so I thought.
Condon is too extreme for open communication, and McIntyre doesn’t want the feud to ever end (it’s his personal Sildenafil citrate), but I hope there is still or will be some form of communication with O’Donnell behind the scenes. It would be a shame for something that looked promising to end like this.
But that’s just my personal opinion, I don’t know all the ins and outs.
“ClimateAudit might be from an entirely different planet.”
It certainly describes a different one.
“What actually would be more useful for climate, is a lot of borehole temperature records.”
Are you suggesting looking at firn over a wide area? That could be incredibly useful, if it’s sensitive enough as a temperature proxy. Is it? Everything I read about firn related to CO2, and I know nothing about its other useful properties and problems.
In response to Steve Bloom (25) and others, I’m not sure why you’re astonished that science reporters can’t always judge between competing scientific arguments. Without some pretty advanced training in the relevant science, the back-and-forth claims and counter–claims can get very technical very fast. To cite just one example:
“Eric recommends that we replace our TTLS results with the ridge regression ones (which required a major rewrite of both the paper and the SI) and then agrees with us that the iRidge results are likely to be better.”
I hope you aren’t seriously expecting that science journalists can generally judge the relative merits of two different regression methods, which are just a small selection (I presume) of the analytical tools used to process data. Remember that almost no science journalist has the luxury of covering just one field–or, more appropriately, one subfield, since “climate science” covers oceanography, atmospheric chemistry, glaciology, atmospheric physics and a whole lot of other areas.
Science journalists have to rely instead on other factors, including the track records, training and reputations of the scientists involved, and with gut feelings about who’s making sense and who isn’t. Those of us who have been doing it for a long time tend to develop a sense of who’s reliable and who isn’t–but it’s not infallible. And it’s never going to be.
Eric,
I’m still reading through this mass of comments, but just a few quick words.
In my own profession (computer systems), I’ve suffered from totally unfounded attacks (and flat out lies) from people in politically (within corporations) well positioned spots to make me out to be the liar, in spite of all of evidence to the contrary, or my long history with my clients of both competence and integrity. My situation is very analogous to yours. I’m in computers, so the arguments often turn very technical and confusing, so even other computer people can get lost (everyone has their area of specialty, no matter how experienced or educated), while the decision makers (corporate presidents and vice presidents) have their eyes completely glaze over in total confusing, and they understand nothing of what is being said. In the end it often boils down to trust, not facts, and that trust is very often misplaced.
In the end, sometimes I’ve won, and sometimes I’ve lost, but even when I’ve lost, eventually, the truth is realized. Years later I get calls from clients saying “so and so was just let go, oh my goodness, you were right, we’re so sorry, what were we thinking?… and can you please come back and fix things?”
In climate issues, whenever I get frustrated with the shear audacity and evil of the denial camp, I have the sadly satisfying thought that, of all things, on this one time will tell. 99% of the denial obfuscation is going to be revealed to be exactly what it is as the planet warms and changes beyond any refutation or doubt. And the people on “that” side are going to have to live with the long history of what they’ve said and done, and, in the end, are going to have to defend themselves when no defense is possible.
30 years from today, you may well find yourself being interviewed by someone writing the next version of Merchants of Doubt, but by then it will be titled What Were We Thinking? — The story of how we trusted the wrong people at the worst possible time in human history. People will look at things and say “wow, science, real science got a whole lot of this right, so what the heck were we doing listening to hordes of weathermen and retired engineers and retired physicists and and retired astronauts and wannabe-Lords and random, enthusiastic bloggers?”
Take heart. Breathe deeply, and let the frustration escape.
Time will tell.
[In the meantime, get used to being a guest contestant the Internet’s 21st century specialty version of “reality television”… i.e. “reality science for mass consumption and entertainment.”]
Maybe this whole episode shows that a reviewer shouldn’t be selected, who has a profound conflict of interest with the material they’ve been asked to revue?
The O’Donnell et al paper, is statistically based, so wouldn’t it have been better for a statistician to be the reviewer.
The fact that Dr Steig’s review comments, appear to exceed the length of the paper, does suggest that it was rather more than a review to pick up grammar, spelling & mathematical errors.
[Response: Eric has answered this point so many times before that he’s probably tired of it, so I’ll jump in. Scientists submit papers dealing with other peoples’ work all the time. It’s as it should be, since science is cumulative. Some of that is critical, but that does not generally create a “conflict of interest” since most scientists respond well to constructive and correct criticism. But nonetheless, it is absolutely standard and correct practice to send a paper commenting on some work to the author of that work, since (in my experience as an editor of JAS) the author of the criticized work is often in the best position to understand what is going on. But a journal article in almost all fields (astrophysics being an exception) gets several reviews — generally three, sometimes more — and that protects against any possible bias by any one reviewer. It is the editor’s job to weigh the merits of the arguments given by the various reviewers. I also think you misunderstand the point of reviews. It is not to find grammar and spelling errors and routine algebra errors. That’s the job of the authors. The point of the review is to explore broader methodological issues. I have gotten reviews as long as the one Eric wrote on some of my papers, just because the reviewer was interested, even though the paper in question did not involve any hot-button climate issues. Eric went to a lot of trouble to help make O’Donnel’s paper into a credible scientific contribution but as you can see, no good deed goes unpunished. –raypierre]
#63 P. Lewis:
“Are these emails on the UWS server?
“Calling all hackers!”
Yes, I’m kidding, honestly.”
There’s an entity over at CA called ‘RationalDoubt’ who is calling for FOI requests to get at the UWS server over this, and he’s not kidding. What a bunch of angry little teapots they are over there.
If you want to understand how these folks so badly misunderstood your review, and how they come to be convinced of your “duplicity”, consider their mindset.
http://climateaudit.org/2009/06/07/banned-at-sudbury-airport/
When I arrived in Sudbury Airport on Friday night, I logged onto the airport internet terminal (conveniently free) and tried to access Climate Audit. Access was blocked. I was –
Banned in Sudbury.
To verify that Climate Audit was specifically blocked, as opposed to blogs, I visited realclimate, which loaded without event.
It turned out that someone had downloaded parental control software onto the local network. And it was blocking anything that contained a certain preset list of words (some of which were quite innocuous). But the software also works from a centralized list of banned websites, and McIntyre was convinced there was a real possibility that ClimateAudit was on that list. Lots of fingerpointing about who might have been behind this ensued.
I downloaded the same software and demonstrated the blocking via an innocuous use of the “wrong” word on the actual “Sudbury” post. McIntyre still claimed there was evidence that he might have been blacklisted, although most of the regulars were jumping off ship at that point. (Yeah, I might write this whole incident up).
Anyway, the very first comment was:
Ryan O
Posted Jun 7, 2009 at 8:51 PM
1984 was such a good book. Good thing it was fiction.
Getting back to the current case …
As far as I can see, the first O’Donnell post hasn’t changed much, except that the words “duplicity” and “dishonesty” have been snipped. But it’s still all high dudgeon and “disgust” at Eric’s criticism of a change he supposedly forced on them. What retraction?
I see now Delingpole has weighed in.
http://blogs.telegraph.co.uk/news/jamesdelingpole/100075232/realclimategate-hits-the-final-nail-in-the-coffin-of-peer-review/
Just when you thought they couldn’t go any lower.
With you approach, is it true that if the temperatures on the peninsula are decreased, then the temperatures in West Antarctica will increase? This seems counter intuitive. Can you help explain this relationship?
Adam Gallon, That is absolute horse puckey. Where did you get the idea that peer review was supposed to be easy. The whole point is to subject one’s thesis to critical review prior to publication–and who better to do that than one’s peers–meaning rivals.
ODonnell’s behavior in this incident has been reprehensible–enough so to make me question his stability, and certainly his judgment.
In shor: Science works. Let it.
So why was eric not a reviewer of the last draft? I may have missed it amongst the car crash, but I really would like to know.
Raypierre,
Or, in today’s environment, no good science goes unpunished.
[Response: The length of the reviews did not exceed the length of the paper. The length of O’Donnell’s responses did. Just thought I’d set that record straight.]
Here’s an interesting paper on confirmation bias in the peer review system that’s worth reading:
http://pages.stern.nyu.edu/~wstarbuc/Writing/Prejud.htm
Unfortunately while peer review is not without its flaws, it’s the best system we’ve got.
As one of those lay persons having trouble following the techie details, but really doing their best to make sense of the on going disputes surrounding climate science in general, I found this post very helpful. Sadly, however, the final few paragraphs did seem to descend into the sort of name calling and making of accusations (without reference or back up details) that Mr Steig claims that the other side of the dispute is prone to indulge in. Maybe they are, but responding in kind really does not help. Can we please try to stick to the science and the data here guys?
Just because they’re paranoid Deep, doesn’t mean everyone’s not out to get them.
84, Deep Climate,
Oh my gosh. I’m still in shock. Delingpole basically ended by saying that… wait for it… WUWT shows the way forward after the death of peer review.
Shoot me now. I don’t want to live on this particular planet anymore.
Professor Steig – Just adding to the chorus of sympathy about your mistreatment. This is really an abuse of your collegiality and the peer review process. I work in a very different field but have considerable experience as a reviewer. Perhaps the Journal of Climate doesn’t have explicit policies about public airing of reviews or attempts to breach reviewer anonymity but so what? With the exception of experiments like the EMBO Journal, reviewer anonymity and review confidentiality are generally assumed norms of scientific practice. There really should be some sanctions for this kind of misbehavior. If something like this occurred with one of the journals for which I serve on the Editorial Board, I’d be in favor of banning the violaters from submitting to that journal.
jason — 10 Feb 2011 @ 12:51 PM
I expect there isn’t anything mysterious about that. It’s quite likely that the editor considered that the last draft had addressed sufficient of the reviewers concerns that the paper didn’t need to be sent back to any or all of the reviewers. Four submission drafts is a lot of drafts! (Very,very occasionally I’ve had to submit 3 drafts before acceptance, but the vast majority of accepted papers go through the submission-review-resubmission-acceptance route, at least in my field).
Ultimately it’s up to the editor…
[Response: I agree. I in no way am suggesting the editor behaved unprofessionally. Editors are busy, and they have to balance things as best they can.–eric]
So Erik, can you honestly, and I can’t emphasize “honestly” enough, say that you were not stonewalling O’Donnell’s paper?
[Response: First off, spell my name right. Second: yes. I spent many many many hours on those reviews, specifically to be as thorough and fair as possible. I was reminded at least once by the editor that if I thought it should be rejected, I should say so. I did not say that, ever. What I said in my first comment to the editor was that although I thought the first draft read like a series of poorly thought out blog posts (which is of course what it was), the authors should “not be discouraged from submitting a revised version.”
Do you honestly — and I cannot emphasize “honestly” enough — believe O’Donnell is an ethical player here, given his fabricated rant against me, based on (what he now admits to be) hearsay evidence? Believe it not, I still do (mostly because he said he would apologize to me publicly, though I don’t think he really has, but there is always hope) because one of my weak points is believing the best of people in spite of compelling evidence to the contrary.–eric]
Adam Gallon – it is easy to fool yourself and since science is about getting the best model of reality, it is often the best to suggest the person who will hate the paper most as a reviewer when submitting. It slows publication but its better to get hard critical review early rather than embarrassing errors being pointed out later.
And when a paper is a critique or extension of another work, it should absolutely go to the original author.
It seems that providing fodder for the denizens of Climate Audit, WUWT, and beyond has become more important for some than contributing anything to science. These people claim to want a ‘debate’ whilst doing their damnedest to poison any debate. Shame on them.
“The length of the reviews did not exceed the length of the paper. The length of O’Donnell’s responses did. Just thought I’d set that record straight”
Last I looked 24 (review) was greater than 8 (paper).
Just setting the record straight.
“So why was Eric not reviewer of the last draft?”
Because he was replaced by the editor. (Just setting the record straight)
Reiterating a point made by JBowers #64 earlier, and I think, to a certain extent by Ged also. This is not entertaining, I don’t know either of personally and you may both be difficult characters to get along with, I don’t know ;)
But is there really no possibility of a behind the scenes reconciliation? The earlier exchanges here appeared to be quite amicable, but have degenerated somewhat into a …he did, he said, he responded, then this happened …. not pretty to watch.
Is it possible to get back to the science? (Ged’s point) To the extent that there were flaws in both papers, I don’t believe either were disingenuous.
I imagine a joint ‘definitive’ paper would be welcomed by all here (and elsewhere). This may be just naivety on my part, but it would prove better men for both of you.
Dr. Steig,
In my experience, both you and Dr. O’Donnell bear responsibility for where this matter has gotten to. If I assessed responsibility, I’d say Dr. O’Donnell is 60-70% at fault, and you are 40-30% at fault. But the percentages are immaterial.
What’s true is this (as far as the rules of peer-review go):
1) you never ask someone if they reviewed your own work
2) you never ask someone else to ask that someone if they reviewed your own work
3) you never tell anyone if you reviewed their work
4) reviews must be at least single blind; at best, they are double-blind (neither author nor reviewer knows the other’s name or affiliation)
5) authors get to see the reviews
6) authors get to respond to the reviews
7) reviewers get to respond to the response
8) editors decide how many times around the loop you go
9) reviews are not published; only the final, accepted work gets published
The only truth lies in what gets published in a peer-reviewed journal; not what gets tossed around like a rag doll on a blog.
[Response: You know what? I appreciate your points 1-9 — they are correct, except that my telling O’Donnell privately, in full confidence, the answer to something that he had made clear he already pretty much knew was completely and utterly a reasonable thing to do — but I do not appreciate your having the audacity to decide what percentage of ‘blame’ I get for someone making up stuff about me and broadcasting it over the internet.–eric ]