As many of you will be aware, a large number of emails from the Climatic Research Unit (CRU) at the University of East Anglia webmail server were hacked recently (Despite some confusion generated by Anthony Watts, this has absolutely nothing to do with the Hadley Centre which is a completely separate institution). As people are also no doubt aware the breaking into of computers and releasing private information is illegal, and regardless of how they were obtained, posting private correspondence without permission is unethical. We therefore aren’t going to post any of the emails here. We were made aware of the existence of this archive last Tuesday morning when the hackers attempted to upload it to RealClimate, and we notified CRU of their possible security breach later that day.
Nonetheless, these emails (a presumably careful selection of (possibly edited?) correspondence dating back to 1996 and as recently as Nov 12) are being widely circulated, and therefore require some comment. Some of them involve people here (and the archive includes the first RealClimate email we ever sent out to colleagues) and include discussions we’ve had with the CRU folk on topics related to the surface temperature record and some paleo-related issues, mainly to ensure that posting were accurate.
Since emails are normally intended to be private, people writing them are, shall we say, somewhat freer in expressing themselves than they would in a public statement. For instance, we are sure it comes as no shock to know that many scientists do not hold Steve McIntyre in high regard. Nor that a large group of them thought that the Soon and Baliunas (2003), Douglass et al (2008) or McClean et al (2009) papers were not very good (to say the least) and should not have been published. These sentiments have been made abundantly clear in the literature (though possibly less bluntly).
More interesting is what is not contained in the emails. There is no evidence of any worldwide conspiracy, no mention of George Soros nefariously funding climate research, no grand plan to ‘get rid of the MWP’, no admission that global warming is a hoax, no evidence of the falsifying of data, and no ‘marching orders’ from our socialist/communist/vegetarian overlords. The truly paranoid will put this down to the hackers also being in on the plot though.
Instead, there is a peek into how scientists actually interact and the conflicts show that the community is a far cry from the monolith that is sometimes imagined. People working constructively to improve joint publications; scientists who are friendly and agree on many of the big picture issues, disagreeing at times about details and engaging in ‘robust’ discussions; Scientists expressing frustration at the misrepresentation of their work in politicized arenas and complaining when media reports get it wrong; Scientists resenting the time they have to take out of their research to deal with over-hyped nonsense. None of this should be shocking.
It’s obvious that the noise-generating components of the blogosphere will generate a lot of noise about this. but it’s important to remember that science doesn’t work because people are polite at all times. Gravity isn’t a useful theory because Newton was a nice person. QED isn’t powerful because Feynman was respectful of other people around him. Science works because different groups go about trying to find the best approximations of the truth, and are generally very competitive about that. That the same scientists can still all agree on the wording of an IPCC chapter for instance is thus even more remarkable.
No doubt, instances of cherry-picked and poorly-worded “gotcha” phrases will be pulled out of context. One example is worth mentioning quickly. Phil Jones in discussing the presentation of temperature reconstructions stated that “I’ve just completed Mike’s Nature trick of adding in the real temps to each series for the last 20 years (ie from 1981 onwards) and from 1961 for Keith’s to hide the decline.” The paper in question is the Mann, Bradley and Hughes (1998) Nature paper on the original multiproxy temperature reconstruction, and the ‘trick’ is just to plot the instrumental records along with reconstruction so that the context of the recent warming is clear. Scientists often use the term “trick” to refer to a “a good way to deal with a problem”, rather than something that is “secret”, and so there is nothing problematic in this at all. As for the ‘decline’, it is well known that Keith Briffa’s maximum latewood tree ring density proxy diverges from the temperature records after 1960 (this is more commonly known as the “divergence problem”–see e.g. the recent discussion in this paper) and has been discussed in the literature since Briffa et al in Nature in 1998 (Nature, 391, 678-682). Those authors have always recommend not using the post 1960 part of their reconstruction, and so while ‘hiding’ is probably a poor choice of words (since it is ‘hidden’ in plain sight), not using the data in the plot is completely appropriate, as is further research to understand why this happens.
The timing of this particular episode is probably not coincidental. But if cherry-picked out-of-context phrases from stolen personal emails is the only response to the weight of the scientific evidence for the human influence on climate change, then there probably isn’t much to it.
There are of course lessons to be learned. Clearly no-one would have gone to this trouble if the academic object of study was the mating habits of European butterflies. That community’s internal discussions are probably safe from the public eye. But it is important to remember that emails do seem to exist forever, and that there is always a chance that they will be inadvertently released. Most people do not act as if this is true, but they probably should.
It is tempting to point fingers and declare that people should not have been so open with their thoughts, but who amongst us would really be happy to have all of their email made public?
Let he who is without PIN cast the the first stone.
Update: The official UEA statement is as follows:
“We are aware that information from a server used for research information
in one area of the university has been made available on public websites,”
the spokesman stated.“Because of the volume of this information we cannot currently confirm
that all of this material is genuine.”“This information has been obtained and published without our permission
and we took immediate action to remove the server in question from
operation.”“We are undertaking a thorough internal investigation and we have involved
the police in this enquiry.”
Update II: Please comment on the next thread.
Prominent among the people crowing most loudly about this supposed scandal is Ian Wishart, author of the scientifically illiterate denialist screed Air Con and publisher of Investigate Magazine. The magazine has a proud track record of uncovering other scientific conspiracies, suggesting for instance that NASA has covered up evidence of Egyptian-style pyramids and rock carvings on Mars – see http://www.investigatemagazine.com/pdf's/julsec33.pdf.
Cherry-picking and distortion doesn’t begin to cover it.
“Funny how Watts feels free to publish others’ emails.”
Anthony does not receive public funds to do his work.
[Response: Ethics doesn’t stop at the exit of the town hall. – gavin]
Why was this data released in such a way? As it is we’re never going to know if the e-mails are genuine or not, simply because someone could, if they wanted to, modify the original files so that they look different from the hacked files. If, however, instead of releasing the files the hacker had got the authorities to seize the original files, there would not have been any way to modify them. Hence there would be no doubt about their authenticity.
FWIW, my opinion is that this is a last ditch attempt to muddy the waters before Copenhagen.
“John Bunt says:
20 November 2009 at 2:46 PM
What has happened?? In the 1970’s, when the New York Times published the “Top Sectret” Pentagon papers they were HEROES! And Nixon was thrown out of office for, among other lesser things, trying to find out that Daniel Ellsburg was the leaker. Now, some are saying it was criminal to print this information. How convenient. I guess that it depends upon the issue, and which side you are on.”
United States != United Kingdom
The two “cases” are completely different. These CRU emails aren’t threatening national security, but they were illegally obtained. Not sure if the NYT receiving the papers was deemed illegal, but there sure was a fight over the right to publish them.
Article: Scientists often use the term “trick” to refer to a “a good way to deal with a problem”, rather than something that is “secret”, and so there is nothing problematic in this at all.
Response: lol….
Trick:
“a cunning or deceitful action or device; “he played a trick on me”; “he pulled a fast one and got away with it”
“Something designed to fool or swindle; ”
“flim-flam: deceive somebody; “We tricked the teacher into thinking that class would be cancelled next week””
[Response: Wrong. Wrong and wrong. – gavin]
This information will undoubtedly be the opportunity for skeptics to vindicate themselves. ‘Science’ is about ‘Truth’ and in many instances these emails depict their respective authors as untruthful. Unfortunately, we reap what we sow.
Last week I submitted an admittedly rather facetious post pointing out that the public are on the side of the deniers blogs (since confirmed by opinion polls on both sides of the Atlantic) and chiding The Team for publishing a speculative post about a small part of Antarctica rather than tackling the issues head on. The post was rejected. I suspected it was because this blog is tightly controlled; the hacked emails appear to confirm this.
To see how this is playing out on 80 top climate blogs go to:
http://www.climatedata.info/Opinions/Opinions/search.html
Instead of being contrite at having been caught in disgraceful behavior you are brazenly and aggressively attempting to defend these e-mails and what they have exposed. Mann is calling for the criminal prosecution of whoever let the cat out of the bag. I guess “the team” is re-grouping to defend itself at all costs. Have you no decency? I know the answer to that one!
Don’t you know that once a con is exposed its all over?
pdboddy:
It does reflect badly on those who would breach the privacy of others to dig for dirt. Does it reflect badly of those who express themselves in confidence to colleagues and friends? No.
/ Per
The problem with the contents of these emails is that they seem to confirm many peoples’ suspicions that prominent climate scientists are:
a)more interested in manipulating data to make it substantiate a predetermined narrative (“It would be good to remove at least part of the 1940s blip…”) and
b) often merely tweaking the variables that are actually known and understood to make data fit what is supposed to “look right” given those knowledge constraints and their preconceptions/preferences (“When you look at other blips, the land blips are 1.5 to 2 times (roughly) the ocean blips — higher sensitivity plus thermal inertia effects. My 0.15 adjustment leaves things consistent with this…”).
c) Suppressing contradictory views internally (let’s delete these emails!) and externally by manipulating and controlling the pier review process, a problem reinforced by the buddy-buddy fraternity that exist across organizations (“I’ve just completed Mike’s Nature trick…”).
In short, there is something here to lend credibility to pretty much every suspicion that skeptics have. This is very damaging, and I think a lot of people here are in denial about how meaningful this will be in most peoples’ minds.
[Response: Fair point, but each of the points you raise are taken out of context and do not imply what you take them to mean. – gavin]
The only way to gain credibibity is to:
a) allow full access to all the source data – which you have so fat refused, and
b) cooperate with the requests made under the Freedom of Information Act – which the hacked emails appear to show you are going to great lengths to frustrate
David Evershed BSc(Eng) PhD
…and I say that AFTER reading up on the Freedom of Information Act (current revision). Requesting information which a decision was based upon through due process is something completely different from unauthorized entry and theft of personal, professional correspondence.
/ Per
Frank(~52, 20 November 2009 @ 2:13 PM):
I am curious what monumental truth the hackers provided you.
Steve
The cards played by ‘deniers’ isn’t about it warming or not. It’s about to what “danger” we feel from (potential) warming. We do not share the fear of world warming that our funded researchers do. Why? In some ways because we are not searching or seeing evidence for it. There is probably a psychological term for that. But if thing outside of your body are out of your control, then ‘seeing is believing’. If you don’t see, you don’t believe. If you see, then you believe. Danger and global warming is about dealing with a threat, either real, not real, exaggerated, or whatever the case. Priorities are set from danger. And warming is not dangerous enough to warrant action. Action is warranted from weather, not climate. This is what true skeptics feel on a primitive level. And hiding data and restricting access is a good way to start a rage against the machine of global warming and all its robots.
#73–Nixon was thrown out of office for ordering burglary and then lying about it. Remember “It’s not illegal when the President does it?”
So I imagine John Bunt hopes that those who cracked this server be brought to trial, as Daniel Ellsberg was, and that the owners of WUWT, CA, etc be hammered with the full force of the legal system just as the NY Times was, right?
I do …
Even better would be to see the crackers lose their trial, unlike Ellsberg …
Somebody naively wondered why there are so few comments on this post.
IMO the answer is that RealClimate is effectively content-censored to a large degree for conformance with the prevailing ideas here. Uninformed dissenters are sometimes let in so the comment crew can bash them around, but reasoned dissenters are usually banned outright.
Many don’t bother trying to post here for that very reason.
Gavin in the interest of transparency would you at least roughly estimate how many of the comments have been moderated out for this post? I would guess 95% have been zapped.
Scientists often use the term “trick” to refer to a “a good way to deal with a problem”
Could you give a few examples? I searched right here at Realclimate for uses of that term they tend to relate to “trickery”, not good science.
[Response: My time to spend on this is limited so moderation takes time when we are being swamped with comments (1 down, 100 to go). – gavin]
“However, the emails do suggest that certain people try to use this site in such a way: filtering out the naysayers.”
How does that square with the fact that lots of naysayers post comments here?
Having read through all the emails and several of the attachments my take on it is this.
There is very little that has to do with climate science per se. The biggest issues are
the behaviors of the institutions of climate science. In particular the behavior of
CRU with regards to FOI.
IF these mails are accurate ( the headers appear to be real based on unix time stamping
protocals) then it would appear that there maybe some issues surrounding the destruction
of emails ( government property ).
In an email with the subject FOI and IPCC one individual using government property
instructs other individuals to delete emails ( government property)
It’s always about the cover up.
The correct procedure is to let the FOI process work.
Watts doesn’t hide anything, neither does McEntyre. Only people with something to hide, hide things. It’s really that simple.
[Response: I’ll take you seriously when either of them opens up their Inbox for everyone to look at. – gavin]
I have seen several comments here referring to WUWT as a source of this information. I haven’t read all of the comments, nor have I looked at the CU site. BUT: I did read the original posting that their had been a leak. Anthony NEVER suggested anything other than the FACT that there had been a leak reported, and that there was now a file available at a ‘russian web-site’, he did not name the site. Further, he stated explicitly that he advised against downloading any file from an unknown server, and that the veracity of the contents could almost certainly never be verified, except it be from those from whom the files had been stolen.
So, to those who accuse the WUWT website of complicity or being a source for the file itself, know this; you are spreading disinformation, and engaging in what amounts to ad hominem attack on Anthony Watts. His web site and his personal opinions regarding AGW are far far more balanced and scientific than most of what’s allowed on this site.
Finally; some of the comments on this board are very damning; several of you have here admitted that these are in fact your correspondence. Some of the emails regarding FOI are in fact conspiracy to commit felonious acts. If those were proven to be yours, your only recourse would be ‘poisoned fruit’, which likely wouldn’t work, as this act was (ostensibly) not engaged by law enforcement. You could be prosecuted.
Doubt me?
I have a serious challenge to RealClimate. First, I will express my discomfort in how this information became visible. Second, I have a feeling that any independent person can not get enough information. Must one allways be worried what an independent person might CONCLUDE? The serious challenge is: Now that the “damage” is done. Could RC tell us if there is “made up” mails /documents in the original FOIA2009.zip file? Would you assist the curious seeker of information?
I do not mean to be critical, but someone must say it:
In my business, even in our private conversations, we strive to maintain a decorum that fits with the need for our exercise of impartial and unbiased judgment. In science, this is critical of course.
If these emails are real, some of them should give the CRU scientists an occasion for healthy self reflection. At times, what comes across in the materials widely quoted is a devotion to a particular viewpoint that seems closed of from rigorous scientific scrutiny. The snippets reportedly suggesting an effort to squelch the publication of opposing viewpoints are especially antithetical to good science.
We are right. WE have nothing to be afraid of. The marketplace of ideas will out the most persuasive methodologies and analysis and to suppress contrary views only smacks of a bias that will destroy what we are working for.
Strive for THE answer not YOUR answer and the interest of science will always be served.
It is interesting that those of you are upset about things being taken out of context are the same people who take Levitt and Dubner’s Superfreakanomics out of context. Of course, that won’t fulfill your agenda, so you have to throw sand on this fire quickly.
I notice Joe Romm did the same thing. Acknowledge it and then ignore anybody who cries foul.
Neat trick for sure, but not nearly as transparent as people want it to be.
Is there somewhere I could find a graph or model that depicts the ocean levels over the past 2000 years, if such a model or graph exists? thanks.
[Response: http://en.wikipedia.org/wiki/Sea_level_rise – gavin]
Thank you for this posting. I hope that, in the coming days, you will follow up with a comprehensive discussion, point by point, and with an overall view that establishes the context, and in terms that laymen can follow. It will be time-consuming and laborious, but given the tenor of the accusations and the gravity of the underlying subject, a necessary project.
At first glance, I don’t see anything particularly discrediting to the climate change hypothesis emerging from the purloined e-mails. Yet, as I’ve said, I think it’ll be important to untangle the ball of string and lay it out straight.
If these emails turn out to be genuine then at the very least the world of climate change research has changed forever. For one any research generated, reviewed or relying on the input of these scientists will be called into question. This is not to say it will be invalidated, rather that it will need to be reexamined. This examination will have to be completely open and transparent with all data and methodologies open to outside scrutiny by anyone who wishes to make the effort. The whole concept of peer reviewed science will have to be abandoned. This is a step forwards to greater transparency. “Trust us we’re scientists” will no longer wash.
[Response: Strawman. – gavin]
The contents of these e-mails is truly shocking–climate scientists are (gasp!) human beings! Oh, the horror… the horror…
On a more serious note, there is nothing I’ve seen about the content of these msgs that comes anywhere close to being a scandal or improper in any meaningful sense of those words.
I’ve worked in university, computer business, and publishing environments, and I can say without hesitation that the contents of these e-mails are extremely tame in comparison to what I’ve read and written more times than I can count.
But of course, this will turn into A Big Thing in the echosphere, and it will get endlessly rehashed. Just look at all the absurd claims on some sites accusing James Hansen of single-handedly cooking data, all based on not even stolen e-mail. We should expect to see this event and the fantasies it will help produce become a permanent fixture of the discussions online, as well as on certain broadcast outlets. And the whole time, CO2 levels will rise, temps will rise, permafrost will melt, methane hydrates will be released, polar and glacial ice will continue to shrink, and the ocean will become more acidic.
Honestly, there are times when I wonder how human beings were smart enough to get as far as we have.
“[Response: The paper and journal in question were indeed a scandal. But the scandal was that it was ever published. Six editors of the journal resigned in protest at the publication, not because of pressure. – gavin]”
That’s fortunate. I note that in mail #1051190249
“Note that I am copying this view only to Mike Hulme and Phil Jones. Mike’s idea to get editorial board members to resign will probably not work”
It wasn’t necessary, but there had clearly been discussion about leaning on them to resign…
Do you understand that when there is an argument over a scientific matter, both sides think they are right and think that the other sides science is (relatively) poor ? Trying to squeeze one side out of publication so that you can then crow about their papers not being “published” is abhorrent and an embarassing shame on those undertaking that approach. The correct response is to respond to the papers, and NOT to try to stop them publishing (or commenting). Apparently improving the signal to noise ratio is only important when it’s your signal ?
All the talk about the science being settled, when there are discussions amongst yourselves about not being able to explain the current lack of warming is also shameful – why is the discussion taking place on private emails instead of in established climate journals ? Is it because it may damage the signal to noise ratio, and confuse the under class ?
[Response: You completely mischaracterise what happened. It was the unjustified publication of SB03 that was the corruption of the peer review process, pushing back against that corruption was what motivated the resignations. – gavin]
“I’ve been such a supporter ….”
Citation needed.
You know, I’ve always wondered why we scientists never seem to use anything like pgp in email communication with each other. Maybe its time to start?
I’ll be a lot more interested in this discussion after Fox News, or the WSJ editorial page, or the Heartland institute, or the US Chamber of Commerce releases a random 60 MB chunk of email.
Group: “Scientists often use the term “trick” to refer to a “a good way to deal with a problem”, rather than something that is “secret”, and so there is nothing problematic in this at all.”
Well, since this happens “often”, it would be good to see a couple of examples of the word’s usage from other fields to understand why it is not problematic. Thank you.
[Response: Sure. It’s mostly used in mathematics, for instance in decomposing partial fractions, or deciding whether a number is divisible by 9 etc.etc.etc. – gavin]
Thanks to the whistle blowers. We have legislation in the UK to protect them. Mind you the Russians are sceptical. Maybe they did it.
[Response: This is a moderated site, and always has been. We do screen out a lot of the random squawk of the blogosphere and the baseless accusations of malfeasance that are commonplace on open forums. We do that unapologetically in order to maximise the signal-to-noise ratio in threads…]
Carrying the signal processing analogy further, improper filtering can also add noise to a signal.
Whatever filtering strategy you’re using, in a few of the emails it appears that some people thought they could take advantage of it and game you.
I noticed you have yet to respond to this comment
“Asking people to delete emails, in an email, with the SUBJECT: line containing FOI is indeed a serious problem.”
What is the justification for this email?
While I’m what you would consider a skeptic of global warming, I can appreciate the concept that internal commentary among researchers is both private and potentially misleading. That said, I think the Hadley CRU is obligated to release all the data sets and the rationale for any adjustments made to the research community for full peer review and debate about their validity. That alone would allay any doubts about the integrity of these data sets and, indeed, that of the researchers involved.
To me, the most damning comment I’ve read is Kevin Trenberth saying that it was a “travesty” that they “can’t account for the lack of warming at the moment”. He makes the candid admission that his observation model is “inadequate” — because the CERES data on 2008 shows that more warming should’ve happened, but obviously didn’t.
While I’m open to the possibility that there could be a defensible explanation for these comments, it sounds an awful lot like a presumed conclusion in search of supporting evidence.
[Response: Trenberth is talking about our inability to be able to measure the net radiation balance at the top of the atmosphere to the requisite precision to be able to say on short time scales what the energy budget is doing. The observations are inadequate for that – not sure who is saying otherwise. – gavin]
This too shall pass. In this instance of hacking, ideologically driven, delusional ends are being used to justify unethical means. Many in the global environ. management and governance community globally are trained and experienced in analyzing science outputs regardless of the particular field (it comes down to a matter of sufficient time which is increasingly scarce, but that’s our problem and responsibility); many in this community also know how science works and that scientists are actually humans with all attendant emotions and proclivities. This episode is worrisome no doubt to those in the trenches dealing with the science and the denialists; and it will no doubt create another time sink. But it also may motivate governments to deal a little more forthrightly with fringe denialism which is ideologically, not scientifically, driven. We know what’s coming in terms of the risk spectrum of global climate/ocean change and we’re going to deal with it as best we can as nations and as a global community. Our respect and appreciation for field of climatology and its sister disciplines is strong.
Gavin, today’s pre-emptive strike and your comment responses might help with RealClimate’s credibility – at least among those who hope to continue their dismissive attitude towards the rest of us.
But to me it reads like a CYA piece, misdirecting blame.
Don’t you see the real lesson here? It’s past time for transparent debate about man-made global warm- (oops! sorry… “climate change”). Instead of being defensive, why not use this incident to call for more openness and debate among climate scientists?
My other suggestion for long-term credibility is to encourage your colleagues and those on the left to stop with the insults. I am not a “denier,” or “sceptic,” or “climate crank.” I respect the scientific method and I believe that human understanding of our global climate is in its infancy. I’m guessing that the insular world of pro-AGW scientists has left you unable to see how few people trust your early conclusions.
Meanwhile, the rest of aren’t afraid of who might see what we write in our business email.
Please, push for transparency, in both science and funding.
All the emails I have read deal with research that has been publicly funded and therefore would fall under FOIA and IMO, can not be considered “private” to protect or enable the authors of the emails a guise of righteousness to be protected.
James Sexton:
BPL:
Dream on.
#83: In other words, you just rephrased what scott just said and then agreed with him. Lacking an explanation for current warming is exactly what he was saying and said, and you agree. This doesn’t necessarily mean that long-term projections are wrong, but obviously credibility is reduced as model error gets larger and larger.
BTW, on my previous comment: two recent papers have been transparent to the level required by science, although they still will not give all of the data used to go along with the code. This is hardly an excuse for over a decade of not giving code OR data, and is completely irrelevant when talking about FOIA requests that came *before* those papers even existed.
Tim:
BPL:
No, federal employees still have a right to privacy.
Not climate-related, but I heard that someone I know was told by a supervisor not to supply any research findings to the supervisor and has since been ignored by the supervisor. This all seems to be part of an effort to maximize plausible deniability prior to expected scrutiny over the next while. I only know of this for one researcher under this one supervisor, so that’s not an expression that this kind of thing is common. This kind of stuff does happen, however, and it’s a damn shame no matter how rarely it occurs.
Let he who is without PIN cast the the first stone.
Oh, cle-ver! But you’re not looking so Jesus-like these days.
[Response: Maybe I should grow the beard out? – gavin]
To Scott Mandia: Watts did not “warn” you that the email exchange was private. He will simply promising that he would keep it private himself. Your quote of
“Fair warning – this communication is private” is obviously deconstructed to that the “Fair warning” part refers to the preceding part of the email (probably a reply to a point of yours), and then pledges privacy on his part. You have dishonestly posted a willful misrepresentation.
David (#76),
Here is a link to the paper in question: http://www.int-res.com/articles/cr2003/23/c023p089.pdf
Read through the paper and look at the authors’ methodology. If two or three “showstopper” problems don’t jump right out at you, then you are in over your head here.
SDann:
BPL: 40 lashes for all of them, I say! How dare scientists have opinions!
For scientists, your research is rather poor.
It´s NOT 60mb of emails.
It´s also pdf, docs, jpegs, and
interestingly spreadsheets.
That´s a nice retirement plan.
http://spreadsheets.google.com/ccc?key=0Ah4XLQCleuUYdFIxMnhMNnlXb2JQcDZUendjUXpWWUE&hl=en
[Response: Scientist’s retirement plans are not related to the grants they get to cover researchers they have working for them. I wish. – gavin]