As many of you will be aware, a large number of emails from the Climatic Research Unit (CRU) at the University of East Anglia webmail server were hacked recently (Despite some confusion generated by Anthony Watts, this has absolutely nothing to do with the Hadley Centre which is a completely separate institution). As people are also no doubt aware the breaking into of computers and releasing private information is illegal, and regardless of how they were obtained, posting private correspondence without permission is unethical. We therefore aren’t going to post any of the emails here. We were made aware of the existence of this archive last Tuesday morning when the hackers attempted to upload it to RealClimate, and we notified CRU of their possible security breach later that day.
Nonetheless, these emails (a presumably careful selection of (possibly edited?) correspondence dating back to 1996 and as recently as Nov 12) are being widely circulated, and therefore require some comment. Some of them involve people here (and the archive includes the first RealClimate email we ever sent out to colleagues) and include discussions we’ve had with the CRU folk on topics related to the surface temperature record and some paleo-related issues, mainly to ensure that posting were accurate.
Since emails are normally intended to be private, people writing them are, shall we say, somewhat freer in expressing themselves than they would in a public statement. For instance, we are sure it comes as no shock to know that many scientists do not hold Steve McIntyre in high regard. Nor that a large group of them thought that the Soon and Baliunas (2003), Douglass et al (2008) or McClean et al (2009) papers were not very good (to say the least) and should not have been published. These sentiments have been made abundantly clear in the literature (though possibly less bluntly).
More interesting is what is not contained in the emails. There is no evidence of any worldwide conspiracy, no mention of George Soros nefariously funding climate research, no grand plan to ‘get rid of the MWP’, no admission that global warming is a hoax, no evidence of the falsifying of data, and no ‘marching orders’ from our socialist/communist/vegetarian overlords. The truly paranoid will put this down to the hackers also being in on the plot though.
Instead, there is a peek into how scientists actually interact and the conflicts show that the community is a far cry from the monolith that is sometimes imagined. People working constructively to improve joint publications; scientists who are friendly and agree on many of the big picture issues, disagreeing at times about details and engaging in ‘robust’ discussions; Scientists expressing frustration at the misrepresentation of their work in politicized arenas and complaining when media reports get it wrong; Scientists resenting the time they have to take out of their research to deal with over-hyped nonsense. None of this should be shocking.
It’s obvious that the noise-generating components of the blogosphere will generate a lot of noise about this. but it’s important to remember that science doesn’t work because people are polite at all times. Gravity isn’t a useful theory because Newton was a nice person. QED isn’t powerful because Feynman was respectful of other people around him. Science works because different groups go about trying to find the best approximations of the truth, and are generally very competitive about that. That the same scientists can still all agree on the wording of an IPCC chapter for instance is thus even more remarkable.
No doubt, instances of cherry-picked and poorly-worded “gotcha” phrases will be pulled out of context. One example is worth mentioning quickly. Phil Jones in discussing the presentation of temperature reconstructions stated that “I’ve just completed Mike’s Nature trick of adding in the real temps to each series for the last 20 years (ie from 1981 onwards) and from 1961 for Keith’s to hide the decline.” The paper in question is the Mann, Bradley and Hughes (1998) Nature paper on the original multiproxy temperature reconstruction, and the ‘trick’ is just to plot the instrumental records along with reconstruction so that the context of the recent warming is clear. Scientists often use the term “trick” to refer to a “a good way to deal with a problem”, rather than something that is “secret”, and so there is nothing problematic in this at all. As for the ‘decline’, it is well known that Keith Briffa’s maximum latewood tree ring density proxy diverges from the temperature records after 1960 (this is more commonly known as the “divergence problem”–see e.g. the recent discussion in this paper) and has been discussed in the literature since Briffa et al in Nature in 1998 (Nature, 391, 678-682). Those authors have always recommend not using the post 1960 part of their reconstruction, and so while ‘hiding’ is probably a poor choice of words (since it is ‘hidden’ in plain sight), not using the data in the plot is completely appropriate, as is further research to understand why this happens.
The timing of this particular episode is probably not coincidental. But if cherry-picked out-of-context phrases from stolen personal emails is the only response to the weight of the scientific evidence for the human influence on climate change, then there probably isn’t much to it.
There are of course lessons to be learned. Clearly no-one would have gone to this trouble if the academic object of study was the mating habits of European butterflies. That community’s internal discussions are probably safe from the public eye. But it is important to remember that emails do seem to exist forever, and that there is always a chance that they will be inadvertently released. Most people do not act as if this is true, but they probably should.
It is tempting to point fingers and declare that people should not have been so open with their thoughts, but who amongst us would really be happy to have all of their email made public?
Let he who is without PIN cast the the first stone.
Update: The official UEA statement is as follows:
“We are aware that information from a server used for research information
in one area of the university has been made available on public websites,”
the spokesman stated.“Because of the volume of this information we cannot currently confirm
that all of this material is genuine.”“This information has been obtained and published without our permission
and we took immediate action to remove the server in question from
operation.”“We are undertaking a thorough internal investigation and we have involved
the police in this enquiry.”
Update II: Please comment on the next thread.
So, for example, when the emails are clearly discussing manuscripts sent to various climate scientists in confidence for peer-review, and coordinating responses by email, how does that square away with journal policies?
By the way, I’m an active researcher, and I certainly don’t do this. The ‘everybody does it’ response is nonsense. We don’t.
[Response: Huh? You don’t collaborate with your co-authors on responses to reviews? Really? And you don’t suggest potential reviewers to journal editors when they ask for suggestions? Really? – gavin]
The revelation is the degree of backbiting, smear and sabotage indicated in the leaked emails. Reviewers of papers are pre-selected, plans are hatched to remove an editor for publishing skeptical research, instructions shared for deleting emails in anticipation of FOI requests.
Now I am sure there is some naivety on my part as to how I expect scientists to behave at public funded institutions, but these emails are a damming indictment of a culture of smug entitlement.
[Response: This is a typical over-reaction. Perhaps you are unaware that almost all journals demand that you submit names of potential reviewers as part of the submission? Perhaps you are unaware that 6 editors of Climate Research resigned because of the way the Soon and Baliunas paper was handled? Or aren’t scientists allowed to give their opinions to colleagues? – gavin]
Can you explain the multiple references in the emails to evading FOIA responses, for example as in “delete all email [on certain topic] and I will do the same”?
[Response: No. But I am not party either to those FOIA requests, nor the timing and nor do I know what happened or what the scope was. – gavin]
“You’re also welcome to do a followup guest post, etc. think of RC as a resource that is at your disposal to combat any disinformation put forward by the McIntyres of the world. Just let us know. We’ll use our best discretion to make sure the skeptics dont’get to use the RC comments as a megaphone…”
[Response: This is a moderated site, and always has been. We do screen out a lot of the random squawk of the blogosphere and the baseless accusations of malfeasance that are commonplace on open forums. We do that unapologetically in order to maximise the signal-to-noise ratio in threads. When there are technical issues that we aren’t qualified to judge, we often ask people more involved to comment – and these comments appear inline with the questions so that the answers are not lost and are provided at the same time as the question appears. This leaves a record for later readers that is much easier to follow. We don’t screen out comments just because they disagree with us (as is evident in any comment thread). – gavin]
No one is saying that you screen out comments just because they disagree with your viewpoint.
However, the emails do suggest that certain people try to use this site in such a way: filtering out the naysayers.
To JACK KELLY:
It’s not about not wanting to fight a climate in peril. It’s about how it is spun to the public, and the intend that some people have with doing so…
You cannot deny that the climate debate and how to handle the future changes, have yielded big business for a few people.
The problem is, when bit money is added to the equation, the interest in doing something about the changes fast declines. The longer the idea of discussing is going on, the longer you can make the public pay irrational taxes, buy eco friendly gear, then get the “all new and better” model next year, because of course we didn’t have that on manufacturing belt already.
There’s no doubt in my mind that some or many of you so-called scientists report or feel responsible to certain people. But you have to remember who it is that pays grants and “owns” your institutions. All the way at the bottom (top?), you’ll find the public, and it is us you report to,
NOT the likes of Al Gore and friends.
You owe us the whole truth and we will have it… too bad that some hacker had to deliver it to us, when you actually had the chance yourselves.
Amount of trust and respect left now = 0
#1047388489
“This was the danger of always criticising the skeptics for not publishing in the “peer-reviewed literature”. Obviously, they found a solution to that–take over a journal! So what do we do about this? I think we have to stop considering “Climate Research” as a legitimate peer-reviewed journal. Perhaps we should encourage our colleagues in the climate research community to no longer submit to, or cite papers in, this journal. We would also need to consider what we tell or request of our more reasonable colleagues who currently sit on the editorial board…What do others think?”
@Gavin #41
You’re concerned about ‘signal-to-noise ratio’? I’d much prefer transparency and a more open and reasonable debate.
As bad as it’s going to be in the blogosphere with these emails being disseminated and discussed once “Old Media” gets a hold of the story things are really going to get out of hand. I can just imagine the spin that’s going to get put on this once it hits television.
> It will be another one to add to the long, long list that is ‘final proof that Global Warming is a hoax’. And it won’t be the last.
Thank you for acknowledging, even as you refuse to accept, that there is a “long, long list of reasons” to question AGW science. I realize that wasn’t your intent, but still, thank you.
[Response: He’s pointing out that there is a long list of claims that GW is a hoax, not that there is any actual evidence that it is. – gavin]
“…Can you explain the multiple references in the emails to evading FOIA responses, for example as in “delete all email [on certain topic] and I will do the same”?
[Response: No. But I am not party either to those FOIA requests, nor the timing and nor do I know what happened or what the scope was. – gavin]…”
You may not be a party, but, can you explain why anyone in climate research would write emails that attempt to evade FOIA responses?
[Response: Everyone involved in any FOIA request is always asking what is required to be responsive, what the scope of the request covers and what issues there might be that would provide exemptions. Such discussions go on every time. I can only speak about my own experience with that and the guidance we were given with respect to those requests. Partial views of that process without the full context would make it extremely easy to jump to unwarranted conclusions. – gavin]
Transparency can’t hurt.
I’ve been looking at the data for the last couple of hours and most is unremarkable, although emails gloating over the deaths of skeptics are unseemly at best.
CO2 is a greenhouse gas regardless of what are in the emails. CO2 production is increasing globally regardless of how data are fitted to trendlines. Nature imposes variability that is very difficult to pin down so models won’t be predictive for a while (except in the long term). All pretty simple.
It would help kill off the insane wing (OK – the bulk) of the skeptic community to simply make the research process and data production open. Absolutely open. As for people protecting their methods for professional reasons I think the risks at hand make that too expensive of a luxury for the planet to afford.
Trying to reason with the denialists in this thread reflects how that works proceeds in other areas. Trying to explain to some people that they are selectively misreading 62 megabytes of personal e-mails and in-progress documents, when they have already misread the consensus on climate science as a whole, can not possibly succeed.
“When the FOI requests began here, the FOI person said we had to abide
by the requests. It took a couple of half hour sessions – one at a screen, to convince
them otherwise
showing them what CA was all about. Once they became aware of the types of people we were
dealing with, everyone at UEA (in the registry and in the Environmental Sciences school
– the head of school and a few others) became very supportive. I’ve got to know the FOI
person quite well and the Chief Librarian – who deals with appeals. The VC is also
aware of what is going on – at least for one of the requests, but probably doesn’t know
the number we’re dealing with. We are in double figures.
“Response: Nonsense. Anger is restricted to people who misrepresent views and make up false accusations. Asking real questions about real issues is welcome. – gavin]”
Come on, man. “Where is the code so I can run this reconstruction?” is not a real question? [edit]
[Response: Sure it is. Which reconstruction did you have in mind? Try these Wahl and Amman or Mann et al 2008 for starters. – gavin]
If the biggest idea in the denier world is a stolen corpus of other people’s scientific correspondence, processed data files, and code, then it doesn’t really speak well of what they can produce on their own, does it?
Another inconvenient truth perhaps?
Nice try.
Two of those emails appear to document a means of “responding” to an FOI request that, if true, evidences unlawful action.
That is, a request to intentionally destroy material that was specifically requested.
In the US at least that sort of “response” gets you a date with Bubba. Who knows what the law is on this in the UK, but I bet they look dimly on it as well.
This is just sad, on so many levels, for so many of us.
I’ve been such a supporter and now this feeling of doubt creeping into me has me shaken and confused.
Well, it seems that the truth is, in one respect, out.
I’m lighting a cigar as I type. Perhaps we should all take time out to reflect for a moment.
I’m not talking about published responses, Gavin, I’m talking about manuscripts sent to referees for peer review. Those are sent in confidence. Collusion in preparing such reviews is completely unethical.
[Response: I have no idea to what you are referring. – gavin]
“…[Response: Everyone involved in any FOIA request is always asking what is required to be responsive, what the scope of the request covers and what issues there might be that would provide exemptions. Such discussions go on every time. I can only speak about my own experience with that and the guidance we were given with respect to those requests. Partial views of that process without the full context would make it extremely easy to jump to unwarranted conclusions. – gavin]…”
Gavin,
Would you, personally, delete any email or other data to avoid it being released to a FOIA request?
[Response: Of course not. -gavin]
I’ve frequented this site for years. I think this might be the most ‘open’ discussion I’ve read. Thanks at least for that. I do think this leak casts big ‘science’ in a very bad light. Figuring out creative ways to ovoid release of data, etc., is embarrasing at least.
I posted a few comments on WUWT but I am sure that they fell on deaf ears.
Anthony Watts and I had a private email message exchange back in early October when the whole Briffa thing was trumped up and glorified at WUWT. I defended science by using a medical example and because Briffa was sick, Watts publically accused me of being insensitive – even though WUWT and CA were flaying Briffa and the credibility of temperature reconstructions.
In some of those emails, RC was discussed and Watts warned me:
Fair warning – this communication is private. and
As I said before this communications (sic) is private, share it with Gavin or anyone online at your own risk.
I will keep my promise and these emails from Watts will be kept to me.
Funny how Watts feels free to publish others’ emails.
Am I taking these out of context?
From the file “ipcc-tar-master.rtf”:
[quote]
General Comments
The idea that climate without human intervention can only undergo “natural variability”, and that “climate change” can only result from human activity is false and fallacious. It is in conflict with all that we know of evolution and geology. It is simply wrong to assume that “ climate change” automatically implies human influence on the climate.
This fallacy is embraced by the Framework Convention on Climate Change, but the IPCC (Footnote to “Summary for Policymakers. Page 1) claim that they are prepared to accept “natural variability” as “climate change”. They are, however, unwilling to accept the truth, which is that climate can change without human intervention.
….
47 out of 91 models listed in Chapter 9 assume that carbon dioxide in the atmosphere is increasing at the rate of 1% a year when the measured rate of increase, for the past 33 years, has been 0.4% a year. The assumption of false figures in models in order to boost future projections is fraudulent. What other figures are falsely exaggerated in the same way?
[/quote]
[Response: Definitely. This is a mis-informed review comment on the ipcc draft (I think). Models can be run with any number of scenarios – 1% increasing CO2 is a standard that is useful for comparing model responses. Looking at comparisons with the real world requires scenarios closer to what has actually happened – and those scenarios are used too. Nobody has any problem with the idea that climate change can occur naturally. Why else would we study the ice ages? – gavin]
How about this? FOI, by the way, is a reference to the U.S. Freedom Of Information Act.
[Response: Almost certainly the UK version I would imagine, and would not apply to US based correspondents. – gavin]
Please keep that in mind when you state the proper context.
[quote]
From: Phil Jones
To: “Michael E. Mann” , “raymond s. bradley”
Subject: A couple of things
Date: Fri May 9 09:53:41 2008
Cc: “Caspar Ammann”
….
2. You can delete this attachment if you want. Keep this quiet also, but this is the person who is putting in FOI requests for all emails Keith and Tim have written and received re Ch 6 of AR4. We think we’ve found a way around this.
[/quote]
JohnH are you just going to keep cutting and pasting potentially fraudulent emails? Care to actually say something?
The internet– guilty until proven innocent.
I hope that the prudent and rational ones amongst us will decide to reserve judgement until the content of the emails can be verified as being true. Nothing is what it appears to be on the web. Also, context is everything, and people like JohnH it seems are only too happy to take things out of context.
That said, should there be proof of serious scientific misconduct by climate scientists down the road, then I for one will feel betrayed by groups like CRU and RC.
The sad thing is that the globe will continue to warm and this fiasco is going to make it extremely difficult to make people proactive.
“Per Edman says:
20 November 2009 at 2:29 PM
Trying to reason with the denialists in this thread reflects how that works proceeds in other areas. Trying to explain to some people that they are selectively misreading 62 megabytes of personal e-mails and in-progress documents, when they have already misread the consensus on climate science as a whole, can not possibly succeed.”
“Andrew says:
20 November 2009 at 2:30 PM
If the biggest idea in the denier world is a stolen corpus of other people’s scientific correspondence, processed data files, and code, then it doesn’t really speak well of what they can produce on their own, does it?”
I haven’t seen anyone here (yet) deny GW/CG, only asking for an explanation and clarification of what has been exposed/leaked to the public. And it does look poorly upon those who are on the “non-denier” side of the climate change debate. Ignoring the dirty laundry (who hasn’t said disparaging/discourteous things about colleagues/coworkers/opponents from time to time, even when we shouldn’t?) and the numbers (which can be taken and twisted any number of ways with enough time), talking about deleting emails regarding FOIA requests, and exerting pressure to hold up reports or comments, doesn’t reflect well upon those who sent the emails, does it?
What has happened?? In the 1970’s, when the New York Times published the “Top Sectret” Pentagon papers they were HEROES! And Nixon was thrown out of office for, among other lesser things, trying to find out that Daniel Ellsburg was the leaker. Now, some are saying it was criminal to print this information. How convenient. I guess that it depends upon the issue, and which side you are on.
thanks for the past reply. it was necessary.
but at the moment, i think the whole affair is demonstrating the mindset of those, who call themselves “sceptics”.
they immediately published the story. from a really dubious source. no scepticism anywhere.
Okay, between messages here and what can be read in the e-mails, Gavin’s response is clear: “We are the signal, you are the noise.” Maybe a bit more honesty and a few apologies and this could end simply as “good scientists behaving badly”.
The cat is out of the bag. You can damage control all you want, but we now know numbers are fudged and there are exterior motives for wanting global warming to be true. Al Gore – it’s over…unfortunately, you’ve still made millions of dollars off a hoax.
[Response: Perhaps you can tell that to my bank manager? Feel free to deliver the money to my personal jet (see below). – gavin]
Hacking? Looks more like whistleblowing.
Oh just great. Some of the media is now reporting this was done by a ‘whistleblower’ not a hacker, and that there is a lot more to come. Great job guys. Never, ever discuss how to clean up data in emails. You should be smarter than that.
Gavin, thank you for your civil and thoughtful responses to this. I don’t envy you your task today.
RichardC @64
>I’ve been such a supporter and now this feeling of doubt creeping into me has me shaken and confused.
Why?
The physics are pretty simple – CO2 reflects infrared. The planet radiates heat as light from the Sun bounces off of it. The more CO2 the more radiated heat stays here and less radiates to space. Result – warming. None of that is undermined by anything in the hacked files. How quickly the planet will heat under what CO2 concentrations is not pinned down yet to absolute certainty yet, but these files neither significantly contribute to nor really undermine the efforts to pin that down.
“MapleLeaf says:
20 November 2009 at 2:43 PM
JohnH are you just going to keep cutting and pasting potentially fraudulent emails? Care to actually say something?”
One of the CRU folks has said that the emails appear legit. And yes, if you had an axe to grind, wouldn’t you cut out the pertinent pieces of information? I don’t know about you, but if you took all the work emails I’ve sent over the past ten years, you’d probably give up trying to find anything incriminating after a few hours. So I am not surprised that the emails appear to be cherry picked for a specific purpose, likely an attempt to derail the Copenhagen event to some degree.
I call upon someone to specifically target those emails from CRU with a proper FOIA request. So that they can fairly and legally be brought to light.
By the way, is possession and redistribution of stolen data legal?
Eh … since you brought it up … what are the mating habits of European butterflies?
I think it’s revealing to the public for the first time that you scientists who are studying global warming are not just following the science and where it leads. You are actively seeking to manipulate the data to, in the immortal words of Governor William J. LePetomane, “…..protect our phoney-baloney jobs.”
I read about Bali and how you all went aboard your own private jets. How the airport ran out of space for all of them. Have you people never heard of the INTERNET and how you can TELECONFERENCE to anywhere in the world?
“I’ll believe it’s a crisis when the people who are telling me it’s a crisis start acting like it’s a crisis.” Glenn Reynolds
[Response: Please let me know where I can pick up my personal jet. Will they have bike parking there? Otherwise it’ll be a little difficult. – gavin]
This is truly a vexing day for GW believers. But there is reason to hope that this will all blow over. GW has always been more about the idea then actual science. The idea of GW will be little diminished and in the weeks and months to come new science will be brought forth to bolster flagging confidence.
Luckily for us, Oprah just quit or the major networks would be all over this.
I’ve had a look at HARRY_READ_ME.txt . Forget the conspiracy, forget the emails: the software development and database management practices are atrocious. I’m sure their CS colleagues are mortified. You could not get a simple medical device on the market with that kind of software development, yet these results justify fundamentally changing the entire world economy?
Clearly it is wrong to leak/ hack and publish private correspondence. But your response is of denial and rationalisation; stop and reflect on how this makes you look!
This is a sad day for science, a sadder day for academe and a downright disaster for what looks like a previously well orchestrated publicity machine.
John
“…JohnH are you just going to keep cutting and pasting potentially fraudulent emails?…”
Gavin,
There is thought that some or all of these emails that were leaked are fraudulent? As far as you know, are any of them fake? If some/all are, could you post which ones are not real?
[Response: I’m not in a position to tell for the majority of them. The ones I sent have not been tampered with, but then there is nothing particularly interesting in them either. – gavin]
I see a problem when it comes to suggesting that the “skeptics” need to publish in the peer-reviewed literature (which is something that I continually push as well), all the while working to try to prevent them publishing in the literature. I can pretty much guarantee that several recent papers that I (and co-authors) have submitted to the peer-reviewed literature would have been accepted had they carried different authorship. And I would bet that this is not only limited to my co-authors. I grow more suspicious that submitted papers that include particular authors are red-flagged for ‘special treatment.’ For a long time I denied (to myself and others) that this was the case, but recent experience has me thinking differently. I hope that I am wrong. Today’s information has done little to reassure me.
-Chip
John Bunt, good point, except they did not have the internet back then. These days nothing is what it seems, even if in B&W. They need to verify that the alleged damning emails are in fact real first. Anyone, can fabricate and or edit text. So how about we reserve judgement until this has all been resolved?
Alas, WUWT CA and other denier blogs are all assuming that this information is legitimate because it gives them a lifetime worth of ammunition. I’m sure if it were their data and emails being questioned that they would hope and expect others to not disseminate the potentially fallacious/edited material.
So regardless of which side of the debate you are on, disseminating private emails in the public forum which have been obtained illegally is criminal, and to do so without their authenticity being verified first is downright irresponsible. The internet is truly a nasty medium.
Remember that the comments here are moderated, so you can’t really trust the comments here to be representative of anything but the opinions of the moderator.
For all those hrumpfing on about “In the business world, we expect our emails to be read in open court!” I too live in the business world, and that attitude, at least in the companies I work with, is a direct result of the Enron trials, which started in 2006. My memory is that few people worried about their emails being made public 10 years ago. Even today, I run into folks that are astonished that their employer has the right to read all the mail on company accounts.That someone removed from the litigious jungles of the business world would be less that paranoidly cautious in emailing collaborators is no sin.
[Response: It’s worth pointing out that many of these emails date to well before 2006. – gavin]
How I love the smell of denialist concern trolling in the morning. :)
As Dano says, they (still) got nothin’.
You say that this disclosure shows evidence of the scientists’ humanity. I have read hundreds of the emails and what I see demonstrated is immature petulance and anger. All the complaints of the burdens of disclosure would drop to near zero if these people were simply transparent from the beginning.
I saw the “hide the decline” email being repeated all over the internet, and so I had already made a check on the background for myself. I found the “divergence problem” pretty quickly, and had already pointed this out at Deltoid. Nice to have my inferences confirmed.
These emails obviously lack useful context, and the maliciously clueless are going to keep making these kinds of misunderstandings. It says a lot about their ethics in running so quickly with accusations of hoax before they’ve made any attempt to figure out what is being discussed.
Its clear these folks are not “skeptics”, but credulous simpletons who jump at any chance to have their own presumptions confirmed.