As many of you will be aware, a large number of emails from the Climatic Research Unit (CRU) at the University of East Anglia webmail server were hacked recently (Despite some confusion generated by Anthony Watts, this has absolutely nothing to do with the Hadley Centre which is a completely separate institution). As people are also no doubt aware the breaking into of computers and releasing private information is illegal, and regardless of how they were obtained, posting private correspondence without permission is unethical. We therefore aren’t going to post any of the emails here. We were made aware of the existence of this archive last Tuesday morning when the hackers attempted to upload it to RealClimate, and we notified CRU of their possible security breach later that day.
Nonetheless, these emails (a presumably careful selection of (possibly edited?) correspondence dating back to 1996 and as recently as Nov 12) are being widely circulated, and therefore require some comment. Some of them involve people here (and the archive includes the first RealClimate email we ever sent out to colleagues) and include discussions we’ve had with the CRU folk on topics related to the surface temperature record and some paleo-related issues, mainly to ensure that posting were accurate.
Since emails are normally intended to be private, people writing them are, shall we say, somewhat freer in expressing themselves than they would in a public statement. For instance, we are sure it comes as no shock to know that many scientists do not hold Steve McIntyre in high regard. Nor that a large group of them thought that the Soon and Baliunas (2003), Douglass et al (2008) or McClean et al (2009) papers were not very good (to say the least) and should not have been published. These sentiments have been made abundantly clear in the literature (though possibly less bluntly).
More interesting is what is not contained in the emails. There is no evidence of any worldwide conspiracy, no mention of George Soros nefariously funding climate research, no grand plan to ‘get rid of the MWP’, no admission that global warming is a hoax, no evidence of the falsifying of data, and no ‘marching orders’ from our socialist/communist/vegetarian overlords. The truly paranoid will put this down to the hackers also being in on the plot though.
Instead, there is a peek into how scientists actually interact and the conflicts show that the community is a far cry from the monolith that is sometimes imagined. People working constructively to improve joint publications; scientists who are friendly and agree on many of the big picture issues, disagreeing at times about details and engaging in ‘robust’ discussions; Scientists expressing frustration at the misrepresentation of their work in politicized arenas and complaining when media reports get it wrong; Scientists resenting the time they have to take out of their research to deal with over-hyped nonsense. None of this should be shocking.
It’s obvious that the noise-generating components of the blogosphere will generate a lot of noise about this. but it’s important to remember that science doesn’t work because people are polite at all times. Gravity isn’t a useful theory because Newton was a nice person. QED isn’t powerful because Feynman was respectful of other people around him. Science works because different groups go about trying to find the best approximations of the truth, and are generally very competitive about that. That the same scientists can still all agree on the wording of an IPCC chapter for instance is thus even more remarkable.
No doubt, instances of cherry-picked and poorly-worded “gotcha” phrases will be pulled out of context. One example is worth mentioning quickly. Phil Jones in discussing the presentation of temperature reconstructions stated that “I’ve just completed Mike’s Nature trick of adding in the real temps to each series for the last 20 years (ie from 1981 onwards) and from 1961 for Keith’s to hide the decline.” The paper in question is the Mann, Bradley and Hughes (1998) Nature paper on the original multiproxy temperature reconstruction, and the ‘trick’ is just to plot the instrumental records along with reconstruction so that the context of the recent warming is clear. Scientists often use the term “trick” to refer to a “a good way to deal with a problem”, rather than something that is “secret”, and so there is nothing problematic in this at all. As for the ‘decline’, it is well known that Keith Briffa’s maximum latewood tree ring density proxy diverges from the temperature records after 1960 (this is more commonly known as the “divergence problem”–see e.g. the recent discussion in this paper) and has been discussed in the literature since Briffa et al in Nature in 1998 (Nature, 391, 678-682). Those authors have always recommend not using the post 1960 part of their reconstruction, and so while ‘hiding’ is probably a poor choice of words (since it is ‘hidden’ in plain sight), not using the data in the plot is completely appropriate, as is further research to understand why this happens.
The timing of this particular episode is probably not coincidental. But if cherry-picked out-of-context phrases from stolen personal emails is the only response to the weight of the scientific evidence for the human influence on climate change, then there probably isn’t much to it.
There are of course lessons to be learned. Clearly no-one would have gone to this trouble if the academic object of study was the mating habits of European butterflies. That community’s internal discussions are probably safe from the public eye. But it is important to remember that emails do seem to exist forever, and that there is always a chance that they will be inadvertently released. Most people do not act as if this is true, but they probably should.
It is tempting to point fingers and declare that people should not have been so open with their thoughts, but who amongst us would really be happy to have all of their email made public?
Let he who is without PIN cast the the first stone.
Update: The official UEA statement is as follows:
“We are aware that information from a server used for research information
in one area of the university has been made available on public websites,”
the spokesman stated.“Because of the volume of this information we cannot currently confirm
that all of this material is genuine.”“This information has been obtained and published without our permission
and we took immediate action to remove the server in question from
operation.”“We are undertaking a thorough internal investigation and we have involved
the police in this enquiry.”
Update II: Please comment on the next thread.
At the very least, the emails reveal the bias of the researchers. Ethical research tests a hypothesis, not proves a hypothesis. There is a fever amongst the scientists that their mission is to provide stunning evidence of AWG balanced with subtle disclaimers. This in itself should raise question about the validity of their research.
[Response: Nonsense. So people concluding something from their own work and that of others proves that their own work is invalid. Try thinking about it a little more. – gavin]
I’ve just worked out something quite important and personally I find it rather reassuring, actually. Bear with me, and give this theory a go.
Those who have criticised AGW for years are behind the email thefts. Now, it has been alleged by these same people for years that academics are involved in a giant conspiracy to cover up the truth about the data and the science. Of course, the headlines here are supposed to show exactly this. However, so far I haven’t read one single thing that suggests this – the closest is the allegations of suggesting the peer-review editors, and this does need a better explanation to the general public. But at the very least, this way of working does appear to be pretty much the norm across science.
But is that it? Really? Because these leaks are (presumably) the edited highlights of the most incriminating private emails right at the very heart of the conspiracy – the place where the battleground is drawn, and the tactics discussed in great detail (conspiracies are tough to organise, of course). And what do they have to show for it? Nothing.
If people take a step back, this release of emails is an own-goal – it more or less proves that the alleged conspiracy is a figment of peoples’ imagination. If it were real, we’d see something – anything – that wasn’t easily explainable as normal professional discussion.
However, I’m concerned that this is a moot point. The news stories all use the bogus “trick” quote, and that’s as far as the supposed-sceptics will ever read.
I think RC has done well with its initial post, which was quickly put out there and very useful. I think the public need to hear more candid detail on how the peer-review process works in practice, and lets hear from other science disciplines too to put it into context. But as to the substance here – there appears to not be any. And that’s extremely telling indeed.
@ Jinchi #750
I think it is quite clear that there have been legitimate concerns raised that do not have anything to do with ‘conspiracy’ or ‘hoax’, but with the apparent politicisation of science. It seems obvious to me (and I am not a ‘denier’) that the scientists involved do have questions to answer regarding evasion of FOI requests, and in my mind the process of peer-review does not come out of this episode undamaged.
Jinchi, Eli has a somewhat different take on this.
Whatever validated & audited datasets exist in support of upcoming global decisions,they need to be secured in one place outside of CRU,GISS or wherever they currently exist and subjected to full INDEPENDENT review outside of the current clique. ( that includes Hadley ,CRU RC, WUWT etc etc).No global actions will be acceptable to any government without these basic prerequisites being achieved and seen to be achieved. Currently they are not…..
Your (indirect) response to my earlier query included the following:
“They got access to some extra data that some National Met. Services normally only sell, or was given with the express proviso that it not be passed on to third parties. CRU is not at fault for honoring those agreements – even if everyone wishes they didn’t exist.”
I must say, I didn’t get the impression from the various emails that everyone wished all the data were public – indeed there was talk of deleting information rather than giving way to FOI requests!
More generally, surely either this private data was not essential to build your case for anthropogenic global warming – in which case why use it – or you are, in effect, asking the world to divert trillions of pounds to a project whose justification is being kept secret!
[Response: Your definition of secret is obviously a little different from mine. – gavin]
I’m a farmer who’s been growing vegetable crops in central Indiana for the past 29 years. When I began we experienced some -20 below temperatures pretty much every winter until the middle 90’s. Since that time I could count on both hands the number of sub zero days. On Christmas day 1983 I carried my newly born son (Now a bio chemist researcher at the UNC, Chapel Hill Medical School) down our 600 foot snow drifted driveway in -25 degree weather. We did get down to -5 one evening last winter, the first time in several years. Conversely our summers have been milder for the last decade.
Three years ago we didn’t really have a winter. January saw many 50 degree days and some 60 degree days and in February we had several days with temps in the 70’s. By mid March the leaves on the trees we’re nearly leafed out something I’d never seen in my 60 plus years. In mid early April we had three nights of 19 degree lows which reduced many of the tree leaves to goo. I lost over half of my blueberry crop. My wife a botanist was concerned that if a pattern were to develpe the hardwood trees would find it difficult to survive.
Today for the first time in my farming career we harvested broccoli, cauliflower and lettuce from the open fields. We planted these crops late on purpose betting that the warmer winters might continue. We’ve been blessed but how about those folks in other parts of the world?
Yes I know that weather is local and that the warmer winters and the cooler summers in my area don’t necessarily reflect what’s going on elsewhere but my observations have added to my believe that something is happening and I don’t need some factoids to convince me otherwise.
@802 Bunyip: Changing is mind is nothing to be ashamed of.
780
Bunyip says:
22 November 2009 at 9:00 AM
“Please, will someone tell me how to counter the criticism tomorrow at work.”
The best “counter” to ignorance is knowledge. There really aren’t that many emails to read through. The vast majority of them are just standard science discussions. The “inflammatory” materials really aren’t, especially if you read through all the related emails. At best you can only get a partial picture of what was going on since there are large gaps.
Unfortunately, this means little to people who have an emotional investment as opposed to a logical. Logic and reason left this stable about ten years ago. People, especially the Armchair Climatologists, already have their conspiracies and psuedo-science in place and no amount of logic or reason will change that.
“Please, I beg you, tell me what to think!”
No. It’s far better for you to gather all the facts and come to your own conclusions. The email archive has been on bit-torrent for awhile. I’d recommend doing your own research into this as opposed to having someone give you their own impression.
That being said, after reading through the emails my impression is that there isn’t anything in them at all that points to anything nefarious. However, I did come away with an even worse impression of McIntyre et al. than I already had. He’s one of main reasons they wanted to “deny” FOI requests in the first place and it had nothing to do with wanting to hide data.
~X~
Wow, 17 pages and still going. The next Nobel Peace Prize should go to Gavin and the RealClimate guys for their superhuman forbearance.
813
Jinchi says:
22 November 2009 at 9:49 AM
“..why are they wasting their time with snippets of how much Ben Santer hates Steve McIntyre?”
Why attack the science when attacking the person is SO much easier? Ad hominem attacks are so much more satisfying it seems.
“Shouldn’t they be showing us charts, graphs, raw data that destroy the climate science community once and for all?”
Because that requires solid research, and that is something the skeptics do not have. If the “debate” about climate change had adhered strictly to facts then sites like CA wouldn’t exist.
~X~
Gavin,
I don’t think it matters whether or not the hacking and subsequent publication of emails is illegal. The point is that damage has been done to the credibility of climate science. I have been a long believer in the transparency of science and the need for free access to base data and corresponding meta-data, techniques used and arguments for inclusion of exclusion of data. We faced this problem early in the TOGA period and developed a 2-year policy that has been widely accepted in funding agencies world wide.
If the CRU data had been made publicly available for scrutiny at an earlier time, then these recent events would have been irrelevant. The data sets and their interpretations would have sunk or floated on their merit. Given the importance of a surface temperature record for the wide-ranging implications to society, science and etc., these data should have been made available a long time ago. The only way for credibility to be regained or earned is for the data sets in question to be made available for wide scrutiny. For data sets collected by a PI (e.g., ice cores, coral, and other proxy sets) the 2-year TOGA should perhaps apply. Accumulated data sets (e.g., collections of surface temperature records and etc.) need to be made available immediately along with the accompanying data I have mentioned above. I believe that it would be great folly for backs to be turned at this time. I know that I will hear that there are agreements with different countries in place that preclude making data widely available. I am sorry to say that would ring a little hollow. Data was made available to some of us and I am grateful for that as it has proven extremely important in trying to understand the 1935-45 warming. Perhaps a way around the fiscal issue is to state which data is precluded from being made available. I guess that is meta-data as well. But I think that we have to move on as openly as we can be.
#805 turtle
“And, seriously, who stands to gain money if they do? Some two-man business in California selling solar panels? Those companies that sell ‘green’ yoga mats and water bottles? THIS is who you think is funding some kind of global conspiracy?”
Turtle, my understanding is that many people are suspicious of AGM because of the tax-raising implications. It’s a question of liberty for these people – more tax goes from your pocket to government which often then goes to corporate welfare, which in turn promotes monopolies and oligolipolies. Small businesses are hurt by this.
Given that the fledging global carbon trading framework (which would be the key architecture behind nation-state taxation schemes) is being designed by many of the same people (Goldman Sachs et al) that are currently bringing Western democracies to their knees through unregulated derivatives, credit default swaps, etc, one can understand some of their concerns.
Jinchi – No, we are saying that we have no idea why data is being kept from independent researchers.
That is all. If you think your science stands up, why not give that data to your critics and silence them.
[Response: Perhaps you like to show us where that has happened? The more usual situation is that the data is made available to all, and the misrepresentations increase (cf. Mann, GISTEMP etc.). That doesn’t mean that data shouldn’t be made available (it should), but the argument that doing so silences the critics is bogus. – gavin]
“Mine is somewhere in the 30’s I think. – gavin]”
Accurate as ever Gavin.
Scopus has you on 23; which is better than my 20, [edit]
[Response: Happy to be corrected. Forgive me if self-googling is not high on my priority list this morning. – gavin]
Re Gavin’s reply to 776 (oversight question)
Gavin
Thanks for the response. It doesn’t feel as rigorously independent as the FDA/EMEA big boots, but maybe that’s just my perception of those ‘people’, and I’m just moaning! I do sympathise with your having to put up with the shit (and dim questions like mine!)
Thanks
Ian
Gavin, in view of the new-found openness on RealClimate (which unfortunately we only have the hacker of a computer to thank for), can we now look forward to you (or your associates) publishing the archive of correspondence between yourselves and the foreign Met Offices you mention – seeking to persuade them of the absolute importance of their releasing into the public domain the material upon which your science arrives at its consensus? Given the profound global changes required to respond to AGW – not to mention the vast sums of money it will require – I am sure this correspondence will have expressed to all such Met Offices the utmost urgency of their complying with this request (and thereby overriding any prior commercial interest they may have had in the material), so that the quality of your work can be tested and verified by the wider scientific community.
[Response: Here. – gavin]
Dear Gavin
Thank you so much for doing this very important job of riding the storm.
You shold not need to spend your time on these same questions over and over again. But I for one am very glad you take the time of patiently answering.
Gavin–You seem to believe that the peer review process as revealed in the emails is normal. As someone who has participated in review in other fields, I can assure you it is not. Perhaps your field is too small to ensure the required anonymity and diversity required, but the review process in climate science as it exists today cannot possibly function properly.
[Response: Not true. Peer-review is of course imperfect – people don’t have enough time, there are tens of thousands of papers to review, editors don’t always know who appropriate reviewers might be, and sometimes the process messes up. The three examples I mentioned above are great examples. But there is plenty of good critical reviewing going on and it generally leads to better papers in the literature. Having seen poor initial drafts morph into well-argued journal articles many, many times, I know this to be true. – gavin]
One wonders what the independent science bodies in Asia as shown here
http://www.earthtimes.org/articles/show/261395,thai-chinese-scientists-launch-climate-change-research.html
and other areas who clearly have done their own due diligence in advising their governments to spend billions upon climate change response and moving to a lower carbon society, think of this nonsense.
Does anyone really countenance the idea that this will have any impact beyond the deniers blogs? Other than perhaps prompting a bit more security in research centres.
The deniers seemed puffed up with their own importance, as abundantly shown in this case, when in reality, as shown in Copenhagen, the climate science and international community have moved on to dealing with the changes afoot.
I’ll try one more time to ask my questions (Post #734). Is it appropriate to remove observed data to maintain the validity of a model? Is all unexplained warming aside from stated natural variability automatically relegated to anthropogenic in nature? Why is any uncertainty suppressed? Is the current cooling understood?
Excerpt from [1255553034.txt]
At the risk of overload, here are some notes of mine on the
recent lack of warming. I look at this in two ways. The first is to
look at the difference between the observed and expected anthropogenic
trend relative to the pdf for unforced variability. The second
is to remove ENSO, volcanoes and TSI variations from the
observed data. Both methods show that what we are seeing is not unusual. The
second method leaves a significant warming over the past decade.
These sums complement Kevin’s energy work. Kevin says … “The fact is that we can’t account for the lack
of warming at the moment and it is a travesty that we can’t”. I
do not agree with this.
[end excerpt]
[excerpt]
Michael Mann wrote:
thanks Tom,
I’ve taken the liberty of attaching a figure that Gavin put
together the other day (its an update from a similar figure he
prepared for an earlier RealClimate post. see:
https://www.realclimate.org/ind…..pulation/). It is indeed worth a thousand words, and drives home Tom’s point below. We’re planning on doing a post on this shortly, but would be nice to see the Sep. HadCRU numbers first,
I did
[End excerpt]
[Begin excerpt]
Mike,
The Figure you sent is very deceptive. As an example, historical
runs with PCM look as though they match observations — but the
match is a fluke. PCM has no indirect aerosol forcing and a low
climate sensitivity — compensating errors. In my (perhaps too
harsh) view, there have been a number of dishonest presentations of model
results by individual authors and by IPCC. This is why I still use
results from MAGICC to compare with observed temperatures. At least
here I can assess how sensitive matches are to sensitivity and
forcing assumptions/uncertainties.
Tom.
[End excerpt]
[Begin excerpt]
Hi Tom,
thanks for the comments. well, ok. but this is the full CMIP3
ensemble, so at least the plot is sampling the range of choices
regarding if and how indirect effects are represented, what the cloud
radiative feedback & sensitivity is, etc. across the modeling
community. I’m not saying that these things necessarily cancel out
(after all, there is an interesting and perhaps somewhat disturbing
compensation between indirect aerosol forcing and sensitivity across
the CMIP3 models that defies the assumption of independence), but if
showing the full spread from CMIP3 is deceptive, its hard to imagine
what sort of comparison wouldn’t be deceptive (your point re MAGICC
notwithstanding), perhaps Gavin has some further comments on this (it is his plot after
all),
mike
[End excerpt]
[Begin excerpt]
Tom, with respect to the difference between the models and the data, the
fundamental issue on short time scales is the magnitude of the internal
variability. Using the full CMIP3 ensemble at least has multiple
individual realisations of that internal variability and so is much more
suited to a comparison with a short period of observations. MAGICC is
great at the longer time scale, but its neglect of unforced variability
does not make it useful for these kinds of comparison.
The kind of things we are hearing “no model showed a cooling”, the “data
is outside the range of the models” need to be addressed directly.
Gavin
[End excerpt]
[Begin excerpt]
Gavin,
I just think that you need to be up front with uncertainties
and the possibility of compensating errors.
Tom.
[End excerpt]
[Response: ;) Gosh disagreements amongst scientists… I will post the figure concerned in a post shortly (a few other things have come up in the mean time) – but basically it is just a figure showing all the IPCC models and the observational data plotted together on the 1980-1999 baseline that was used in IPCC. It shows that the observed temperatures are well within the expected model envelope. Tom’s point is (I think) that this doesn’t necessarily prove that the models have perfectly encompassed exactly what has been going on, and I would agree. That doesn’t mean it isn’t useful to show what the models actually did show. – gavin]
To emphasize my claim that Jones was not speaking of a cooling spell that he was trying to minimize the appearance of, merely look at any instrumental temperature chart and note that his email was written in 1999 when there was no decline in temperature to “hide” whatsoever.
I note that for pointing this nearly universal misconception out, the moderators deemed fit to include a comment that my comment was “In the running for the most foolish denial comment of the year.” It might be a a badge of honor if I had much respect for the average skeptic who refuses to pare down crappy arguments that do indeed amount to mindless “talking points” which they have not been skeptical about.
Shall I try for two? Then I will ask an extremely “silly question”:
Can you explain this chart which is a simple and accurate plot of thermometer records which shows no obvious AGW signal in 350 years?: http://i45.tinypic.com/iwq8a1.jpg
I’m sincerely curious. The chart is not an outlier. Note that HadCRUT3 (in blue) confirms that it matches the instrumental record very well.
[Response: Note that with such a huge number of comments moderation is slow and haphazard and comment numbers are not stable. Please use links and/or names directly. – gavin]
Release the data.
Let people see that it supports your conclusions.
If you don’t release the data, people have every reason to be skeptical.
[Response: All data and codes for the GISTEMP temperature record. Let me know if that assuages your criticism. – gavin]
re~857. Clearly there are so-called’ peer reviewed papers published and included in IPCC without any independent verification of the datasets. As Gavin has diligently explained ,there are instances where the original raw data has not been made available, citing copyright or other issues.
Gavin, in your response to #743, you make a statement that “science is a work in progress”. Why, then, does your side (not necessarily the scientists but the ill-informed politicians and Hollywood crowd)always insist that the debate is over?
[Response: The debate on whether CO2 is a greenhouse gas is over. The debate on whether CO2 is increasing because of human activity is over. The debate on whether the globe is warming is over. The debate on whether this and other human impacts can be seen in the climate system already is over. These are not the issue upon which scientists are expending their energies. We are trying to work out exactly how the different impacts intersect, how we can better understand specific processes in the system, how to constrain uncertainties in projections of the future. I’d be happy to say this to any politician or Hollywood celebrity. – gavin]
Please respond to the charge that the exaggerated claims of AGW have been falsified by the new Lindzen & Choi paper “On the determination of climate feedbacks from ERBE data” which shows from satellite data that the feedback is negative (not positive) and sensitivity is only one sixth of that claimed by AGW advocates. Roy Spencer finds the same thing.
Failure to respond will be taken as your admission that Lindzen and Spencer are correct, and that temperature increases from greenhouse gases are modest, certainly nothing to cause alarm.
[Response: Funnily enough, it is Roy Spencer who has the most accessible criticism of Lindzen’s paper. Let me know which of the mutually inconsistent versions you want us to respond to. – gavin]
Here’s something on this so-called “trick” …
Trick’n by DarkSyde Sun Nov 22, 2009 at 07:59:20 AM PST at website
http://darksyde.dailykos.com/
and it ends.
“But consider; it’s taken me several grafs, and you a few minutes of reading, just to get a glimmer of what that one email was all about. The same effort would be required to untangle other stolen, out of context emails now brandished by skeptics as evidence of some kind of shadowy conspiracy. That’s how easy it is to pluck something out of context and make it sounds ominous, if your goal is to misinform, prostitute yourself to the energy industry, and — pardon the pun — trick your readers. “
This statement is ‘not even wrong’.
Emails are never personal / private whenever they are composed while being paid by an employer. The employer has provided the employee-writer all the equipment and support structure and the salary that allows the writing to be done when at the place of employment.
Whenever all the equipment and support structure and salary are paid for by public funds, the emails belong to the public.
More specifically, there are local, state, and federal-level laws against use of any publicly-supplied equipment for personal applications. Use of publicly-supplied computers for writing personal emails falls under these laws.
Many, almost all, private companies will have conditions of employment that are essentially the same as these.
What you do at work belongs to your employer. If you are a public employee, what you do at work belongs to the public.
This is not rocket science. Millions of people go to their place of employment every day fully aware of the rules / regulations / conditions of their employment.
Climate “science” attempts to invoke yet another exemption from what is standard operating procedures for the remainder of the universe.
This string has grown so fast that I have not been able to keep up – hopefully what follows is not too redundant. First, I would like to thank Gavin and others for his demonstrably more liberal moderating of these comments – though I would note that I have personally experienced what I would see as inappropriate editing in the past at Real Climate.
Anyone who has worked in a highly competitive environment will not be surprised by many of these emails. The notion that they somehow either prove or disprove a scientific conspiracy as opposed to a very cohesive coalition of like-minded individuals is an over-reach. They do, however, reveal some questionable behavior by people who should know better. With respect to the peer review process, many academics are probably all too familiar with the politics and personal animosities associated with article publication in prestiguous scientific journals. This is an important issue but now new. Dr. Roger Pielke Snr has already written on this point and I suspect that he will have a much stronger hand now when he pursues further action as a result of the revelations in these emails.
So what is the issue? At core, it is the behavior of key scientists in response to requests to release data. The emails show a pattern of deliberate efforts to undermine the existing legal process for freeing information. The emails are unambiguous as to the efforts of Dr. Jones on this count. Given the title of the file, its content and the timing of its release, in close proximity to a rejection of an appeal by Steve McIntyre for releasing data and other information – the odds have dramatically increased that this was not the action of hackers but of whistleblowers. This is very important and significantly changes the import of the content and the likely consequences.
If it is whistleblowers, Dr Jones and the UEA administration have dug themselves a very deep hole as revealed by these emails. If the person who released this file was privy to the FOIA discussions and objected to the stonewalling in writing or made contemporaneous notes, then any efforts to pursue them may result in even more damage to the credibility of Dr. Jones et al. It is a genuine Catch-22.
So, folks should try to keep an eye on the pea here. The assertions above by advocates for CAGW that skeptics believe that these emails somehow demonstrate a Michael Crichton-like conspiracy is a smokescreen that hides the simpler and more fundamental issue. There is and never has been a real reason for not disclosing the data and the code. The rather juvenile, silly and short-sighted efforts to stonewall McIntyre and others has produced the real scandal. The notion that McIntyre and many others are part of some vast conspiracy to delay action on CO2 emissions, besides being neurotic, vastly underestimates the sheer puzzle value of climate issues to those of us used to doing large scale data analysis in other fields. Love him or hate him, nobody has any grounds for doubting McIntyre’s (and a growing number of other “amateurs”) abilities to analyze complex data sets and uncover large and small data and analysis errors. The remarkable defensiveness displayed in these emails by many of the scientists has led IMHO to a continuous unwillingness to accept Steve McIntyre at face value. Releasing the data and the code in accord with sound scientific practices now looks like it would have been a smarter choice.
Very interesting read. My favorite email:
Hi all
Well I have my own article on where the heck is global warming? We are asking that here in Boulder where we have broken records the past two days for the coldest days on record. We had 4 inches of snow. The high the last 2 days was below 30F and the normal is 69F, and it smashed the previous records for these days by 10F. The low was about 18F and also a record low, well below the previous record low. This is January weather (see the Rockies baseball playoff game was canceled on saturday and then played last night in below freezing weather). Trenberth, K. E., 2009: An imperative for climate change planning: tracking Earth’s global energy. Current Opinion in Environmental Sustainability, 1, 19-27,
A.A.Tsonis’ A Dynamical Method for Determining Climate Shifts is a recommend read for Dr. Trenberth
You may be interested in what I’ve just been postng this around various sites.
What a reaction to the alleged disclosure of E-mails and other data from the University of East Anglia’s Climate Research Unit! This information has been flying around the Internet since 19th and if genuine potentially blows the lid off the all of the propaganda that has been promulgated about The (significant human-made global climate change) Hypothesis. Despite this there apparently has not been a word from yourselves, any political party member or broadcaster about it. This has much more significance than what today’s celebrities had for breakfast. Why no news coverage or political reaction?
In an article on this subject in the UK’s Daily Telegraph (Note 1) mention is made of John Daly. It says QUOTE: One of the alleged emails has a gentle gloat over the death in 2004 of John L Daly (one of the first climate change sceptics, founder of the Still Waiting For Greenhouse site), commenting: “In an odd way this is cheering news.” UNQUOTE. This alleged E-mail is presented more fully elsewhere (Note 2).
It is important to remain sceptical about the validity of this “leak” of information and await the results of a thorough investigation. (Is anyone in the news media doing something along these lines?). Despite this, there is a saying “there’s no smoke without fire”. It is interesting to see that there appeared to be an exchange of E-mails between John Daly and Phil Jones back in 2001 (Note 3). This item starts with QUOTE: After several requests by visitors to this website for details of the two emails which were sent by Phil Jones of CRU, demanding withdrawal of the articles about recent errors in CRU hemispheric temperatures, the following exchange of emails was made via a very large CC (110 addressees), with both of Jones’ emails signed in his official capacity as professor at CRU. UNQUOTE. It is followed by an apparent exchange of E-mails between John Daly and Phil Jones.
I leave you to read them and draw your own conclusions. While you’re at it, have a read of the comments at Wattsupwiththat (Note 2). There are some interesting comments about that site favoured by supporters of The (significant human-made global climate change) Hypothesis, Realclimate (Note 4). It leads off with an article spinning the motivations behind what appears in the E-mails followed by some uncharacteristic defensive responses to readers’ comments by Gavin Schmidt.
Another interesting commentary on this is at ClimateAudit (Note 5).
NOTES:
1) see http://blogs.telegraph.co.uk/news/jamesdelingpole/100017393/climategate-the-final-nail-in-the-coffin-of-anthropogenic-global-warming/
2) see http://wattsupwiththat.com/2009/11/19/breaking-news-story-hadley-cru-has-apparently-been-hacked-hundreds-of-files-released/
3) see http://www.john-daly.com/cru/emails.htm
4) see https://www.realclimate.org/index.php/archives/2009/11/the-cru-hack/comment-page-10/
5) see http://www.climateaudit.org/?p=7810
Pete Ridley, human-made global climate change agnos(cep)tic
“the review process in climate science as it exists today cannot possibly function properly.” Considering that garbage like S&B 03 or the recent miserable Carter piece can make it through, there might be some truth to that. Heck, even in Physics journals it occasionally fails, as we have seen with Gerlich & Tscheuchneuer…
[edit] here’s the link to the Washington Post.
It don’t look good…
http://www.washingtonpost.com/wp-dyn/content/article/2009/11/21/AR2009112102186.html?nav=hcmodule
Gavin,
I am sure your time is short at the moment, but I would really like to reach a conclusion regarding my previous two questions.
As I understand it, you have in the past released only part of the data relating to AGW on the grounds that some of it was private – but that CRU regretted this fact.
I pointed out that this regret was not exactly evident in the emails that have been released, and that for a conclusion of such momentous importance, it was vital that it be based on publicly available data and computer code.
This in turn provoked a response that questioned my use of the word ‘secret’, so let me try again! Would it be fair to say that:
a) The private data was not necessary for demonstrating AGW – so anyone could, in principle have reproduced your results from publicly available information.
b) The private data was necessary to demonstrate AGW, and the CRU was doing all it could to honour the FOI requests so that everyone could see the full picture (despite emails that discussed deleting information rather than supplying it in response to FOI).
c) Something else.
[Response: Who is ‘you’ in your question? I do not work for CRU, and I do not not work on the GISTEMP stuff either. But to answer your question, GISTEMP only uses publicly available data and correlates to 0.97 (or so) with the CRU global mean data. So no, you don’t need the restricted stuff to come to the same conclusion. Answer (a). – gavin]
Excuse me but I should have added something to my comment above. To my mind it is remarkable that Gavin’s comment above, Andrew Revkin’s piece, the pieces in the BBC, the Guardian, NYT, and most of the others I have seen fail to highlight the FOIA issue. I suppose the other stuff has greater voyeuristic value, but it really does miss the whole point of someone posting a file of emails and data title FOIA.
Keep your eye on the pea!!
I’m sure someone has already posted this, but I just had to laugh at one of oracle’s comments.
If all the data doesn’t fit the theory properly, then the theory is false.
If all the data does fit the theory, then the theory is false.
Gavin, you have the patience of a saint. :)
Re; Let people see that it supports your conclusions.
If you don’t release the data, people have every reason to be skeptical.
[Response: All data and codes for the GISTEMP temperature record. Let me know if that assuages your criticism. – gavin]
No this does not !! This thread resulted from the now-publicised mess which has accrued at CRU and Gavin well knows this.
[Response: What I know is that your comments have nothing to do with access to data or desire to see what the records are based on. I have no problem with you not wanting to use CRU data for your stated concern with accessibility to the raw data, but when you reject a suggestion that you look instead at a very similar product that does not have any such issue, you reveal your real motivation very clearly. – gavin]
Gavin, regarding the “will start deleting” comment being obviously in jest, you say: [Response: It’s obvious because I know the people involved. – gavin]
Fair enough. How about the later request that emails with Keith be deleted, was this also obviously in jest or is it possible that was an explicit request. (Note, I’m not saying those mails were FOIed or even FOIable. My interest is just in the behaviour of deleting things and of the writing in jest or seriously intended.)
Thanks in advance, man! Hang in there. Let the chips fall where they may. If there were things that skeptics are overtouting, call them out, sure. I do, all the time. Despite being more conservative than most of them (no kidding). Also, if there are other things that were wrong from your buds (“unfortunate”) be explicit as well. Principles are more important than sides getting traction in a meta-debate.
Gavin, above you said:
To me your response would mean that someone, or group, has hacked the individual accounts of the persons whose emails have been released, focused on certain topics to include and at the same time filter out the good science.
These people must also have sufficient experience and expertise in highly technical subject areas to know what to filter out and what to include. And not simply broad aspects of the subject areas, but focused on very specific content and time frames and persons and issues. The individual accounts must also have been under hack for sufficient time for reading and understanding of all the emails in each account to know which to include and which to exclude.
Note, too, that not just emails have been released, but also documents and computer code and data. A rogue outside hacker is very unlikely to have had time to digest the contents of this much and kind of material. Again all these materials are very focused relative to specific content, time-frames of interest, and individual persons involved.
To me this seems like it would require a very significant amount of time for even someone well-versed and focused on the objective material. Isn’t it very unlikely that this much outside activity could go undetected.
[Response: Most of the files that weren’t email, I think were attachments. And the selection seems likely to have been by searching for names rather than anything else. – gavin]
I must say, Gavin’s responses all tend to not really answer anything at all, most of the time. Mostly just pithy words, but often only tangentially responsive at best. Just looking over the comments and responses.
Here’s an idea…settle this once and for all and release the data and codes for all work. How about it, Gavin? Just release all your data and codes. If you have done so, great. Now get your colleagues to do the same. Criticism and scepticism will make your conclusions stronger if they were “correct” (i.e. non-falsifiable) to begin with. Otherwise, I’m afraid, your credibility suffers greatly.
[Response: People seem to think that I’m somehow in charge – Sorry, but I’m not. If you want a completely open sourced, publicly accessible surface temperature record go to GISTEMP. If you want the code that I work on go look at ModelE. If you want raw temperature data, go to NOAA. All this stuff is out there, don’t miss it in your haste to score political points. – gavin]
The way to settle these debates is to lean on sites like intrade.com to expand their climate markets.
If there were a large market, with real money at stake, the truth would have a much better chance of coming out.
As it is, folks on both sides are under pressure to exaggerate the strength of their respective cases. This applies just as much to one side as it does to the other.
Gavin, yes we agree that:
The debate on whether CO2 is a greenhouse gas is over. The debate on whether CO2 is increasing because of human activity is over. The other gases e.g., methane, get scant attention. Over the last ten years which gas is rising faster and how due you factor the relative potency of each gas’s contribution? What ordinary people want to know is why hasn’t there been a correlation between rising gases over the last decade and temperature?
[Response: We’ve discussed both those exact questions in recent posts. Oddly enough they didn’t get quite the traffic this one is getting. It’s all about me (thane) and A warming pause? – gavin]
For the fellow earlier who posted a link to a picture of a chart, labeled as from the Central England Temperature numbers
— it’d help if you’d give the source of the picture, as without the source it’s just a picture. We don’t know whose or what it means.
— I’d bet the red lines on that picture you have are drawn between extreme individual years selected to make the slope as steep as possible. That’s cherrypicking if so. Pointer to its source please?
Compare this discussion linked below and the charts presented with it.
Same data:
http://tamino.wordpress.com/2008/04/28/central-england-temperature/
But here the discussion looks properly at long time series data:
http://tamino.files.wordpress.com/2008/04/cetsmooth.jpg?w=500&h=391
Re#380: So everything rests on the GISStemp dataset as ,from what I am reading,there is no likelihood of getting the HADCRUT original raw data. Hence there is no verifiable 2nd dataset on which to base the serious forthcoming regulatory decisions. So, we need to validate the only available dataset and publish the results of the validation exercise so that the public can judge whether its sufficient for their govermnents’ to implement any proposed actions.
If anything what these emails show are honesty and they show humanity. They show people, scientists yes, but nonetheless people being people who frustrated by the fact that they know damn well that the professional deniers are are lying to the public. They demonstrate that climate scientists themselves believe what it is that they are telling the public about climate change. There’s no, “OMG, we have to shut up [so-and-so] skeptic because he’s going to spill the beans and expose our fraud”. If that were the case simply disallowing publication in a particular journal would hardly stop the exposure now would it? It’s, more like “This SOB is lying and we won’t be a part of it, and in fact we should do everything in our power to stop them using their positions to deceive people about such a critical issue in legitimate science publications.”
Who among us has not, in the privacy of their conversations with friends and family, said things about others that they would never dream of saying in public. Those self-righteous here who say, “hey, if you got nuthin ta hide you got nuthin ta fear from having your private conversations shown to the public” are simply apologists for Big Brotherism. If we have nothing to hide then we should have NO objections to having our every move, every word, even every thought recorded right? I doubt that anyone would truly want that. And I suspect that the skeptics are rightwingers who in other areas loudly decry the influence of “Big Government” in their private lives.
There should be no astonishment with the fact that few things in science just fall into our laps, that fianl published papers are more polished than rough drafts. That is the natural order of things from grade school on. Learning is usually a process of fits and starts, often a messy process not unlike making hamburger. Slowly, though, a consensus begins to emerge and people come together. No one agrees with every word but they all agree on the main idea.
People have to realize that the objective of the professional deniers is not to prove actual fraud, because they cannot. No, the goal is to create the APPEARANCE of fraud and thus doubt in the public mind. That’s all they think that they have to do. And all this just to keep the profits rolling into EXXON/Mobil as long as possible.
Comment by cm — 21 November 2009 @ 11:27 PM
“It amazes me is the how hard the CRU worked to hide their underlying data. If the data is right and matches the theories, it should be able to withstand any scrutiny. If the kooks analyze it wrongly, why not point out why they were wrong? Is there something to hide in the data?”
They aren’t hiding their data. In fact, they’ve given professional responses to reasonable requests. You should read the MannHouseReply.pdf in the bundle for a better idea.
The peer-reviewed data and results are available for review. That is all that is necessary to very or refute the science being done. Anything is NOT peer-reviewed. It is NOT verified, and therefore making it available or, worse, using it is not a good thing.
“These e-mails also make some of the writers look unethical. Even if they are not, if someone is thought to be unethical, it makes everything they say suspect. That is the problem with these disclosures. This seeming lack of ethics tends to devalue all of their other work.”
If you read through the 1000+ emails you’ll see that they aren’t hiding anything, nor are they being unethical. They’re getting irritated by skeptics using poor science. They’re getting irritated by a supposedly peer-reviewed journal ignoring peer-reviewers. They’re getting irritated by repeated (time wasting) requests for data that is already public or data that shouldn’t be made public (like emails marked as confidential, private code, data that is under third party agreements so can’t be released, etc.). Anything of any scientific value is made public as part of the peer-reviewed process.
Do not rely on the media or skeptic blogs for interpretations of the materials. You can get the files yourself and read through them. Make your own conclusions.
~X~
38 Jay says:
20 November 2009 at 1:54 PM
“My only questions now is…
I hear a lot about the FOIA and data that was being withheld that is now lost or destroyed. Is there an explanation or a reference to that which would answer what I have been hearing on the other end?
[Response: No data has been lost or destroyed. – gavin]”
This response from CRU explains why they cannot produce the original raw data behind their global temperature record:
“We are not in a position to supply data for a particular country not covered by the example agreements referred to earlier, as we have never had sufficient resources to keep track of the exact source of each individual monthly value. Since the 1980s, we have merged the data we have received into existing series or begun new ones, so it is impossible to say if all stations within a particular country or if all of an individual record should be freely available. Data storage availability in the 1980s meant that we were not able to keep the multiple sources for some sites, only the station series after adjustment for homogeneity issues. We, therefore, do not hold the original raw data but only the value-added (i.e. quality controlled and homogenized) data.”
[Response: Yup. If you want the very original hand-written records from individual stations, ask the National Met. Service in the relevant country, not the people who collate the homogenised records for use in tracking climate change. – gavin]
Peter Webster in #859 says “Data was made available to some of us and I am grateful for that as it has proven extremely important in trying to understand the 1935-45 warming. Perhaps a way around the fiscal issue is to state which data is precluded from being made available. I guess that is meta-data as well. But I think that we have to move on as openly as we can be.”
It appears that you have not followed the FOI events of last summer.
The CRU stated that, because they think they have confidentiality agreements with some data providers, but that they don’t know which, that they must refrain from releasing any data whatsoever.
In the November 12 reply to Steve McIntyre’s appeal on the rejection, they say that just because they made a mistake and gave you the data earlier doesn’t mean that they should repeat the mistake.
In other words, they have “corrected” their policy and if you asked today, you would not be sent the data you were sent early this year.
[Response: People seem to think that I’m somehow in charge..]
You seem to be in charge when you state for CRU that no data was ever deleted or ever will be and that so-and-so was just joking, and this guy didn’t really mean what he said.
What you are in charge of seems to vary at your convenience.
[Response: You confuse knowledge (easy to get), with being in charge (much harder). The reason I’m interacting here is, as the site says, to provide context that is very clearly missing in most of the discussion. I can do that because I know most of the people involved, what it is they are discussing and what is known about those issues outside of what is in the emails. This is a far cry from being the Head of Global Institute for Climate Science should such a thing even exist. – gavin]
I gave a public lecture last Friday evening titled: Global Warming: Separating Fact from Fiction that I have converted into a .PDF file. I have been told that the information is very user friendly so feel free to download and distribute to friends of yours that may be skeptical.
http://www2.sunysuffolk.edu/mandias/global_warming/fact-from-fiction.pdf