This is a continuation of the last thread which is getting a little unwieldy. The emails cover a 13 year period in which many things happened, and very few people are up to speed on some of the long-buried issues. So to save some time, I’ve pulled a few bits out of the comment thread that shed some light on some of the context which is missing in some of the discussion of various emails.
- Trenberth: You need to read his recent paper on quantifying the current changes in the Earth’s energy budget to realise why he is concerned about our inability currently to track small year-to-year variations in the radiative fluxes.
- Wigley: The concern with sea surface temperatures in the 1940s stems from the paper by Thompson et al (2007) which identified a spurious discontinuity in ocean temperatures. The impact of this has not yet been fully corrected for in the HadSST data set, but people still want to assess what impact it might have on any work that used the original data.
- Climate Research and peer-review: You should read about the issues from the editors (Claire Goodess, Hans von Storch) who resigned because of a breakdown of the peer review process at that journal, that came to light with the particularly egregious (and well-publicised) paper by Soon and Baliunas (2003). The publisher’s assessment is here.
Update: Pulling out some of the common points being raised in the comments.
- HARRY_read_me.txt. This is a 4 year-long work log of Ian (Harry) Harris who was working to upgrade the documentation, metadata and databases associated with the legacy CRU TS 2.1 product, which is not the same as the HadCRUT data (see Mitchell and Jones, 2003 for details). The CSU TS 3.0 is available now (via ClimateExplorer for instance), and so presumably the database problems got fixed. Anyone who has ever worked on constructing a database from dozens of individual, sometimes contradictory and inconsistently formatted datasets will share his evident frustration with how tedious that can be.
- “Redefine the peer-reviewed literature!” . Nobody actually gets to do that, and both papers discussed in that comment – McKitrick and Michaels (2004) and Kalnay and Cai (2003) were both cited and discussed in Chapter
2 of3 the IPCC AR4 report. As an aside, neither has stood the test of time. - “Declines” in the MXD record. This decline was
hiddenwritten up in Nature in 1998 where the authors suggested not using the post 1960 data. Their actual programs (in IDL script), unsurprisingly warn against using post 1960 data. Added: Note that the ‘hide the decline’ comment was made in 1999 – 10 years ago, and has no connection whatsoever to more recent instrumental records. - CRU data accessibility. From the date of the first FOI request to CRU (in 2007), it has been made abundantly clear that the main impediment to releasing the whole CRU archive is the small % of it that was given to CRU on the understanding it wouldn’t be passed on to third parties. Those restrictions are in place because of the originating organisations (the various National Met. Services) around the world and are not CRU’s to break. As of Nov 13, the response to the umpteenth FOI request for the same data met with exactly the same response. This is an unfortunate situation, and pressure should be brought to bear on the National Met Services to release CRU from that obligation. It is not however the fault of CRU. The vast majority of the data in the HadCRU records is publicly available from GHCN (v2.mean.Z).
- Suggestions that FOI-related material be deleted … are ill-advised even if not carried out. What is and is not responsive and deliverable to an FOI request is however a subject that it is very appropriate to discuss.
- Fudge factors (update) IDL code in the some of the attached files calculates and applies an artificial ‘fudge factor’ to the MXD proxies to artificially eliminate the ‘divergence pattern’. This was done for a set of experiments reported in this submitted 2004 draft by Osborn and colleagues but which was never published. Section 4.3 explains the rationale very clearly which was to test the sensitivity of the calibration of the MXD proxies should the divergence end up being anthropogenic. It has nothing to do with any temperature record, has not been used in any published reconstruction and is not the source of any hockey stick blade anywhere.
Further update: This comment from Halldór Björnsson of the Icelandic Met. Service goes right to the heart of the accessibility issue:
Re: CRU data accessibility.
National Meteorological Services (NMSs) have different rules on data exchange. The World Meteorological Organization (WMO) organizes the exchange of “basic data”, i.e. data that are needed for weather forecasts. For details on these see WMO resolution number 40 (see http://bit.ly/8jOjX1).
This document acknowledges that WMO member states can place restrictions on the dissemination of data to third parties “for reasons such as national laws or costs of production”. These restrictions are only supposed to apply to commercial use, the research and education community is supposed to have free access to all the data.
Now, for researchers this sounds open and fine. In practice it hasn’t proved to be so.
Most NMSs also can distribute all sorts of data that are classified as “additional data and products”. Restrictions can be placed on these. These special data and products (which can range from regular weather data from a specific station to maps of rain intensity based on satellite and radar data). Many nations do place restrictions on such data (see link for additional data on above WMO-40 webpage for details).
The reasons for restricting access is often commercial, NMSs are often required by law to have substantial income from commercial sources, in other cases it can be for national security reasons, but in many cases (in my experience) the reasons simply seem to be “because we can”.
What has this got to do with CRU? The data that CRU needs for their data base comes from entities that restrict access to much of their data. And even better, since the UK has submitted an exception for additional data, some nations that otherwise would provide data without question will not provide data to the UK. I know this from experience, since my nation (Iceland) did send in such conditions and for years I had problem getting certain data from the US.
The ideal, that all data should be free and open is unfortunately not adhered to by a large portion of the meteorological community. Probably only a small portion of the CRU data is “locked” but the end effect is that all their data becomes closed. It is not their fault, and I am sure that they dislike them as much as any other researcher who has tried to get access to all data from stations in region X in country Y.
These restrictions end up by wasting resources and hurting everyone. The research community (CRU included) and the public are the victims. If you don’t like it, write to you NMSs and urge them to open all their data.
I can update (further) this if there is demand. Please let me know in the comments, which, as always, should be substantive, non-insulting and on topic.
Comments continue here.
Gavin, Thanks for your prompt reply to my previous question (post 1075: RaymondT 22 Nov2009 at 10:45pm ) and congratulations for your excellent website. The paper by Kevin Trenberth (An imperative for climate change planning: tracking Earth’s global energy) has rekindled a question I have about the methodology used in climate modelling concerning the tuning of the radiative forcings used in history matching the temperature history. Dr Trenberth in his paper outlines a multitude of potential explanations for where the radiative forcing has gone in order to explain the stabilization of the surface temperature in recent years. This paper shows that we are just starting to understand natural variability. Mojib Latif in his power point presentation given at the WCC3 asks: “How much did internal decadal variability contribute to the warming during the recent decades ?
My question is then: How do we remove the natural variability beyond the ENSO time scale (i.e. on a multi-decadal scale) from the temperature record ?
[Response: Great question. And if you have a good answer, write it up and submit it to Science straight away! Joking aside, it is a hard problem. There are multi-decadal variations in both the internal variability and the forcings. Our measurements of the historical variations don’t extend back over more than one or two multi-decadal cycles so it is almost impossible to do a statistical separation. Therefore you are left with a model-based approach, but there is quite a lot of variation in the model realisations at this frequency, and so it is not particularly definitive. One could theoretically extend the instrumental data with the proxy records – at least for a couple of centuries – to get some bounds on the magnitude though, and that indicates that it is relatively small in the global mean, but larger in certain regions. But of course, that comes with its own issues. – gavin]
the really interesting bits appear to be the programmers’ notes in HARRY_READ_ME.TXT. No idea if they are real, but a summary here appears to be damning.
Excerpt:
Here, the expected 1990-2003 period is MISSING – so the correlations aren’t so hot! Yet the WMO codes and station names /locations are identical (or close). What the hell is supposed to happen here? Oh yeah – there is no ‘supposed’, I can make it up. So I have :-)
..You can’t imagine what this has cost me – to actually allow the operator to assign false WMO codes!! But what else is there in such situations? Especially when dealing with a ‘Master’ database of dubious provenance (which, er, they all are and always will be)…
[Response: Read the full transcript starting a page or five up. The issue he/she is working with is that different databases had different metadata and had been updated differently. He is then trying to reconcile the data as best he can with “reference to the good databases”. These things happen. The issue is whether he was able to fix this and reconcile the issues. Note too this is a reference to the CRU TS 3.0 dataset which is not the same as the more commonly used HadCRU data – it includes different stations, and does much more interpolation. Errors in databases are inevitable (try working with one over 30 years), but these work records indicate that there is a thorough and ongoing attempt to update and check them. What would you have them do instead? – gavin]
The why and how of the information release is disturbing. The abuse about skeptics and manipulation over contrary publications is human – if sometimes verging on the infantile. But it is unsurprising when folk who hold a strong opinion position feel challenged – regardless of whether they are right or wrong.
These are not the issues for me. The law is. The people at CRU are public servants, and there is no case for their work correspondence to be “private”. In UK, government officials’ correspondence is “in the public domain” from the moment it is signed off.
The FOI in UK is a very strong law and non-compliance has to be based on the national interest (security). I doubt whether climate science can invoke that as a cause for exemption. Even the University of East Anglia (UEA – home of CRU) states on its website:
“What does the Freedom of Information Act mean for the University?
There are two main obligations imposed by the Act on the University:
1. That UEA must maintain a Publication Scheme, which lists the types and format of information the University routinely provides to the public
2. That any individual making a request for information is entitled to be informed in writing by the University whether or not the University holds the information, and if it does to have the information communicated to them within the specified time limit of 20 working days.”
The e-mails show that:
– there was encouragement to stop people being told that UK has a FOI;
– there was instruction to delete e-mails, and;
– the UEA FOI administrator was persuaded by Prof Jones not to comply with a FOI request.
These appear to show clear intent to pervert the law. The UK Information Commissioner has been asked to investigate, and if he finds against CRU, UEA or an individual, they can be heavily fined. This is what may cause the lasting damage and could be the mud that sticks.
So it is all fine and dandy decrying tha hacker/leaker. If Jones or his employers are found guilty of a crime, that will be the real issue.
Quick note of support from an ordinary member of the public.
As someone who has been subjected to commercial advertising for most of my life and seen how politicians distort facts, in my opinion you don’t need to worry too much about this affair. Us ordinary citizens have a gut feel for when a ruler or merchant lies to us and they do that often. Always when I see a scientist say something, the sentence has a very precise meaning. It almost seems as if it can be either true or false and the person who says it has made sure of the facts before the time. To me it shows the integrity of the speaker. For trying to make a difference, you are my heroes. Good luck in your battle of defence against those who would attack your work.
Re #79: Speaking of tobacco, the Washington Post has a very nice interview with Spencer Weart, who points out just how weird this is:
He goes on to emphasize: and compares this to the recent Bear Sterns trial:
There seems to be some doubt about the timeline of events that led to the emails hack. For clarification and to save me going through this again, this is a summary of my knowledge of the topic. At around
6.20am7.20am (EST) Nov 17th, somebody hacked into the RC server from an IP address associated with a computer somewhere in Turkey, disabled access from the legitimate users, and uploaded a file FOIA.zip to our server. They then created a draft post that would have been posted announcing the data to the world that was identical in content of the comment posted on The Air Vent later that day. They were intercepted before this could be posted on the blog. This archive appears to be identical to the one posted on the Russian server except for the name change. Curiously, and unnoticed by anyone else so far, the first comment posted on this subject was not at the Air Vent, but actually at ClimateAudit (comment 49 on a thread related to stripbark trees, dated Nov 17 5.24am (Central Time I think)). The username of the commenter was linked to the FOIA.zip file at realclimate.org. Four downloads occurred from that link while the file was still there (it no longer is).The use of a turkish computer would seem to imply that this upload and hack was not solely a whistleblower act, but one that involved more sophisticated knowledge. If SM or JeffID want to share the IPs associated with the comments on their sites, I’ll be happy to post the IP address that was used to compromise RC.
has realclimate done an article on the divergence problem and it’s relevance to paleo reconstructions? I have found separate points on it, but not an overall article, although I may have missed it. I ask because there are a lot of arguments around to the effect that the problem makes paleo temperature reconstructions impossible, but these arguments are given without context to published science and sound a little extreme. An article laying out some scientific context about the subject would be cool.
[Response: Try this review (page 10) – gavin]
Gavin, lets be honest.
You are obviously very well connected to the people and interests of CRU. While on this blog you claim to only filter noise, you until recently have used your position of moderator to filter most differences of opinion. While I understand how as a blogger you would want to push your point of view, I raise issue with the fact that this website is run by Climate Scientists. WUWT and other don’t claim to be scientists, although many contributors are. Therefor it is less important that they remain unbiased. The efforts here at RC are meant to sway the public and involve activism rather than pure science. The reasons for this are numerous including involvement with Fenton Communications. I am not trying to be insulting, I am just frustrated when those whom you trust to give unbiased information, much like doctors, have vested interests in the outcome of their research and experiments. I understand that you and the other climate scientists are no different than any of us in your humanity, we all look a little rough under a microscope.
[Response: We are unbiased when it comes to the science. We are not when it comes to politically-motivated nonsense that passes for science in forums where people seem to be most frequently motivated by wishful thinking. And Fenton did our press release.. ooh… 5 years ago. We’ve never talked to them since. – gavin]
#53 Ray Ladbury says:
Ray, no doubt you’ve read about the Dunning-Kruger effect, “a cognitive bias in which ‘people reach erroneous conclusions and make unfortunate choices but their incompetence robs them of the metacognitive ability to realize it.'” It’s a hypothesis with great explanatory power.
The E mails make fascinating reading, especially:
0889554019
[Response: Really? It is likely a draft of this, rather dull, report. – gavin]
“1. There needs to be a clear statement of why the instrumental and proxy data are shown on the same graph. The issue of why we dont show the proxy data for the last few decades ( they dont show continued warming) but assume that they are valid for early warm periods needs to be explained.”
– 1150923423.txt
Can anyone please point me to a paper/discussion on why this data is not valid now, but is valid for early warm periods? Thanks.
Saw your response on #17, makes sense
Are you going to post the updated code to dispel this mess?
[Response: Not my code or project. Ask ‘Harry’ ;) – gavin]
What do you guys think, should I tell him?
Is this so-called “hacking” being investigated by the authorities?
It’s troubling that realclimate seems to be single-mindedly defending all the data and emails from the CRU hack as being “no big deal… all of this happens all the time”. I suspect more credence would derive from a bit less-obvious defense of the status quo on the AGW side. The fact that data has not been released, peer-review has been affected, and so on, should result in some calls from Real Climate for oversight and not so much “nothing here folks… move on”.
Is publicly posting IP addresses a wise choice here? Assuming there is some sort of investigation here into what happened, you might want to consult with whoever is doing that investigation.
On why not all data is public: this was raised about HadCRUT after a bit of a tussle with deniers. CRU had 2 reasons for not releasing all raw data: some of it was paid for with limited use licenses, and data processed in the 1980s could not be kept because of the cost of disks. At the time that issue was doing the rounds, I calculated what it would have cost in 1980 to store one year of climate station raw data and the amount was in the millions of dollars. Hard to believe today when you can buy a 1TB USB disk for next to nothing, but this is the effect of an ongoing exponential drop in the price of computer technology; my bottom of the range cell phone has more computing power than a 1975 $5-million Cray.
No serious nature-based science project at the cutting edge of available data volumes can afford to store all its raw data indefinitely. Today we could store all the data from weather stations, but we should also be pushing the envelope by including more data from more diverse sources. I would be surprised if absolutely all that data were stored long-term, professionally curated and easy to find. I’ve certainly run into problems in research in areas less in the public eye than climate science where data isn’t where it’s supposed to be, the wrong version was uploaded to the repository, everyone doing related work has a different standard for data layout or the software that used the data is now in a new version, and no longer works to reproduce the reported result. Despite all this, good discoveries are made and science advances. Why? Because scientists are for the most part jealous of their reputations and highly competitive. If they make a major mistake, most try to make good before someone else spots it. There are exceptions of course, many of which are found in the climate denial camp like those who have no compunctions about flatly contradicting themselves or talking complete rubbish (e.g. Bob Carter recently in The Australian said we should be planning for rising sea level based on local tide gauges rather than global averages: excuse me Bob, but didn’t you notice that the oceans are all connected)?
Sadly, we live in a world where science funding is constrained. Presumably the people who are demanding that every scrap of raw data be preserved, easy to find, well documented and free to access are utopian social[not spam]ists who believe that the sciences should be funded to whatever degree necessary to achieve good results, but the rest of us live in the real world where we have to live with these imperfections. Good on Gavin and those who do go the extra mile to make their data accessible.
In Australia, the denial camp for some time has been demanding a Royal Commission (an enquiry with wide powers to assemble evidence) into the alleged manipulations of climate scientists. I’ve countered by proposing a similar investigation into their campaign, at which point they fall very silent.
What we are dealing with here is developing all the characteristics of a war, without the shooting. Why should we be surprised? The rational response to AGW is to move rapidly to renewable energy, a hard but not impossible project even with currently known technologies. The fossil fuel industry and their acolytes don’t want this to happen any time soon, and are willing to risk the biosphere for a few more years of profits. The sorry thing is that if we reach a point where extreme measures are called for, they will go out of business very fast, rather than slowly over decades – and this will most likely be at the point where catastrophe is undeniable. This isn’t like the tobacco companies’ war on health, where it was always possible to find a new gullible group once old markets were tapped out and regulated to the extent of making expansion hard. This is all or nothing, because the fossil fuel industry wants it that way. If some magic pudding solution (unknown to us and certainly unknown to them, otherwise they would have produced it by now) saves us from extreme climate change, they keep on as before. If not, they go down with the rest of us.
The thing they seem to forget is that they don’t have another planet any more than the rest of us do.
UEA should share all the data-sets? Yes that would be a good goal. But not now!
We don’t know the cause of the break-in. But
whatever the answer, appeasing the anti-scientific campaign now, would be seen as the success of a highly aggressive campaign which assumes that it is entitled to do anything and tell any lie. What would the next stunt be ? I don’t think this gang should appear to be rewarded for its misdeeds or it will just escalate them. Any group of people might be discredited this way by harrassment, invasion of privacy, hyping and misrepresenting the revelations.
This morning BBC’s Radio 4 interviewed Nigel Lawson and Prof. Robert Watson over the issue. The former argued for a public enquiry to avoid British science being tainted! I think we should first have an enquiry into the lack of proper science reporting on Climate issues by Channel 4 and BBC 2. For years, Lawson , who has been trying to emulate James Inhofe, has been campaigning for the abolition of the IPCC with a knowledge of climatology based on short term trends with no value and the hockey stick libel (see RC and Open Mind on both issues).
Prof. Watson was quite effective in the short time available but of course had to give priority to defending the UEA so that the short term temperature behaviour got rather too little attention. This is a bad omen. The contrarians are using this ad-hom nonsense to flood the media so that there is no time to deal with their climate convusionism… just as planned.
The hack is a shame, and so are all these “skeptics” comments. I just hope it will appear that the hackers were paid by some wuwt or morano…
The worse is that none of these stupids will recognize they’re wrong when the temps go higher than 1998 in 201X.
Anyway, I hope this stupid noise will stop soon. It seems another occurrence of their “delay” strategy. Isn’t this a bit too close from Copenhagen ? Will we see other stolen emails, from, say Jim Hansen, in the following days ?
Hope you can soon go back to your terrific real work we badly need !
Hi Gavin,
I hate to bother you with more questions about the attack, but perhaps you might have a few more details:
1. Who initially discovered the hacking and prevented Real Climate from being hijacked?
2. Did this same person identify the offending IP as originating in Turkey?
3. Are you aware of any measures being taken presently to track down the offender?
4. It sounds like a fairly sophisticated attack; are you confident the hole has been plugged by IT or are the methods used by the hacker still obscure?
5. Is there any additional reason–besides the obvious–to suspect the timing of the attack might be suspicious?
[Response: Me. Me (via ARIN Whois for what that’s worth). No. Yes… errr… maybe. Not really (though see the BBC blog by Hudson). – gavin]
There are claims from spokespeople defending the criminal acts in this case.
Several have suggested that these criminal actions were the work of whistleblowers. These spokespeople suggest in fact, that the hacked file was prepared under a FOI (Freedom of Information) directive. If that is the case, then it opens up some intriguing possibilities:
A whistleblower is someone in an organization, that realizes that it is hiding or concealing information from legal authorities and decides to go directly to the legal authorities. Or alternatively, the whistleblower realizes the organization is publicly saying one thing, and internally has information saying significantly different. The second case is damaging, if opponents or critics don’t have access to the information.
In this case, apparently the emails were contained in a file that was assembled for a FOI demand in Britain. The British authorities apparently turned down the request, likely because the information wasn’t material to use as requested by the applicants, i.e. it isn’t useful for scientific purposes.
Surely the attorneys for the applicants saw this information during discovery, so agents for the applicants did have private access to the emails. But the British courts said no to public disclosure.
It is possible that in order to get around the British courts, the emails were \”hacked\” and used for cyber sabotage. If so, any legal defense of these criminal acts disappears. And in no way, can these acts be considered whistleblowing. Only if the information was illegally withheld from the British authorities, could whistleblowing be considered justified.
It appears possible, that the applicants didn’t like the decision by the British authorities, and decided to do an illegal end run. The possibility exists that court documents have been illegally disclosed.
It also interesting to note, that some of the spokespeople defending these criminal acts, received the hacked files either before or contemporaneously with the cyber sabotage attack on RealClimate.
George Monbiot at the Guardian http://www.guardian.co.uk/commentisfree/cif-green/2009/nov/23/global-warming-leaked-email-climate-scientists is more or less suggesting that the emails change everything, and calling for the resignation of Jones, and apologising to his readers. I have taken him to task – others might like to have a quiet word to him too. The thread, is, not surprisingly, full of triumphant denialists, including one, bizarrely (or perhaps not) saying that not only is AGW all a hoax, but so was the evidence for tobacco causing harm, and all of that nonsense about anti-smoking campaigns should be stopped. We are in fantasy world once again.
The Wall Street Journal put an interesting spin on the editors of Climate Research resignation in protest of the publication of the Soon & Baliunas paper. “The result: Messrs. Soon and Baliunas were treated as heretics and
six editors at Climate Research were made to resign.” Making it appear that editors were forced to resign because they supported the paper instead of resigning to protest that such crap got published over the objections of peer reviewers. Of course the “true skeptics” buy this kind of subtle misinformation hook, line, & sinker.
Interesting, Gavin, have you double checked that this IP isn’t a proxy server?
I can’t see how this information about the RC hack, which you already vaguely mentioned, squares with the screeching about how this was a legitimate whistle blowing by a concerned FOI officer. This was a malicious attack perpetrated by someone aiming to cause humiliation.
Where does coal come from? Does coal contain carbon? Where did the carbon in the coal come from? Where was the carbon before it was in the coal?
“Climate scientists have suffered a huge black eye.”
Ed — 23 November 2009 @ 12:34 PM
You mean they’ve been mugged? I couldn’t agree more.
George Monbiot has just written that Phil Jones should resign for unscientific behaviour, and he apologises to his readers because “I was too trusting of some of those who provided the evidence I championed. I would have been a better journalist if I had investigated their claims more closely.”
(He then engages is some of the most leaden satire I have read in quite a while). I think he is jumping the gun and I’ve expressed this in the comments (PhilClarke).
BBC 2 Newsnight just came on with this as its lead item. Poor and shallow. Bob Watson was confronted with Fred Singer who made the same demands as Nigel Lawson, without any reservations, and managed to combine it with an attempt to rubbish MMGW (he does not want the public to know there is such a thing as science , only data-sets).
Bob Watson said that there were two other data-sets which showed similar results. The public will not know about Singer’s record as lobbyist as a campaigner for the tobacco and CFC manufacturers. The item was introduced by its BBC science correspondent Susun Watts and Jeremy Paxman, the main presenter. On this occasion Jeremy seemed better clued up (or less timid?) than Susan.
Someone said that one climate model forecast no warming for a couple of decades ; isn’t this just a continuation of the misunderstanding over Latif’s talk? It was not contradicted. What has it to do with the emails?
Since you equate climate science with intelligent design and other pseudo-science it seems clear that nothing will convince you. Your post also makes it clear that it’s the political implications that lead you to the position that it must be pseudo-science.
Meanwhile, whether you believe it or not …
1. Burning fossil fuels adds CO2 to the atmosphere
2. CO2 absorbs IR
If it were possible to prove either of these statements wrong, it would’ve been done long ago. The truth of both makes it clear that our activities will warm the planet, the only question is, “how much?”.
And, of course, going back to this absolutist statement:
This is a strawman because no scientist in the field would make such an absolutist statement. Human emissions of CO2 and other changes to the planet *contribute* to climate change. Obviously other factors like changes in the sun’s output, orbital factors, etc also contribute.
Gavin (125) — Unfortunately one cannot assume just because the IP address used was one assigned to a computer in Turkey that in fact that (or any other specific) computer was actually part of the attack. The ability to “spoof” IP addresses is rather widely known and attempting to track down the computers actually used is rather a nightmare and best left to speciialists.
Instead, I have a quaetion: if the attempt on RealClimate a crime in the locality of the server hosting RealClimate? If so…
I have to admit that the skeptic side is extremely open with their findings and methods … of course, this is because they have none to speak of, making the task easier …
I’d never heard of RealClimate until today, when I was reading through posts on the CBC (Canadian Broadcasting Corporation). The e-mail hack was already being jumped on by “skeptics”, and someone referenced this site for an explanation. I’m glad he/she did.
I’m literally bored to tears with the ongoing “debate” over AGW. How there isn’t still a global debate over whether smoking causes lung cancer is beyond me, given the lengths to which the “skeptics” will go.
Anyway, please keep up the good work, gavin et al. It is a powerful thing to see someone SMART deflecting one moronic comment after another. I wish I could do as well on the CBC!
Cheers.
Re: 90
“It is exactly this attitude, that we must save the world, that has caused these people to forget their scientific training, so that the end justifies the means, and science violations occur. You are not helping anybody. Keep your secret knowledge of impending doom to yourself, and let some real science ensue.”
I’m sorry, I think I’ve been misinterpreted, which is not surprising. I don’t have any knowledge that anybody else lacks — I do think that the question of what is going on with our climate is the most important question facing our species at the moment. As my background is in science, I assumed it would go without saying that the greatest service a scientist can provide to the species is to provide accurate information about the physical world. Apparently it does not go without saying, and needs to be stated explicitly.
I don’t think any scientist would be doing the world any favors by distorting data or misrepresenting science, and I get the impression Gavin understands that.
I did not intend to imply that scientists should allow these considerations to influence their judgment on scientific issues — rather, quite the opposite, they must hold themselves to a much stricter intellectual standard.
My point, and I admit I put it in a rather eccentric manner (it was quite late at the time) was simply that, given the extremely high stakes of this matter, scientists working in this field can expect to be held to a higher standard of behavior than others, and maybe that’s a good thing. My intent was to provide encouragement and inspiration to continue this vitally important work despite the stiff opposition they are currently facing.
David Appel:
http://davidappell.blogspot.com/2009/11/cru-hack.html
—–excerpt follows—–
I saw that my name popped up in a few places in the emails. I don’t have much to say about them (which is neither a confirmation or a denial of whatever was claimed) except I do not at all recall ever threatening Sonia Boehmer-Christiansen (editor of Energy & Environment) with “litigation” (mail file 1068239573.txt) I can’t imagine doing that as a journalist, and even if I wanted to I don’t know what I’d threaten, unless it was to tell her I was going to file a FOIA, which I certainly did not do in this case since (a) I didn’t think that that any of her internal journal correspondence was subject to a FOIA, as it was unrelated any government function, and (b) I don’t even know if an American can file whatever the equivalent of a FOIA is in Britain. I do, though, plead guilty to having called her multiple times, whether it annoyed her or not. That’s my job.
—–
1089, That’s OK Hank, I wouldn’t say a word you peak at WUWT. However the point is whose curves are more realistic? Whose shows the global temperature flattening over the past 8-10 years, as shown on the figure below?
http://www.imagenerd.com/uploads/climate4u-lt-temps-Ljbug.gif
The physical basis- causality – for AGW was first proposed about 150 years ago, and the physics for CO2s direct contribution more or less fully worked about by the 1950s.
The correlation – the signal separating from the noise – followed.
In other words, warming due to increasing CO2 in the atmosphere was a prediction by science which observations have borne out.
Modern climate science has made other predictions, i.e. stratospheric cooling, which observations have also borne out.
Where are you getting your [mis]information?
Hi, been reading more and more global warming stuff lately. Came to this blog from a link from the Guardian newspaper. Nice site. Just thought I would post my two pennies. I love the subject of global warming; it must be an absolute nightmare for policy makers who are being asked to decide to make policy that will be seriously damaging to the quality of life enjoyed by those in the developed world who elected them based on statistical science. I am left rather bemused by the focus on “proving” that people are responsible for global warming. Surely this adds very little to the debate? Obviously no computer model will be able to (I work in computer modelling for electronics) “prove” global warming (p.s I’m not saying that their is no value in climate science – I’m very impressed by it). Therefore why bother to argue with those who hold a sceptical position on this point? Surely it is enough for policy makers to say that we are a species heavily dependent on a particular ecosystem and we have decent evidence that our activities might be altering that ecosystem. No one (to my knowledge) has been able to convincingly show that human activity does not affect the climate therefore we should stop (or try to reduce) harmful activities until such time that either someone is able to show this or until our level of technology has advanced to the point where we could mitigate any effects? Why do we not hear this sort of argument more often? I think it is much more convincing than hearing the latest confidence factors of the latest climate models quoted or the percentage of leading scientists currently convinced etc… This sort of argument is used to justify other difficult to quantify policies. For example (not a very good example probably) it would be very difficult to “prove” that we will be invaded in the future but we have convincing evidence to justify that we might be – and so we maintain the armed forces? Ben.
hy folks,
why didn`t you make all of the date and codes free open to anybody?
if it is the turn of global meanings, it should be, you do not think like me?
sorry, but here is much to explain and i do not see a positiv reaction by those peoples involved in all these thinks. why are you defending? what ist to defend?
On the first issue I am deeply disappointed to discover this “divergence problem”, if tree rings can’t record the recent climate warming how can we assume they would have recorded warming events in the historical record? If the data has been over-hyped then it needs to be admitted
[Response: If you want a fair assessment of what the real issues are and what is being done to deal with the imperfections that exist in any idea about what happened in past climate, read the relevant chapter in the IPCC report. The ‘divergence problem’ doesn’t affect every tree ring series and many attempts have been made to see how far you can get without using tree rings at all. – gavin]
—————————-
Gavin, if I may ask a follow up question. I don’t understand how correlated, reliable temperature proxies can be gathered from Tree-rings, regardless of where they are in the world. My (probably limited) understanding of what affects tree ring growth can include a combination of Sun, Cloud, Wet, Dry, Shade, Not Shaded, Forest Fire, Nutrients Supply etc etc. Given this, why would we use tree rings as a temperature proxy? Surely with the influences above, it would impossible to know what the climate is like year on year for a thousand years. (Was it Sunny and Wet, Sunny and Dry, Sunny, Dry and with a forest fire that year?)
[Response: All of the above. But the use of tree-rings for temperature means looking for places and situations where temperature is the dominant control. See here for instance. There is also a book by Fritts that is worth googling. – gavin]
Tim Smith
23 November 2009 at 1:49 PM
If you haven’t found them, I think you have never looked for them. The scientific publications underpinning the AGW theory are public and for all to see and falsify. What have you done to find those facts? You could start with this one.
Good luck.
Naw, the news is already out: Jeff ID thinks that Ball and Watts are credible, insightful people.
[removed to prevent confusion]
It’s troubling that you’ve missed the fact that gavin has, more than once, described Jones’ e-mail suggesting people delete e-mail as “ill-advised”.
I’m still waiting for a rational explanation as to why researchers should release data they’ve only gotten access to by signing agreements stating they wouldn’t release it.
I’m still waiting for a rational explanation as to why, if researcher A says “I can’t release data from B”, denialists just don’t get off their lazy ass and ask for data from “B” rather than file FOI requests trying to force researcher A to violate the agreement entered into. And then why the FOI filer bitches when the FOI compliance officer says “sorry, we can’t do that”.
Explain it to me in baby-steps so I can understand, please.
Hi
As a fellow scientist I would be horrified to find my email account had been hacked. I have no doubt I’d be sacked forthwith. We often email internally regarding data etc and how best to present data. Even sending GM cells to other labs require a group meeting to discuss the potential political implications of sending material to potential competitor collaborators.
However badly I feel for fellow scientists this is a lesson to us all. Do not ignore FOI requests. This is where I fear it will all ends in tears. This is no longer about science. When the government become involved heads will roll. It will get the headlines off their corruption and point it to the ‘mad’ scientists. The people involved have failed to recognise they are nothing but political puppets regarding climate change.
This will become too political and the sceintists involved are finished. See the writing on the wall and walk away. You know what will happen if you don’t.
Goodluck.
BTW: to the PhD student asking if he has a future in science after this mess…..if he needs to ask….that’s a good sign. Asking questions is what science is all about. The world goes on, and is not about to end because of this debarcle.
If SM or JeffID want to share the IPs associated with the comments on their sites, I’ll be happy to post the IP address that was used to compromise RC.
Comment by gavin — 23 November 2009 @ 4:00 PM
—————————-
Gavin, I would suggest that in the spirit of co-operation, you should email JeffID and Steve Mc with the IP details. I am sure they will do the honarable thing and respond in kind. At times like this, sometimes you need to trust that the right things will happen.
#
What do you guys think, should I tell him?
Comment by jeff id — 23 November 2009 @ 4:58 PM
—————————–
What do you have to lose? I would do this privately even though we know comments are moderated here. (Sorry Gavin!)
Hi, I’m new to all this and find the controversy fascinating. I downloaded the files and inspected them (what I could understand, anyway) myself, since I don’t like to have anyone else telling me what to think.
One thing I am curious about is the file 1256760240.txt, which contains two emails from a Dr. Don Keiller. He asks six questions in the two emails.
Can you please tell me if his emails were responded to? If so, may we see the response? If not, why not?
Thanks,
David
[Response: You know as much as we do. And we aren’t privvy to the rest of everyone’s emails. However, the answers to his questions are probably addressed in the pages that Briffa put up in response to the Yamal brouhaha. The cru server is down but the Google cache still has them: google “briffa yamal site:uea.ac.uk”. – gavin]
I read Hudson as meaning he recognised an email thread that he was party to back in October in the released archive, rather than seeing the whole thing then. But the man is not the most lucid journalist in the world so I could be mistaken.
I don’t know Gavin, but it’s dubious even if it’s not illegal.
Meanwhile, WUWT is demanding firm evidence that RC was hacked.
Hi Gavin,
First, kudos to you for this marathon, I understand it must not be fun, and appreciate the huge effort.
I have been a programmer for 20 years plus, but like most people looking at this stuff not a climate expert in any way. That said, I do understand working with data and the daily travails of code massage. A lot of things causing arm waving are just normal day in the life stuff imo, but I do have two questions.
1) Just to verify my assessment of the fortan code, it seems that through out the code there is a cutoff of about 1960 (not always exactly that) where proxy data is replaced, weighted, or blended with other more accurate measurements due to the proxy data not matching a known signal. Some of the comments use unfortunate language in retrospect, but people should try to make comments clear in any case so I have no issue with those. So it isn’t a comment issue, just I want to be sure that I understand correctly that the pre-1960 and post-1960 data is coming/influenced from two different sets… is that a fair assessment?
2) The Harry file seems normal enough (not best practices I’m sure, but the poor guy had his work cut out for himself with that data, ugh! Hat tip to him). It seems to have started as just improving existing code, but for most of the remainder been a log of creating the 3.0 datasets and code (is that correct?). It seemed to me in general the new set isn’t considered correct until is gives a close match to the old set. Was that the goal (and I understand that can be a legitimate goal), or was 3.0 meant to be a second set of data/code to compare and verify the first?
Thanks very much for your time,
Robin
[Response: The 1960 issue only applies to a single reconstruction (Briffa et al, 1998), but not more generally. In the WMO figure, pre 1960 was proxy, post 1960 was observed, and the smoothing was done assuming they were coherent. For your point 2), there is no reason why TS 2.1 should be the same as TS 3.0 – new data, fixes etc. will all make a difference. Obviously you would want to know what the reasons for the differences were. – gavin]