This is a continuation of the last thread which is getting a little unwieldy. The emails cover a 13 year period in which many things happened, and very few people are up to speed on some of the long-buried issues. So to save some time, I’ve pulled a few bits out of the comment thread that shed some light on some of the context which is missing in some of the discussion of various emails.
- Trenberth: You need to read his recent paper on quantifying the current changes in the Earth’s energy budget to realise why he is concerned about our inability currently to track small year-to-year variations in the radiative fluxes.
- Wigley: The concern with sea surface temperatures in the 1940s stems from the paper by Thompson et al (2007) which identified a spurious discontinuity in ocean temperatures. The impact of this has not yet been fully corrected for in the HadSST data set, but people still want to assess what impact it might have on any work that used the original data.
- Climate Research and peer-review: You should read about the issues from the editors (Claire Goodess, Hans von Storch) who resigned because of a breakdown of the peer review process at that journal, that came to light with the particularly egregious (and well-publicised) paper by Soon and Baliunas (2003). The publisher’s assessment is here.
Update: Pulling out some of the common points being raised in the comments.
- HARRY_read_me.txt. This is a 4 year-long work log of Ian (Harry) Harris who was working to upgrade the documentation, metadata and databases associated with the legacy CRU TS 2.1 product, which is not the same as the HadCRUT data (see Mitchell and Jones, 2003 for details). The CSU TS 3.0 is available now (via ClimateExplorer for instance), and so presumably the database problems got fixed. Anyone who has ever worked on constructing a database from dozens of individual, sometimes contradictory and inconsistently formatted datasets will share his evident frustration with how tedious that can be.
- “Redefine the peer-reviewed literature!” . Nobody actually gets to do that, and both papers discussed in that comment – McKitrick and Michaels (2004) and Kalnay and Cai (2003) were both cited and discussed in Chapter
2 of3 the IPCC AR4 report. As an aside, neither has stood the test of time. - “Declines” in the MXD record. This decline was
hiddenwritten up in Nature in 1998 where the authors suggested not using the post 1960 data. Their actual programs (in IDL script), unsurprisingly warn against using post 1960 data. Added: Note that the ‘hide the decline’ comment was made in 1999 – 10 years ago, and has no connection whatsoever to more recent instrumental records. - CRU data accessibility. From the date of the first FOI request to CRU (in 2007), it has been made abundantly clear that the main impediment to releasing the whole CRU archive is the small % of it that was given to CRU on the understanding it wouldn’t be passed on to third parties. Those restrictions are in place because of the originating organisations (the various National Met. Services) around the world and are not CRU’s to break. As of Nov 13, the response to the umpteenth FOI request for the same data met with exactly the same response. This is an unfortunate situation, and pressure should be brought to bear on the National Met Services to release CRU from that obligation. It is not however the fault of CRU. The vast majority of the data in the HadCRU records is publicly available from GHCN (v2.mean.Z).
- Suggestions that FOI-related material be deleted … are ill-advised even if not carried out. What is and is not responsive and deliverable to an FOI request is however a subject that it is very appropriate to discuss.
- Fudge factors (update) IDL code in the some of the attached files calculates and applies an artificial ‘fudge factor’ to the MXD proxies to artificially eliminate the ‘divergence pattern’. This was done for a set of experiments reported in this submitted 2004 draft by Osborn and colleagues but which was never published. Section 4.3 explains the rationale very clearly which was to test the sensitivity of the calibration of the MXD proxies should the divergence end up being anthropogenic. It has nothing to do with any temperature record, has not been used in any published reconstruction and is not the source of any hockey stick blade anywhere.
Further update: This comment from Halldór Björnsson of the Icelandic Met. Service goes right to the heart of the accessibility issue:
Re: CRU data accessibility.
National Meteorological Services (NMSs) have different rules on data exchange. The World Meteorological Organization (WMO) organizes the exchange of “basic data”, i.e. data that are needed for weather forecasts. For details on these see WMO resolution number 40 (see http://bit.ly/8jOjX1).
This document acknowledges that WMO member states can place restrictions on the dissemination of data to third parties “for reasons such as national laws or costs of production”. These restrictions are only supposed to apply to commercial use, the research and education community is supposed to have free access to all the data.
Now, for researchers this sounds open and fine. In practice it hasn’t proved to be so.
Most NMSs also can distribute all sorts of data that are classified as “additional data and products”. Restrictions can be placed on these. These special data and products (which can range from regular weather data from a specific station to maps of rain intensity based on satellite and radar data). Many nations do place restrictions on such data (see link for additional data on above WMO-40 webpage for details).
The reasons for restricting access is often commercial, NMSs are often required by law to have substantial income from commercial sources, in other cases it can be for national security reasons, but in many cases (in my experience) the reasons simply seem to be “because we can”.
What has this got to do with CRU? The data that CRU needs for their data base comes from entities that restrict access to much of their data. And even better, since the UK has submitted an exception for additional data, some nations that otherwise would provide data without question will not provide data to the UK. I know this from experience, since my nation (Iceland) did send in such conditions and for years I had problem getting certain data from the US.
The ideal, that all data should be free and open is unfortunately not adhered to by a large portion of the meteorological community. Probably only a small portion of the CRU data is “locked” but the end effect is that all their data becomes closed. It is not their fault, and I am sure that they dislike them as much as any other researcher who has tried to get access to all data from stations in region X in country Y.
These restrictions end up by wasting resources and hurting everyone. The research community (CRU included) and the public are the victims. If you don’t like it, write to you NMSs and urge them to open all their data.
I can update (further) this if there is demand. Please let me know in the comments, which, as always, should be substantive, non-insulting and on topic.
Comments continue here.
Well according to the Nature ‘Climate Feedback’ blog, he and his associates submitted 58 FOI requests over a 6 day period, that included a weekend. So that’s 14 requests a day or about 2 an hour. This is not science, this is spam!
Steve can easily be shut up by setting up a central repository for all the code and data used in climate modeling. Someone mentioned sourceforge before, that would be an excellent idea (alas climateforge.org has been taken).
Hosting of the data can be done on Amazon S3. It costs money but not that much.
More like one every two hours rather than two an hour but still, you’re right.
To 996 Sean: Thank you so much for the link to the licensing agreement.
My favourite excerpts:
“They may not be used for any other projects unless specific prior permission has been obtained in writing from the UKMO by a NERC Data Centre. Note that this applies even for other bona fide academic work.”
“Data sets must not be passed on to third parties under any circumstances. Any scientist requiring data which happens to have been supplied already to someone else, even within the same institute or programme of research, must first approach one of the NERC Data Centres, who have agreed to maintain records of data users for UKMO.”
“Once the project work using the data has been completed, copies of the datasets and software held by the end user should be deleted, unless permission has been obtained for them to be retained for some alternative use.”
“NERC recognises that some data holdings supplied by UKMO under the arrangements are commercially valuable : the recipients of data are under an obligation to respect the terms and conditions of data supply, and to have regard to the security of datasets entrusted to them. Any infringement, whether by deliberate abuse or negligence, will be regarded extremely seriously by NERC, as endangering Council’s own reputation and the integrity of the NERC supported scientific community. The introduction of sanctions against individuals or Departments may be considered if breaches occur.”
Gavin and R Ladbury
If a scientist gets FOI request(s), how is that a headache? How is that obstuction? I’ve handled, or been a part of a team that’s handled the equivalent of FOI requests and I can understand some of the frustration so dont go off on a tanget about how irritating these FOI requests are and how they can serve as a weapon.
You must have read Willis’ reconstruction of the string of emails relating to the FOI requests? You must have first hand knowledge of the sequence. How is it possible not to find Phil Jones’ position at the least faintly damning?
You say FOI requests create a vituperative climate for doing science? He [Willis] is claiming that it is a funny thing for FOI requests to be a medium of doing science at all. How many times does he say Jones talk about it? “I’ll be hiding behind IPR, he is a former employee (to the previous CRU chief), she is a former employee, lets all delete emails, shh no one reveal that we have an FOIA in the UK….”what ‘scientific’ arguments are these?
He apparently convinced the Information Officer of UAE dealing with the matter to come on board. By pointing out the identity of those who were requesting information.
I dont understand why you have to tarnish your reputation defending that which happened.
[Response: This is paranoia. There is no evidence that anyone has massaged the variability in the proxy reconstructions to match estimates of model internal variability. None. How would you even do it if you wanted to? – gavin]
That approaches name-calling, doesn’t it? How would I do it – I guess you’d want a dataset that showed a lower maxima than the instrumental data showed in the last 50 years. You’d want to be able to fit it to some period of instrumental data. And you’d want to present small enough error bands so as to make it useful to the IPCC report. All pretty straightforward, I think. And doable. And in line with the concerns and issues raised in these emails “delete everything related to AR4 CH6”, “hide the decline “(which is really about not showing the full dvergence between tree-ring data and instrumental data).
[Response: Whew… what a ride! You are missing the point entirely. The ‘background variability’ is the stuff that is really hard to constrain. It’s not the mean value that it gets to in the little ice age, but the size of the wiggles, and their spatial distribution. I know of no way to artificially enhance or dampen that without also messing up the calibration and verification of the long term trends (which are usually the object of interest). And check your timelines… the hide the decline comment referred to a figure made in 1999. AR4 discussions weren’t until 2006! – gavin]
Even IF data was manipulated, thrown away, etc., it is in the end just ONE data set at ONE institution. AGW theory, like any scientific theory, was never dependent on a single set data.
This incident is reminiscence of when biblical creationists get all ginned up about the Piltdown Man hoax and trot it out as “evidence” against biological evolution. The fact of the matter is that evolution doesn’t (and at the time, didn’t need) need Piltdown Man to be real in order to be a sound scientific theory. Quite simply there is a mountain of evidence and data from a multitude of disciplines supporting the theory of evolution. The same goes for AGW.
Where? Can you provide me with links to that code?
Sorry, but I needed to express my thanks to Moira, not Sean.
Gavin, in answer to my mesage of 29 November 2009 at 9:04 PM
Gavin, With the approach of the Copenhagen meeting, we see more and more alarming articles on climate change which is vaguely understood to be the same as AGW. Most of these articles seem to emphasize the warming of the artic as proof of climate change. My question is: what contribution does the radiative forcing due to CO2 have on the warming of the artic? Some climatologist believe that the gulf stream will reduce in intensity thus cooling down northern europe with AGW which seems to contradict the observation of a warming of the artic. Is the gulf stream the main mechanism by which heat is transmitted from the tropics to the artic ?
You responded
[Response: The Gulf Stream is part of that process, but there is no good evidence that there has been any significant change in the heat transported north in recent decades. CO2 and the other greenhouse gases do have an impact on the arctic, but than so do aersols like black carbon and ozone – both warming fators. Look up Shindell et al 2008 for a discussion of this. – gavin]
I appreciate having a professional climatologist as yourself de-mystify my thoughts on climatology for me. Correct me if I am wrong but the artic is known for its variability in temperature as seen in the sudden rise in temperatures in the artic in the start of the 20th century (before significant CO2 emissions). Could the mechanisms which explained the sudden rise in temperature in the 1905 to 1945 period also explain in part the rise in temperatures at the end of the 20th century ?
[Response: Sure. The variability in the Arctic is higher, but so is the signal (for very similar reasons). Therefore the signal to noise ratio is actually similar. So the detection and attribution of the changes happens in the same way as for the global mean. – gavin]
I read a lot of the original CRU emails in question. I’m not a scientist, I’m just trying to understand what these emails mean (if anything). I’ve read many of the posts here and I think I’ve formed my opinion. First, the terminology used on this blog is illuminating to me: “scientists” and “skeptics.” Second, “Gavin” does a great job explaining-away the specifics of the emails. I think Gavin has an explanation for every single email (he must be a “scientist” and not a “skeptic”). Note to Gavin: Some of the emails and language used don’t require “explaining away”, they stand for themselves. Overall, after thoroughly reading many of the CRU emails and much of this blog, I think the emails mean something and I think the scientific community has some more “explaining-away” to do. I guess I’m a “skeptic” now.
PS: No need to respond Gavin, I know exactly what you’re going to say. “Scientists” shouldn’t be so predictable.
In the FOIA documents, there is a folder full of tree ring data files many with labels of -CENSORED and -FIXED located at FOIA\documents\mbh98-osborn.zip\mbh98.tar\TREE\ITRDB\NOAMER\. In this folder there is the well discussed BACKTO_1400-CENSORED file and many others e.g. BACKTO_1400-FIXED, BACKTO_1400, BACKTO_1300-CENSORED, BACKTO_1300-FIXED, covering several time periods, as well as a host of files that appear to compare or generate the various versions of the time series.
I posted a plot of BACKTO_1400-CENSORED vs. BACKTO_1400-FIXED vs. BACKTO_1400 with linear regressions at:
http://1.bp.blogspot.com/_nOY5jaKJXHM/SxQebvCIjuI/AAAAAAAAARA/xlK6gks_Bh8/s1600/ScreenShot337.bmp
The red data and red trend line is the CENSORED version, which shows a downtrend.
The green data and green trend line is the FIXED version, which shows an uptrend.
The blue data and blue trend line is the “BACKTO_1400”-no other label- file, which shows very slight downtrend over the period. BTW, these same trends are present as well throughout the entire 1400-1980 data set, but the trends are amplified by a factor of 2.5 during the 20th
century.
It is also clear that the 3 datasets vary markedly in either direction on a yearly basis throughout the entire 1400-1980 and 1900-1980 periods.
Comments?
[Response: “censored” in this context means removing some of the data and seeing how well things perform (for verification of that, look it up in the emails). In MBH98, the more data you remove the worse the reconstruction, and at some point your reconstruction will not pass the verification step. Removal of all tree ring data – in that reconstruction – leads to the method breaking down before around ~1700 (Newer reconstructions are more robust). However, just plotting the numbers in each individual test doesn’t tell you very much, because you seem to be implying that each of these sensitivity tests are somehow equal. They very clearly aren’t. – gavin]
#966 Gavin “On short timescales natural variability is much larger than the expected trend due to rising greenhouse gases.”
Gavin, what do you mean by “natural variability”? Is not the phrase no more than a waving away of what we don’t understand about the climate? If so, how can we claim to understand the role “natural variability” plays? Does it come in predictable cycles? Do we have equations which can accurately predict its effect into the future? And why do we need to wait as much as 30 years to distinguish such natural sometimes-up-sometimes-down effects from the comparatively barely detectable response of global temperature to rises in anthropogenic CO2?
Is this not like trying to explain “the prettier shell” while the whole ocean lies undiscovered?
I, like others here, think Gavin should be congratulated for his persistence!
Am still neither fully here nor there, although I’d like to be.
Even if most conclude the science is sound, methods we appear to be taking to solve this apparent crisis are only adding to the reluctance of some to believe it.
Some details of the Copenhagen Treaty and talk of trade-able carbon are not encouraging for many. And rumours vary on how much extra tax an average individual may end up handing over, directly and indirectly, in the name of helping the environment.
So what of trading carbon credits? A new financial market to play around with, create interesting derivatives for, and to profit from (whom though?). And created out of what? Pieces of paper giving you the right to emit some carbon compounds without worry, but at a cost! Not to mention suggestions about the formation of a small global unit with power to tax individual states in the same vain and exercise immense control over national economies.
This will end up in a large tax code over years – bet on it if you can.
Problem being we know that this approach does not always solve a problem. It does, however, always bring in more income for government to spend wisely (!)
When government stimuli were being handed out all over the place, climate friendly and pro-active ‘green’ business’ only received approx. 20% of the total ‘gifted’. Not particularly widely publicised though, funnily enough.
I hope this isn’t a sign of things to come. The rhetoric is great at the moment.
The opportunism of politics and big business is not helping you. It makes the need for good scientific disclosure more crucial.
When science is the determining factor to tax tens of trillions of dollars then all the data has to be released. According to science peer review is to be reviewed by even the harshest of critics. Full disclosure or don’t publish the study. Without full disclosure it no longer is science.
Manacker, I think that people who have the least amount of knowledge while having the strongest convictions are handicapping society. When I think of what Gavin’s had to put up with during the last week or so it reminds me of what Galileo went through. I don’t think he’ll be put on house arrest by the status quo forces of our day….at least not yet.
[Response: Don’t get carried away here. I’m just blogging. – gavin]
I think all of the DoD data on building fusion weapons would be a lot more interesting than Latvia’s temperature data (or whatever country declares them to be proprietary).
So do me a favor and file your FOIAs and all to get me that cool DoD stuff, OK?
I can’t access the Trenberth paper any longer – the link seems to be broken I wanted to point some people on another blog to it.
RE: Michael Trigoboff
Code point #6:
“Discuss issues science raises for society”
This is how this seems to be working:
1. Science raises the issue of anthropogenic global warming.
2. Politicians create international bureaucratic/scientific body (IPCC) to investigate and report potential impact of AGW.
3. Science creates computer-generated virtual projections of the warming which might occur due to AGW.
4. Science attempts to quantify the magnitude of the potential future impacts on society and the environment, largely based on expert opinion rather than formal attribution studies.
5. Science suggests based on these computer projections and expert opinions that AGW could become a threat to society and the environment.
6. Science advocates that mitigating action should be taken to save society from this threat.
7. Politicians seize on most extreme virtual projections and expert opinions in order to initiate scare mongering campaign to justify carbon taxes (direct or indirect) on society, in order to save society from itself.
simon abingdon (1011) — I’m an amateur here, so I’ll not attempt to list all the sources of so-called natural variability, sticking mainly just to ENSO, the El Nino, La Nina Souther Oscillation as it is called. First of all, the ENSO index offers some predicatability for a few months into the future; all this is available from a NOAA web site. However, it certainly is not periodic or even strictly pseudoperiodic; I don’t know why.
Nonetheless, a part of the ENSO variation is a definitely periodic (or very nearly so) 3.8 year cycle in the equatorial Pacific Ocean. It consists of a slow westbound Rossby internal wave in the water and a fast return Kelvin wave along the equator. If this had a precisely 4 year period I think I could understand it; but the three or four different frequency analyses for the Pacific I have seen all agree on 3.7–3.8 years. Mysterious. In any case, it is the only periodic signal in climate I know about with period longer than a year; the other so-clled oscillations seem considerably less regular, hence less predictable.
The rest of ENSO, probably the stronger component, has a much long quasi-period, being rather spread out rather than regular. Somehow, it seems, the interaction between these two components give rise to stronger or weaker El Nino and La Nina conditions. TO point out my current lack of knowledge, I think I can understand how El Nino arises; not so (yet) for La Nina. By the way, when El Nino happens it tends to supress tropical cyclone formation in the Atalantic, as occurred this last hurricane season.
There are plenty more of these ocean oscillations. I’ll just mention PDO since in appears to play a quite profound role for ocean fisheries in the Pacific Northwest and also parts of Alaska.
Good one here:
http://home.badc.rl.ac.uk/lawrence/blog/2009/11/29/drawn_into_climategate
Has realclimate.org ever responded to the content in this paper by Professor Lindzen at MIT?
http://arxiv.org/ftp/arxiv/papers/0809/0809.3762.pdf
One of the seemingly relevant sections:
—–
The main statement publicized after the last IPCC Scientific Assessment two years ago was that it was likely that most of the warming since 1957 (a point of anomalous cold) was due to man. This claim was based on the weak argument that the current models used by the IPCC couldn’t reproduce the warming from about 1978 to 1998 without some forcing, and that the only forcing that they could think of was man. Even this argument assumes that these models adequately deal with natural internal variability—that is, such naturally occurring cycles as El Nino, the Pacific Decadal Oscillation, the Atlantic Multidecadal Oscillation, etc.
Yet articles from major modeling centers acknowledged that the failure of these models to anticipate the absence of warming for the past dozen years was due to the failure of these models to account for this natural internal variability. Thus even the basis for the weak IPCC argument for anthropogenic climate change was shown to be false.
[Response: Lindzen’s public talks feature much sliming of scientists, much misdirection, a lot of condescension and some very poor logic used in the service of spin. This is a shame because he is actually very smart. He has been minimising the temperature change since the 1980’s when he claimed that the global warming signal was only about 0.1ºC. His comment about 1957 is juvenile – IPCC in 2007 was talking ‘trends over the last 50 years’ not whether 1957 was particularly cold. His characterisation of the attribution process is a fiction. If IPCC was really using that argument how could they distinguish the impact of human-rises in CO2 from the number of monks in Tibet or ice cream sales in New York? The answer is that they use physics. When you’ve predicted decades before that you’re going to see an effect, and it will have a certain signature that is unique to that effect, and then you see it, I think you have a right to conclude (at least preliminarly) that maybe, just maybe, people know what they are talking about. Lindzen is even worse on the issue of natural variability in the models – he knows full well that the issue is the synchronisation of phases of natural oscillations, not whether they exist in the models or not (and they do). Lindzen’s infrequent published papers are of a somewhat better standard, but they haven’t (to date) stood up to much scrutiny, though in those he generally refrains from the bargain-basement illogic he displays in his talks and op-eds. – gavin]
—–
And one of the much less relevant sections but perhaps interesting to you:
—–
Environmental Media Services (a project of Fenton Communications, a large public relations firm serving left wing and environmental causes; they are responsible for the alar scare as well as Cindy Sheehan’s anti-war campaign.) created a website, realclimate.org, as an ‘authoritative’ source for the ‘truth’ about climate. This time, real scientists who were also environmental activists, were recruited to organize this web site and ‘discredit’ any science or scientist that questioned catastrophic anthropogenic global warming. The web site serves primarily as a support group for believers in catastrophe, constantly reassuring them that there is no reason to reduce their worrying.
—–
[Response: Wrong in fact, wrong in history and wrong in practice. None of us are ‘environmental activitists’, no-one was ‘recruited’, we do not claim to be the sole arbiters of ‘truth’, and I can’t recall any post in which we have claimed catastrophe is upon us. Prior to Realclimate though, Lindzen was more free to publish misleading nonsense with the imprimatur of MIT and have few people point out it’s glaring logical flaws. I guess he’s misses those days. – gavin]
I wouldn’t compare Gavin to Galileo.
But I WOULD compare Climategate to the Salem witch trials.
It’s good that that model’s code is available. But all of the code needs to be open source.
On 11/28/09 5:32 PM, RVB wrote to Editor:
The Wizard of Oz cartoon showing what I presume is Al Gore orchestrating the global warming debate plays to the conspiracy theorists who believe that Al Gore is pulling a giant hoax about global warming and is aided and abated by the UN, which seeks to rule the world — I’m not kidding. Google it. Printing such a cartoon is, in my opinion, irresponsible, unless accompanied by an editorial on the shear idiocy of these conspiracy nuts.
On 11/30/09 9:48 AM Editor wrote:
The stolen e-mails from Britain’s Climate Research Unit have done some real damage to the credibility of the scientific consensus on global warmng. I think the cartoon is a statement on that controversy. Today’s cartoon addressed the issue more clearly, however. CRU’s missteps played into the hands of the deniers. The Washington Post editorial that we ran on the same day at the Eric Allie cartoon put the issue into context.
On 11/30/09 4:57 PM, RVB wrote:
Today’s cartoon was okay — it spoke to the issue w/o pandering to the conspiracy nuts. Friday’s Washington Post op ed piece was also okay. No one who has worked in science, as I have, is under the delusion that scientists never fight or fabricate — scientists are human, some more so than others. Scientists who chaff under the constant badgering of deniers, whether they be denying evolution or global warming, can be forgiven if they sometimes, in private correspondence, speak to wanting to “kill” some denier who brings up the same argument over and over again, perhaps under different guises, no matter how often and how completely it has been refuted. Hey, it makes you mad. This is no excuse for hiding or attempting to stifle credible evidence counter to the prevailing science, ergo Galileo. It remains my judgment, based on the prevailing science, that global warming is occurring and is due to human generated greenhouse gas emissions, as well as human initiated deforestation (nothing in the hacked emails suggests otherwise). We are at a point where, what for some will be painful mitigation measures are absolutely essential, if we are to avoid even more painful consequences for all. It is interesting to me that this hacking incident and its “revelations” was timed conveniently to coincide a few weeks from now with the Copenhagen climate conference.
I recommend the following petition:
“Climate-gate is a fiction invented by the global-warming denier community because ample irrevocable independent scientific evidence still speaks in favor of global warming. Climate-gate will not be able to change this scientific evidence by creating a meaningless scandal around a still perfectly valid temperature reconstruction graph published many years ago.”
Oops – I know I’ve missed the bus on this one (given that the last comment was 30th November!) but I’ve had an extremely entertaining couple of hours trawling through theses comments!
You people (at RC) have the patience of saints – your graceful responses to your (predominantly graceless and frequently inarticulate and/or semi-literate) critics are admirable.
It is reassuring to know that, despite having been subjected to an unpleasant breach of security and subsequently a degree of public scrutiny and criticism that most people would wilt under, you have maintained your rationality, dignity and integrity (not to mention your sense{s} of humour…)
Even more impressively you have continued to focus on your work, and this above all impresses me and reassures me that the study of the science of climate change is in good hands.
Hold fast, and keep up the fabulous work!
Warm regards (and getting warmer…) from Melbourne, Australia
Re:
Barani says:
30 November 2009 at 12:00 AM
I see that some people are under an assumption that unless an event occurs it is not REAL, etc.
Barani, if you can explain why the kaon doesn’t really violate time-invariance, then I’ll be more inclined to accept the rest of your jumbled non-sequiturium of assertions.
Cheers.
#1007
Michael Trigoboff
Where? Can you provide me with links to that code?
There’s lots. Check the new RC data sources page
Timesonline has published a very damaging claim that CRU/UEA has dumped and lost raw data in favor of “massaged data”.
Fortunately, Little Green Footballs has debunked this story by showing what was omitted from the quote from CRU/UEA: http://littlegreenfootballs.com/article/35233_Did_Climate_Scientists_Destroy_Data_A-_No:
However, a major problem is that the original statement by CRU/UEA seems to be hard to reach: It’s hard to find a link to it, and the page seems to be basically unreachable.
“Houston, we have a problem”: Right-wing pundits are taking the timesonline story and running with it all over the internet: “Warmists dump real data in favor of managed data!” UEA needs to get that webpage fixed and get timesonline to retract that story. This is a public-relations fire: They need to get on the stick and put it out.
Given the blatant character of the omission of most of the contextualizing quote, and the failure to make any attempt to provide a link to the original statement, maybe this would be a good time to employ the famously “victim-friendly” attitude of the British libel laws – this time in the FAVOR of science.
#1023, When I compared Gavin to Galileo I had my tongue firmly in my cheek. I do however think he does an awesome job of suffering fools.
#1019 David B. Benson. Thank you for your response. I quote:
“I’ll not attempt to list all the sources of so-called natural variability”. (You could have attempted to list only the known sources in any case).
“ENSO…certainly is not periodic or even strictly pseudoperiodic; I don’t know why”. Who does know why?
“a precisely 4 year period I think I could understand…but…all agree on 3.7–3.8 years. Mysterious”. So more understanding needed there.
“There are plenty more of these ocean oscillations”. And plenty more yet to be discovered might you say?
So Gavin, if David B. Benson’s post reflects current thinking despite his “I’m an amateur” disclaimer, our knowledge of oceanic mechanisms (to say nothing of the causes and effects of cloud genesis) does seem pretty rudimentary, does it not? This coupled with my misgivings about scientists apparently pretending to be professional computer programmers (rather like aerodynamicists expecting to be able to land Boeings without the necessary training) gives me considerable pause when asked to have unqualified confidence in the GCMs.
And all this without in any way doubting either the physics involved or your unquestioned sincerity.
Gavin, teensy weensy glitch:
“McKitrick and Michaels (2004) and Kalnay and Cai (2003) were both cited and discussed in Chapter 2 of the IPCC AR4 report”
Umm. Might that have been Chapter 3?
Funny, my PhD students do split-half reliability tests if necessary. Tsk, tsk.
[Response: They do split-half reliability tests to check for typos? Fascinating. Yes, Chapter 3. – gavin]
Re: 998 Gavin’s reply.
Thanks for your comment, Gavin. For the record, I only started looking at GW out of interest while studying for a computer science BSc with a heavy programming bias. Before that, I had been programming for pleasure for around 20 years. So, yes, I have written plenty of buggy code in my time, including some that may have hidden bugs that haven’t yet surfaced.
However, none of my code has been involved in shaping the future of the world – if there was a chance of that I would have been awfully careful to make sure it was not only right, but easily debugged by others. Even without such importance on my work, I haven’t created an overflow error since high school, nor have I written code that blithely ignores data errors and carries on regardless. The code that poor Harry was dealing with does both, and was used to maintain what is ostensibly one of the most important databases in the history of mankind. I find that troubling.
As for your note that we’ve moved on to CruTS 3, that bseems a little disingenious seeing as that move was clearly the object of Harry’s work. He may well have salvaged whatever could be salvaged in that process but, as he notes at one point, “we’ll never know what we’ve lost”.
There’s the rub. The “new reliable” data has been developed from data with unknown, and unknowable, corruption. It can’t therefore, be reliable.
For anyone not into data management, consider a quick and dirty analogy which you can try at home. Take a high-resolution photo from your digital camera and use your favourite photo software to shrink it to, say, 200 x 300 pixels. Now save it and leave it kicking round your computer for a while (it’s only a simulation, so a few seconds should do it).
Reload the photo and try to resize it back to full size. It will be almost unrecognisable because you’ve lost data in the process. Sure, you can use Photoshop to “improve” it but all the improvements are “best guesses” which can never actually recreated the original with any accuracy.
This is actually a fairly poor analogy because the algorithms used to resize and compress photographs are well defined and understood so, when it comes to reconstruction, their effects are much more predictable than the effects of software that inserts random data from overflows and simply throws away data it can’t deal with without any record.
With the photo example, the only way round it is to reload the original from your camera or, if you’ve already deleted that, to go back and take another photo.
Perhaps that should be the next step if climate science is to remain credible – a massive effort to go back to the stations (who presumably retain records), get all the data again – in an agreed, open and consistent format, and rebuild the databases from scratch. Since the underlying data is the same, all databases worldwide could be verified by this process.
It would be an intensive and expensive undertaking – though nowhere near as expensive as the actions being proposed in the name of AGW – but it would at least put the science back on a footing of unquestionable raw data.
Just an illustration of the triumph of politics over science — The November issue of The Costco Connection had a brief article by Al Gore. The December issue has an explosion of invective from readers on the theme that “Costco is not the place for politics” with a not-so-veiled threat of a boycott. Of the 11 letters to the editor, seven were virulently anti-Gore (“lies and garbage science, etc., etc.), three mildly pro-Gore and one just objecting to “politics.” A reader suggests contacting Fred Singer, the most knowledgeable of the climate-change “deniers” to write a balancing piece for you.
The Costco editor naturally complies and S. Fred provides “A scientist’s view on climate change.” The usual stuff: “the climate is always changing … but not in a way that indicates carbon dioxide is the cause … global warming stopped a decade ago … consult the conclusions of several distinguished scientists at http://www.nipccreport.org, etc.”
Very sad. There’s a German proverb “The gods themselves battle stupidity in vain.”
Gavin: You say, on the article at the top of this thread:
“The vast majority of the data in the HadCRU records is publicly available from GHCN (v2.mean.Z).”
Could you confirm that this is all of the CRU data, with the exception of the raw data owned by the Meteorological Services?
In other words, if someone was to get all the stuff from the link you gave and then get all the raw data from the Met Services, would they have all the data needed to check that everything was above board? Or would they need other data?
Gavin: You say, on the article at the top of this thread:
“The vast majority of the data in the HadCRU records is publicly available from GHCN (v2.mean.Z).”
Could you confirm that this is all of the CRU data, with the exception of the raw data owned by the Meteorological Services?
In other words, if someone was to get all the stuff from the link you gave and then get all the raw data from the Met Services, would they have all the data needed to check that everything was above board? Or would they need other data?
Sorry if you’ve already answered this somewhere. I have no idea how you have remained so calm!
Climategate and Watergate have a lot of parallels.
Watergate resulted in the conviction of some participants and the resignation (and later pardon) of the U.S. President.
How Climategate will end up is anyone’s guess today, but it is safe to say that it will have a major impact.
But there is one crucial difference between the two. This is the instant flow of information today, with the Internet, leaked emails, blog sites (including RC), youtubes, etc., none of which existed almost 40 years ago when ‘Deep Throat’ leaked information to the Washington Post reporters.
The WG process was driven by the mainstream media, which is largely ignoring CG (and has actually become irrelevant to the process).
Hi,
Why is Trenberth concerned about not being able to account for the recent cooling? (as per his statement). From what I have been reading, most of the global climate models have error bars that are big enough to account for the temps we are seeing. Maybe this wasn’t the context of his statement, and I don’t expect you to be able to read his mind. Nevertheless, I thought you might have some insight.
Thanks
[Response: He wants to know on a deeper level – where did the energy fluxes change, how much in the ocean, in the clouds, did aerosols play a role etc. Quite right too. -gavin]
simon abingdon (1032) — I doubt I know all the sources of internal variability, just something about the ocean oscillations. NOAA maintains a site about ENSO so you could start there or do as I did by web searching for pertinent literature.
I’d like to have a btter understanding of the rather strangely periodic 3.7–3.8 year Raossby/Kelvin wave in the Pacific but have not attempted to find further papers about it, just enough to see that this signal appears in several different records, including Atlantic temperatures.
I don’t think there are any more ocean oscillations to be discovered except possibly in the Southern Ocean; many people have built good SST records and even those of temperature at various ocean depths.
My knowledge of physical oceanography does not extend beyond reading Carl Wunsch’s excellant lecture notes. I rather suspect the O part of the various AOGCMs reflect a rather deeper understanding than mine. It is probably possible to learn about it from, say,
“A Climate Modelling Primer” by Henderson-Sellers
Introduction to Three-Dimensional Climate Modeling 2nd Edition
Warren M. Washington and Claire Parkinson
http://www.palgrave.com/products/title.aspx?PID=270908
https://www.realclimate.org/index.php/archives/2008/11/faq-on-climate-models/
https://www.realclimate.org/index.php/archives/2009/01/faq-on-climate-models-part-ii/langswitch_lang/tk
http://www.climatescience.gov/Library/sap/sap3-1/final-report/sap3-1-final-all.pdf
with the last linked pef gving one a good sense of the (high) confidence on ought to have in the combined collection of about 23 GCMs.
As for being “professional” programmers, my general sense of the matter is that FORTRAN is fairly easy; what is not is the ability to correctly do the numerical analysis in a GCM. That has been solved and all cliamte modelers know the tricks of the trade, so to speak; see the two books cited or read more briefly about it in “The Discovery of Global Warming” by Spencer Weart:
http://www.aip.org/history/climate/index.html
In your earlier comment you asked about the “30 years”. Many decades ago WMO, in looking a the statistics, figured out that climate was 30 or more years of data. My amateur opion is that given the long times for some of the quasi-periodic ocean oscillations even 30 years might not be long enough for some purposes. According to Tamino, GISS’s ModelE has a relaxation time of 30 years; to me that suggests wanting to look at 60–90 years of data to have high confidence of detecting small signals from the noisy record.
A latest Climategate development:
http://www.uea.ac.uk/mac/comm/media/press/2009/nov/homepagenews/CRUupdate
Gavin, I think that most of the skepticism regarding the numerical modeling of climate would be resolved if these models could be tested for what they are meant to do (i.e. predict temperature, precipitation, water levels over multi- decadal scales). You were mentioning previously that a period of 10 to 20 years would be required to really see the signature of the radiative forcing due to CO2 over the noise from the multi-decadal oscillations in ocean circulation and natural forcings . You also mentioned that the deep oceans require 500 years to really equilibrate. How confident are you that in 10 to 20 years you will be able to predict the temperature and precipitation? Should we not wait 10 years to test the climate model before spending 1% GDP ?
“Charles the Moderator at WUWT has put up a post suggesting and analyzing different scenarios of how the email file was released. He suggests:
I have a theory.”
Is Charles the Moderator at WUWT really Anne Elk in disguise?
pjclarke wrote:
“Well according to the Nature ‘Climate Feedback’ blog, he and his associates submitted 58 FOI requests over a 6 day period, that included a weekend. So that’s 14 requests a day or about 2 an hour. This is not science, this is spam!”
The background to this was a denial of access to data by Jones because of restrictions imposed by other data providers that supplied data to Jones. He claimed that this stopped him from redistributing. Unfortunately he was not able to cite any of these agreements except for 2 minor countries (which didn’t really impose strong restrictions) and a handwaving statement that he “lost” the other agreements.
The subsequent FOI requests were for agreements for each supplier country – since they were the reason for the original denial. So you have an FOI request denied because Jones claims he can’t respond due to agreements he cannot produce and you are upset that people ask for evidence of those agreements?
[edit]
Some random thoughts.
I’d look for phony and real “teabaggers” around Copenhagen time or thereafter. “Climategate”, I presume, will figure into the protests. As per their Rovian penchant for dirty tricks, I wouldn’t be surprised if they pull more out-of-context rabbits out of their hat at crucial moments. “Please treat this information as sensitive … we don’t want critics to know our game plan.”
http://tinyurl.com/o26qfq
I know I’m beating a dead horse but in an issue of this import, I’ll say this, let this be a lesson learned, that being that all the cards should, of course, always be on the table (and I’m not saying they weren’t). Not to only gives ammunition to those looking for anything at all to lay at the door of climate scientists. That includes not just raw data but the A – Z of how the conclusions were reached with it if people want to follow it. Put it as a link right on the RC homepage. It should probably also include money info. I have no idea what climate scientists are paid, as I am not one, but it wouldn’t look ethical if people were getting rich in this endeavor even though it is a far cry from being paid by industry simply to deny (hmmm, I wonder how much M&M are taking and from whom). Make everything as transparently transparent as possible. Where not possible explain why and try to change it. Further RC might consider a post on the specific certainties/uncertainties of the science and the %s thereof. That way people visualize the disparity between the two. To quote Prof Laut in the PDF:
So, it must be recognized that not all research institutions have accepted proper responsibility to maintain the trust which the general public traditionally places in them. In the modern world, many scientific results are extremely difficult to verify independently. They may be produced by a group of several researchers working for months or even years with vast amounts of data, which have to be calibrated employing especially tailored computer programs.
Often it is practically impossible for an outsider to verify the conclusions. That applies also to the referees, who have to decide on the publication of the work. So, trust is in the short run often all we have to judge the authenticity of claimed new developments. Trust in fellow scientists and trust in research institutions. And, at the Copenhagen conference on global climate the decision makers of the world, must be able to trust the scientific basis which is presented to them. They must be able to rely on it when building a strategy to fend off catastrophic climate
developments. So, the scientific community should be careful not to squander this trust.
Next random thought; the deniers often yell about the costs of mitigating climate change. I wonder how much it would cost to build giant levees all along the US coastlines if we do nothing and take an adaptation only approach? And that’s just on US coastlines!
Another thought, RC might consider a denier’s profiles info page complete with pictures and all of their connections to Big Energy. Greenpeace, UCSUSA and Ross Gelbspan could probably help out. Something journalists can access immediately. Right now the info is mostly scattered around the web.
I mentioned this before, but a tool that actually moves, is visual for the average joe like this might be nice.
http://tinyurl.com/yhkdnhy
Neal J. King (1030)
The Little Green Football blog you cited “Did Climate Scientists Destroy Data? A: No.” calls the apparent outrage following the leaking of CRU emails a “tempest in a teapot”, saying “no data was destroyed”.
This is not correct. According to CRU’s Web site, “Data storage availability in the 1980s meant that we were not able to keep the multiple sources for some sites, only the station series after adjustment for homogeneity issues. We, therefore, do not hold the original raw data but only the value-added (i.e. quality controlled and homogenized) data.”
[Response: “Do not hold” is not the same a ‘destroyed’. If I give a bag to my spouse to hold, I do not hold it, and yet it is not destroyed. Enough with the torturing of the language already. – gavin]
BBC News: December 1, 2009
“It is also possible that the institutional innovation that has been the IPCC has now largely run its course.
Perhaps, through its structural tendency to politicise climate change science, it has helped to foster a more authoritarian and exclusive form of knowledge production – just at a time when a globalising and wired cosmopolitan culture is demanding of science something much more open and inclusive.”
Mike Hulme – School of Environmental Sciences at the U of East Anglia
Dr Jerome Ravetz, Institute for Science, Innovation and Society (InSIS) at Oxford University
Peter Watts, eloquently:
http://www.rifters.com/crawl/?p=902
Sean asks, “Why is Trenberth concerned about not being able to account for the recent cooling? (as per his statement).”
Sean, what you are failing to consider is that Trenberth is a climate SCIENTIST, and as such motivated by curiosity. We still have a whole helluva lot to learn about Earth’s climate–it’s just that it won’t change the fact that if you add CO2 to the atmosphere, the planet will heat up. In other news, we have a lot to learn about gravity–but that won’t change the fact that apples fall down rather than up.
Why do I feel that you are standing up completely for them?
Do you not agree that there is a possibility of false facts, basically lies, being put into these reports to achieve a hidden agenda?
I am a strong believer in global climate change, and an avid environmentalist, and still am, but I see that there is a possibility that they have given us selective facts to help lobby us.
If this is a good scientific website, they should have a skeptic of the legitimacy of the letters and, well, they already have you.
Thank you, and I’d be very curious to hear your opinion on all I just said.