Climate Science

This is a continuation of the last thread which is getting a little unwieldy. The emails cover a 13 year period in which many things happened, and very few people are up to speed on some of the long-buried issues. So to save some time, I’ve pulled a few bits out of the comment thread that shed some light on some of the context which is missing in some of the discussion of various emails.

• Trenberth: You need to read his recent paper on quantifying the current changes in the Earth’s energy budget to realise why he is concerned about our inability currently to track small year-to-year variations in the radiative fluxes.
• Wigley: The concern with sea surface temperatures in the 1940s stems from the paper by Thompson et al (2007) which identified a spurious discontinuity in ocean temperatures. The impact of this has not yet been fully corrected for in the HadSST data set, but people still want to assess what impact it might have on any work that used the original data.
• Climate Research and peer-review: You should read about the issues from the editors (Claire Goodess, Hans von Storch) who resigned because of a breakdown of the peer review process at that journal, that came to light with the particularly egregious (and well-publicised) paper by Soon and Baliunas (2003). The publisher’s assessment is here.

Update: Pulling out some of the common points being raised in the comments.

• HARRY_read_me.txt. This is a 4 year-long work log of Ian (Harry) Harris who was working to upgrade the documentation, metadata and databases associated with the legacy CRU TS 2.1 product, which is not the same as the HadCRUT data (see Mitchell and Jones, 2003 for details). The CSU TS 3.0 is available now (via ClimateExplorer for instance), and so presumably the database problems got fixed. Anyone who has ever worked on constructing a database from dozens of individual, sometimes contradictory and inconsistently formatted datasets will share his evident frustration with how tedious that can be.
• “Redefine the peer-reviewed literature!” . Nobody actually gets to do that, and both papers discussed in that comment – McKitrick and Michaels (2004) and Kalnay and Cai (2003) were both cited and discussed in Chapter 2 of 3 the IPCC AR4 report. As an aside, neither has stood the test of time.
• “Declines” in the MXD record. This decline was hidden written up in Nature in 1998 where the authors suggested not using the post 1960 data. Their actual programs (in IDL script), unsurprisingly warn against using post 1960 data. Added: Note that the ‘hide the decline’ comment was made in 1999 – 10 years ago, and has no connection whatsoever to more recent instrumental records.
• CRU data accessibility. From the date of the first FOI request to CRU (in 2007), it has been made abundantly clear that the main impediment to releasing the whole CRU archive is the small % of it that was given to CRU on the understanding it wouldn’t be passed on to third parties. Those restrictions are in place because of the originating organisations (the various National Met. Services) around the world and are not CRU’s to break. As of Nov 13, the response to the umpteenth FOI request for the same data met with exactly the same response. This is an unfortunate situation, and pressure should be brought to bear on the National Met Services to release CRU from that obligation. It is not however the fault of CRU. The vast majority of the data in the HadCRU records is publicly available from GHCN (v2.mean.Z).
• Suggestions that FOI-related material be deleted … are ill-advised even if not carried out. What is and is not responsive and deliverable to an FOI request is however a subject that it is very appropriate to discuss.
• Fudge factors (update) IDL code in the some of the attached files calculates and applies an artificial ‘fudge factor’ to the MXD proxies to artificially eliminate the ‘divergence pattern’. This was done for a set of experiments reported in this submitted 2004 draft by Osborn and colleagues but which was never published. Section 4.3 explains the rationale very clearly which was to test the sensitivity of the calibration of the MXD proxies should the divergence end up being anthropogenic. It has nothing to do with any temperature record, has not been used in any published reconstruction and is not the source of any hockey stick blade anywhere.

Further update: This comment from Halldór Björnsson of the Icelandic Met. Service goes right to the heart of the accessibility issue:

Re: CRU data accessibility.

National Meteorological Services (NMSs) have different rules on data exchange. The World Meteorological Organization (WMO) organizes the exchange of “basic data”, i.e. data that are needed for weather forecasts. For details on these see WMO resolution number 40 (see http://bit.ly/8jOjX1).

This document acknowledges that WMO member states can place restrictions on the dissemination of data to third parties “for reasons such as national laws or costs of production”. These restrictions are only supposed to apply to commercial use, the research and education community is supposed to have free access to all the data.

Now, for researchers this sounds open and fine. In practice it hasn’t proved to be so.

Most NMSs also can distribute all sorts of data that are classified as “additional data and products”. Restrictions can be placed on these. These special data and products (which can range from regular weather data from a specific station to maps of rain intensity based on satellite and radar data). Many nations do place restrictions on such data (see link for additional data on above WMO-40 webpage for details).

The reasons for restricting access is often commercial, NMSs are often required by law to have substantial income from commercial sources, in other cases it can be for national security reasons, but in many cases (in my experience) the reasons simply seem to be “because we can”.

What has this got to do with CRU? The data that CRU needs for their data base comes from entities that restrict access to much of their data. And even better, since the UK has submitted an exception for additional data, some nations that otherwise would provide data without question will not provide data to the UK. I know this from experience, since my nation (Iceland) did send in such conditions and for years I had problem getting certain data from the US.

The ideal, that all data should be free and open is unfortunately not adhered to by a large portion of the meteorological community. Probably only a small portion of the CRU data is “locked” but the end effect is that all their data becomes closed. It is not their fault, and I am sure that they dislike them as much as any other researcher who has tried to get access to all data from stations in region X in country Y.

These restrictions end up by wasting resources and hurting everyone. The research community (CRU included) and the public are the victims. If you don’t like it, write to you NMSs and urge them to open all their data.

I can update (further) this if there is demand. Please let me know in the comments, which, as always, should be substantive, non-insulting and on topic.

Comments continue here.

As many of you will be aware, a large number of emails from the Climatic Research Unit (CRU) at the University of East Anglia webmail server were hacked recently (Despite some confusion generated by Anthony Watts, this has absolutely nothing to do with the Hadley Centre which is a completely separate institution). As people are also no doubt aware the breaking into of computers and releasing private information is illegal, and regardless of how they were obtained, posting private correspondence without permission is unethical. We therefore aren’t going to post any of the emails here. We were made aware of the existence of this archive last Tuesday morning when the hackers attempted to upload it to RealClimate, and we notified CRU of their possible security breach later that day.

Nonetheless, these emails (a presumably careful selection of (possibly edited?) correspondence dating back to 1996 and as recently as Nov 12) are being widely circulated, and therefore require some comment. Some of them involve people here (and the archive includes the first RealClimate email we ever sent out to colleagues) and include discussions we’ve had with the CRU folk on topics related to the surface temperature record and some paleo-related issues, mainly to ensure that posting were accurate.
[Read more…] about The CRU hack

One thing a scientist doesn’t want to mess up is the problem of multiplicity (also known as ‘field significance‘). It’s just like rolling a die 600 times, and then getting excited about getting roughly 100 sixes. However, sometimes it’s much more subtle than just rolling dice.

This problem seems to be an issue in a recent by paper with the title ‘Evidence for solar forcing in variability of temperatures and pressures in Europe‘ by Le Mouel et al. (2009) in the Journal of Atmospheric and Solar-Terrestrial Physics.

[Read more…] about A problem of multiplicity

Some of the highest growing trees in the world are also the oldest—bristlecone pines (Pinus longaeva) from the Great Basin in the western United States (eastern California, Nevada and Utah). The oldest example is more than 4800 years old. Because of their longevity and growth at high elevations (where the growth of trees is generally known to be limited by temperature) bristlecone pines have been of particular interest to dendroclimatologists (paleoclimatologists who study tree rings to reconstruct past climate). Numerous ecological studies carried out at treeline sites all over the world show that temperature imposes a critical limitation on the ability of trees to produce new tissue; mean daily temperatures of 8-9°C are required, so recent warming will have particular benefits for those trees that have managed to eke out an existence for so long, living “on the edge”.

An interesting characteristic of the western bristlecone pines is that their recent growth has markedly increased—ring widths have been higher than in previous decades. Previous studies have debated to what extent this “fact” is real, or just an artifact of the way tree-ring data are analyzed. Because the growth of trees is radial, as trees get older and the diameter of a tree increases, annual ring widths decline in thickness. This is the normal “growth function” that is commonly removed from measurements before further analysis is carried out. The trick is to do this carefully so that as much climate information is retained while the growth function is discarded, and dendroclimatologists know how to do this quite well. However, sometimes the “standardization” procedure can introduce spurious results. This led some to regard the apparent growth increase in bristlecone pines to be a meaningless result of the data processing. In a new article in the Proceedings of the U.S. National Academy of Sciences (PNAS) Matthew Salzer (Laboratory of Tree Ring Research, University of Arizona) and colleagues examine this issue head on. They studied hundreds of trees from treeline sites in the Great Basin, aligned all the samples according to date, and simply averaged the results (Figure 1). Given that these trees are all long-lived, the complicating factor of growth function (which is strongest for the early growth of a tree) was not significant for assessing the most recent growth. Their results show that mean ring width in the last 50 years has been greater than in any previous 50 year period over the last 3700 years. You have to go all the way back to ~1900-1300 B.C. to find mean ring widths approaching recent values. Furthermore, the recent increase in ring widths is seen in trees at the upper forest border at sites hundreds of km away (even when the treelines there were at lower elevations)—but not in trees below the upper forest border. Below the zone closest to treeline, wide rings are formed in cool, wet years, and narrow rings in warm, dry years, and trees from this lower zone do not show the 20th century growth surge.

It is thus clear that the bristlecone pines from the highest regions, close to their growth limit, are showing a very strong response to recent warming, and indicating just how unusual it has been in the context of the last few millennia. Previous explanations have focused on possible CO2 fertilization effects (increasing water use efficiency) but there is no obvious reason why such factors would have affected only trees within approximately 150m of local treeline in different locations. Rather, the high elevation trees, close to the limit of growth, have responded positively to the recent increase in temperature just as ecological studies would have predicted.

One final note: bristlecone pines often have an unusual growth form known as “strip bark morphology” in which annual growth layers are restricted to only parts of a tree’s circumference. Some studies have suggested that such trees be avoided for paleoclimatic purposes, a point repeated in a recent National Academy of Sciences report (Surface temperature reconstructions for the last 2,000 years. NRC, 2006). However Salzer et al’s study shows that there is no significant difference in their results when the data are divided into two classes—strip bark and non-strip-bark cases –when the raw unstandardized data are compared. So that particular issue has apparently had people barking up the wrong tree…

Figure 1: Median ring-widths (non-overlapping 50-year means) of upper forest border Pinus longaeva from 3 sites in western North America, plotted on first year of interval (from Salzer et al, PNAS, 2009)