Kevin Wood, Joint Institute for the Study of the Atmosphere and Ocean, University of Washington
Eric Steig, Department of Earth and Space Sciences, University of Washington
In the wake of the CRU e-mail hack, the suggestion that scientists have been hiding the raw meteorological data that underpin global temperature records has appeared in the media. For example, New York Times science writer John Tierney wrote, “It is not unreasonable to give outsiders a look at the historical readings and the adjustments made by experts… Trying to prevent skeptics from seeing the raw data was always a questionable strategy, scientifically.”
The implication is that something secretive and possibly nefarious has been afoot in the way data have been handled, and that the validity of key data products (especially those produced by CRU) is suspect on these grounds. This is simply not the case.
It may come as a surprise to some that the first compilation of world-wide meteorological data was published by the Smithsonian Institution in 1927, long before anthropogenic climate change emerged as an important issue (Clayton et al., 1927). This volume is still widely available on the library shelf as are updates that were issued periodically. This same data collection provided the foundation for the World Monthly Surface Station Climatology, 1738-cont. As has been the case for many years, any interested party can access this from UCAR (http://dss.ucar.edu/datasets/ds570) and other electronic data archives.
Now, it is well known that these data are not perfect. Most records are not as complete as could be wished. Errors periodically creep in and have to be identified and weeded out. But beyond the simple errors of the key-entry type there are inevitably discontinuities or inhomogeneities introduced into the records due to changes in observing practices, station environment, or other non-meteorological factors. It is very unlikely there is any historical record in existence unaffected by this issue.
Filtering inhomogeneities out of meteorological data is a complicated procedure. Coherent surface air temperature (SAT) datasets like those produced by CRU also require a procedure for combining different (but relatively nearby) record fragments. However, the methods used to undertake these unavoidable tasks are not secret: they have been described in an extensive literature over many decades (e.g. Conrad, 1944; Jones and Moberg, 2003; Peterson et al., 1998, and references therein). Discontinuities may nevertheless persist in data products, but when they are found they are published (e.g. Thompson et al., 2008).
Furthermore, it is a fairly simple exercise to extract the grid-box temperatures from a CRU dataset—CRUTEM3v for example—and compare it to raw data from World Monthly Surface Station Climatology. CRU data are available from http://www.cru.uea.ac.uk/cru/data/temperature. One should not expect a perfect match due to the issues described above, but an exercise like this does provide a simple way to evaluate the extent to which the CRU data represent the underlying raw data. In particular, it would presumably be of interest to know whether the trends in the CRU data are very different than the trends in the raw data, since this could be taken as indication that the methods used by CRU result in an overstatement of the evidence for global warming.
As an example, we extracted a sample of raw land-surface station data and corresponding CRU data. These were arbitrarily selected based on the following criteria: the length of record should be ~100 years or longer, and the standard reference period 1961–1990 (used to calculate SAT anomalies) must contain no more than 4 missing values. We also selected stations spread as widely as possible over the globe. We randomly chose 94 out of a possible 318 long records. Of these, 65 were sufficiently complete during the reference period to include in the analysis. These were split into two groups of 33 and 32 stations (Set A and Set B), which were then analyzed separately.
Results are shown in the following figures. The key points: both Set A and Set B indicate warming with trends that are statistically identical between the CRU data and the raw data (>99% confidence); the histograms show that CRU quality control has, as expected, narrowed the variance (both extreme positive and negative values removed).
Comparison of CRUTEM3v data with raw station data taken from World Monthly Surface Station Climatology. On the left are the mean temperature anomalies from each pair of randomly chosen times series. On the right are the distribution of trends in those time series and their means and standard errors. (The standard error provides an estimate of how well the sampling of ~30 stations represents the full global data set assuming a Gaussian distribution.) Note that not all the trends are for identical time periods, since not all data sets are the same length.
Conclusion: There is no indication whatsoever of any problem with the CRU data. An independent study (by a molecular biologist it Italy, as it happens) came to the same conclusion using a somewhat different analysis. None of this should come as any surprise of course, since any serious errors would have been found and published already.
It’s worth noting that the global average trend obtained by CRU for 1850-2005, as reported by the IPCC (http://www.ipcc.ch/pdf/assessment-report/ar4/wg1/ar4-wg1-chapter3.pdf), 0.47 0.54 degrees/century,* is actually a bit lower (though not by a statistically significant amount) than we obtained on average with our random sampling of stations.
*See table 3.2 in IPCC WG1 report.
References
Clayton, H. H., F. M. Exner, G. T. Walker, and C. G. Simpson (1927), World weather records, collected from official sources, in Smithsonian Miscellaneous Collections, edited, Smithsonian Institution, Washington, D.C.
Conrad, V. (1944), Methods in Climatology, 2nd ed., 228 pp., Harvard University Press, Cambridge.
Jones, P. D., and A. Moberg (2003), Hemispheric and large-scale surface air temperature variations: An extensive revision and an update to 2001, Journal of Climate, 16, 206-223.
Peterson, T. C., et al. (1998), Homogeneity adjustments of in-situ atmospheric climate data: a review, International Journal of Climatology, 18, 1493-1517.
Thompson, D. W. J., J. J. Kennedy, J. M. Wallace, and P. D. Jones (2008), A large discontinuity in the mid-twentieth century in observed global-mean surface temperature, Nature, 453(7195), 646-649.
When you write emails like the leaked ones, you become automatically a suspect, are you still pounding that question? And please stop the “out of context” argument – there is ample context.
But from there to draw any conclusions about the CRU estimates of the temperature developement, is not wise. That we agree on. But bear in mind that it is due to lack of openness that we have got into this situation. I believe that the CRU estimates are not too far off, but now they need to be investigated.
However, there is much more to that discussion, for instance what is the exact objective of the measurement? What is it exactly that CRU measures? What would we want it to be? For instance, should the measurement include the effects of urbanization? extensive change of land use? large area irrigation? etc
Charles Copeland: I know that Patrick Michaels is a persona non grata here but whatever about his shortcomings I think his chapter on the subject in his recent book ‘Climate of Extremes’ is a must-read for RC’s editors…
BPL: Michaels egregiously lied to Congress by showing only the highest of Hansen’s three 1988 scenarios in a chart and claiming that his “prediction” (singular) had been falsified by subsequent events. That makes him an unreliable source of information. Period.
DEB: I was talking here about polar cities to save survivors in 2500 AD and you all thought I was nuts. What do you think now?
BPL: I think you’re nuts.
Sorry, I couldn’t resist. But A) in 2500, there won’t be a North Pole, and B) the survivors won’t have the capital, infrastructure and functioning economy enough to build a domed city in Antarctica.
ZT, the Urban Heat Island effect has been known and compensated for for a long, long time. Here are some references if you’re interested:
Hansen, J., Ruedy, R., Sato, M., Imhoff, M., Lawrence, W., Easterling, D., Peterson, T., and Karl, T. 2001. “A closer look at United States and global surface temperature change.” J. Geophys. Res. 106, 23947–23963.
Parker, DE. 2004. “Large-scale warming is not urban.” Nature 432, 290.
Parker, DE. 2006. “A Demonstration That Large-Scale Warming Is Not Urban.” Journal of Climate 19, 2882-2895.
Peterson, Thomas C. 2003. “Assessment of Urban Versus Rural In Situ Surface Temperatures in the Contiguous United States: No Difference Found.” J. Clim. 16(18), 2941-2959.
Peterson T., Gallo K., Lawrimore J., Owen T., Huang A., McKittrick D. 1999. “Global rural temperature trends.” Geophys. Res. Lett. 26(3), 329.
Jsc
15 December 2009 at 2:29 PM
It’s worse. Being a climate scientist, Eric is of course in on the conspiracy. Do you think he would have bothered even starting the analysis, knowing beforehand that the data was cooked?
You start insinuating wrongdoings by Eric, based on a figment of your imagination. Don’t you think you have to draw a line somewhere?
Oh my, you really think this is the first time a climate scientist does an analysis on the temperature record.
This blog post is a simplified repetition of what climate scientists have done many, many times before in a much, much more rigorous way. This blog post is meant as a simple excercise that any Excel jockey could repeat at home. Don’t pretend it is more than it is.
Gavin, Doug, Steve,
Thank you for your comments, much appreciated. I’ve just discovered that Michaels has published an article on the same subject in the journal Energy and Environment (2008, Vol. 19 No 2). I’ve applied for a free electronic copy via my employer and will forward it to you, if you’re interested, as soon as I receive it (hopefully this afternoon or tomorrow). The title of the article is “Evidence for “publication Bias” Concerning Global Warming in Science and Nature“.
More detailed reply later, time permitting.
#67:
If I read his table 7.1 correctly, 8:1 would be the ‘worse’ to ‘better’ ratio, not the reverse.
here
The “Michaels criterion” for bias is absurd. It would be met only if science were changeless. Moreover, it depends on which particular quantity or phenomenon one is considering. CO2 sensitivity is currently estimated to be roughly 3 degrees per doubling–and this has been the case for a very long time now.
Charles Copeland says, “I’ve just discovered that Michaels has published an article on the same subject in the journal Energy and Environment (2008, Vol. 19 No 2).”
And really, how could one refute one’s own argument more effectively than by publishing in that cesspit of a journal?
Take a look at Figure 4 on my page below:
http://www2.sunysuffolk.edu/mandias/global_warming/global_warming_misinformation_urban_heat_island.html
It sure looks like the “satellite only” measurements from UAH (run by skeptics Chrosty and Spencer) and RSS very closely match the actual surface measurements.
A few more examples here:
http://www2.sunysuffolk.edu/mandias/global_warming/global_cooling.html
Charles: Energy and Environment pretty much publishes anything but mostly contrarian viewpoints. By many standards it is NOT considered to be peer-reviewed. See the link below and scroll down to the E&E section:
http://greenfyre.wordpress.com/2009/11/18/poptarts-450-climate-change-denier-lies/
Even the editor has claimed that the journal has a political agenda and isn’t really a science journal.
Since at least the time of Galileo the right wing has been behind the eight ball as far as science is concerned. It’s a fact that during WW2 the OSS, a pre curser to the CIA went nuts trying to find researchers with the “Right Stuff,” who were not of the political left. They didn’t do very well. Today as then, liberals dominate scientific research. Maybe not because their smarter but because they’re more likely to think outside the box?
If we we’re to take all the detractors posting here and all other like minded individuals and put them all in one world, would we have been able to develop a vaccine for polio? Would we have been able to be successful with the genome project? Would we have been successful in our attempt be develop the atom bomb which ended WW2? I wonder.
(#3): Eric – thanks, I get the point that anyone can confirm the trend with available data. Since I was going to try to do just that over Christmas break, I thought it would be nice to start by replicating these results just to confirm that I’m doing things correctly.
Re: 92:
And Arctic amplification of warming has been observed in spades. However, it’s least marked in winter, since there is virtually no insolation at the highest latitudes during that season.
No insolation=no greenhouse effect.
Positive anomalies for the Arctic for this summer were quite eye-popping–2.5 C +, if I recall correctly.
You can access a bunch of relevant info here:
http://data.giss.nasa.gov/gistemp/graphs/
77, 78. I like the idea of the pictures. It may appeal to a lot of people. Is there a collection of beautiful pictures big enough to make a screensaver?
so I guess my question now is: if there is no statistically significant difference between raw and processed data, why process/correct it at all? why not leave the data that has no problems alone and only process the ones with clear gaps, errors, etc.?
I recommend that readers look at the first two graphs in Ryan O’s post at the Air Vent, referenced by Jeff Id in comment #47 (15 Dec 09 at 4:15pm) supra.
http://noconsensus.wordpress.com/2009/12/15/3649/
These graphs ought to highlight some important areas of agreement between the AGW-Consensus crowd (e.g. here at RealClimate) and the Skeptics (e.g. over at ClimateAudit).
The red traces in those two graphs depict the count of stations with long-term records, by Ryan O’s selection criteria (he plausibly claims that many other cuts would give the same general shape). The post-1990 decline in the count of GHCN stations with long-term temperature series is truly shocking. As the stakes in Climate Change become higher, data quantity nosedives. Huh?!?
Everyone ought to agree on the urgency of a few simple measures.
Raw data should be collected and made public in as transparent a fashion as possible. As much metadata (siting, history of adjustments, photo of site, etc.) as possible should be included. These sorts of databases are essential for identifying and quantitating real trends.
The many station locations dropped from the GHCN between 1990 and ~2005 should be salvaged. Happily, it seems that in many cases, data collection continued past the dropoff point; the lapse was in collation. Most of those records could be backfilled.
For those stations that were physically abandoned, most could be reclaimed, and observations re-started in 2010. The addition of metadata would allow factors such as increased urbanization to be taken into account.
This latter initiative wouldn’t help today or tomorrow. But what if “the science isn’t settled” ten years from now? A much-expanded set of long-term records would be a huge plus, as far as improved understanding of climate.
Funding of such projects by NOAA, NSF, WMO, and similar agencies would seem penny-wise and pound-wise to me.
[Response: See here. – gavin]
#92 – Norman, for a recent comment on the idea of ‘Arctic cooling’, have a look at Jeff Masters’ blog and the references cited therein, on the subject of the emergence of the ‘Arctic Dipole’:
http://www.wunderground.com/blog/JeffMasters/archive.html?year=2009&month=12
NOTE: you need to look at the second article in the list for December, the first being on movement rates of glaciers, hot off the press from AGU!
#110 – Dale, just for your interest, I am myself a geoscientist and have a right of centre political perspective, and I also support the climate science i.e. find it well argued and credible. I think that we should if possible be very careful not to label those persuaded by the science as being only of a liberal or left wing persuasion, as the argument then gets dumped into a simplistic left vs right split, which will get us nowhere.
I also try to ‘think outside the box’; indeed, most of my modelling, meta-modelling and experiment design work involves trying to tackle big problems in new ways …
# Eric – very good post, makes the point very neatly.
“if there is no statistically significant difference between raw and processed data, why process/correct it at all? ”
Because there IS a difference in detail.
5 gallons distributed around 50 bottles with a varying amount in each totalling 5 gallons doesn’t make a difference to how much water there is, when asking “has anyone spilt water and messed up our experiment”, but it DOES become important that the variation be kept when you go on to do the experiment: “which bottle gets drunk first, and does the volume of drink in it make a difference?”
My area of expertise provides raw, unmanipulated temperature data from sites around that world that is completely independent of the surface air temperature data record. The American Society of Limnology and Oceanography (ASLO) has just published a special issue of Limnology and Oceanography titled: “Lakes and Reservoirs as Sentinels, Integrators and Regulators of Climate Change. A large majority of the papers are available as “open access” at the ASLO website (authors pay an optional fee for open access).
http://www.aslo.org/lo/toc/vol_54/issue_6_part_2/index.html
Very high quality data for the last 20-40 years show strong warming of the surface waters of large lakes from California, Washington state, Michigan, Sweden, Switzerland, Sweden, Germany, Switzerland, central Africa and Siberia and other locations (reviewed byAdrian et al. 2009). If the global air temperature data were somehow biased, flawed or even fraudulent, this should be obvious from the data on lake temperatures. However, the agreement is excellent. Since many of the lake data are from northern latitudes, the rates of warming in lakes are, for the most part, considerably higher than the means for global air temperatures.
A paper on the physics of warming in Lake Tanganyika, (the second or third largest lake in the world by volume) documents warming since 1913 and shows, the close relation of its temperature anomaly with local air temperature, which in turn is closely linked with southern hemisphere and global air temperature anomalies (Verburg and Hecky 2009).
Temperature per se is only a small part of the papers presented in the special volume. Lakes are sensitive to changes in climate and this is clear from strong signals in water chemistry and food chains that are apparent from recent warming as well as the record of changes over thousands of years from sediments. For example, my paper shows how changes in temperature stratification has indirectly altered the food chain of a large lake on the Italian-Swiss border (Manca and DeMott 2009).
Objectivity is indeed “picking sites at random”… not sifting through the data for strong correlations.
Also, can Eric Steig be considered an objective evaluator?
[Response: It doesn’t matter whether I’m objective. The methods are what’s objective here. If you don’t believe me, do it yourself.–eric]
“It’s not what you, the western public, our governments, etc think of these emails, etc. It’s what China, India et al make of it all.”
I strongly doubt that leaders in China are concerned about the right wing blogs of the English speaking world. If you know anything about Chinese culture, you would know that teachers, scientists and engineers are held in much greater esteem than in the US. This may be why Chinese have placed so much emphasis on math and science in their educational system. These attitudes, coupled with an authoritarian polical system, is allowing the Chinese adopt new findings in science and technology very quickly.
One substantial advantage in the west, is that western scientists are probably faster to accept new ideas that are contrary to those of earlier scientists and professors. In my experience, scientists in China, Japan and even in continental Europe show more respect for tradition and are more reluctant to disagree with the publications of their teachers.
#118: Bill DeMott:
Thanks. I will likely be adding some of this data to my Modern Day Climate Change page.
Hi gavin
In your response 100, did you see that the data of NASA Antarctic stations are not updated after “for the best”,1992?
In french we say : “donner des bâtons pour se faire battre”
[Response: a) not NASA stations, b) not the only source of data, and c) On peut s’amuser à chercher un poil aux oeufs, mais ca sert à rien. – gavin]
Just a commentary on your argument to prove your case.
(I am skeptical of global warming but I’m also skeptical that there isn’t global warming. I’m generally looking for something convincing either way and generally walk out empty-handed because I don’t find the evidence convincing enough. I do think there is a lack of transparency on the pro-global-warming side which I find unfortunate, and the row over the emails has only reinforced this perception)
Your article is essentially in response to this quotation you cited:
“Trying to prevent skeptics from seeing the raw data was always a questionable strategy, scientifically.”
in response to which you wrote:
“The implication is that something secretive and possibly nefarious has been afoot in the way data have been handled, and that the validity of key data products (especially those produced by CRU) is suspect on these grounds. This is simply not the case.”
There are problems with the argument you chose to use, which is to show how statistically that the warming trend is identical with or without data correction for a subset of all weather stations.
1. You haven’t indicated which weather stations you selected. One of the things that has been revealed in this controversy is that “random” and “cherrypicking” are sometimes intertwined (on either side of the issue). Basically, because you’re not providing which weather stations you selected, it’s impossible to actually replicate what you’ve done.
2. The link to the CRU is dead-ish. There’s no data there, the reader is redirected to the Univeristy’s website.
3. That the trends are identical for a subset of weather data used for the global average temperature does not disprove that there might be something wrong with the way that each weather data is used, nor does it disprove that there might be something wrong with the way that all weather data is combined into the HADCRU global temperature index.
4. I work in building energy simulation, and one thing I know is that urban heat island effect is poorly understood even today. We use weather files in simulating how buildings will behave, but a big wild card we don’t account for is how such buildings will behave in the actual locale in which the building will be in (which is usually in a dense urban location). I’m at pains to figure out how a known, transient, and poorly understood phenomenon can simply be “compensated out” in all weather station data. Your figures show that essentially urban heat island isn’t compensated for in the weather stations you selected, however the CRU has stated that te global index is in fact compensated for it. If they’re indeed not compensated for, I can’t see how global temperatures can possibly be correct. If they’re compensated elsewhere, then the study you have done does not disprove that their homogenisation is flawed or perhaps nefarious.
5. Your argument doesn’t quite constitute a complete explanation as to why some CRU outsiders are trying to get information from CRU insiders (i.e. the quote you were responding to in the first place) and why those CRU insiders are doing to do everything *not* to give it to the CRU outsiders. Everyone’s first interpretation of someone acting evasive is that this someone has something to hide, and the hacked emails (if genuine) reveal that they are actively being evasive with their data. If it’s of no consequence, I can’t understand why they would be so intent on keeping it to themselves.
PS: I’m not trying to make a polemic out of this, I’m just honestly trying to understand what’s going on here and resolve conflicting perceptions I have, so please bear that in mind if you do intend to respond.
[Response: The point of the post is that anyone can do the same analysis. I don’t need to provide you with any data for you to do this. Go to the web, get the raw data and the CRU data, and compare them. You’ll get the same results we did. I gave you the links, the math is trivial. Why make it complicated when it isn’t?–eric]
Because at the single station levels differences can be still significative and important, either in warming direction or cooling direction. Given that all climate model now try to do a better job at prediction local variations, to start with accurate data is imporant.
Verberg and Hecky oddly do not cite Tierney et. al. Science Vol. 322, No. 5899, pp. 252-255, 10 October 2008. This would have shown their data to be the the normal range for the last 60,000 years.
#99 And then Ian George would require the calibration tests of the thermometers. And the manufacturing report.
Which should be included in any report.
Barton Paul Levenson wrote: “… in 2500 … the survivors won’t have the capital, infrastructure and functioning economy enough to build a domed city in Antarctica.”
Which suggests that it would be prudent for those who currently have the wealth and power to command such resources to get started on building their nuclear-powered, climate-controlled domed cities now.
Of course, those cities will only be able to house a tiny fraction of the Earth’s population. Perhaps the “top one percent”.
Seems like a good place for fossil fuel corporation CEOs to invest some of the hundreds of billions of dollars in profit they expect to gain from several more decades of business-as-usual consumption of their products.
> Everyone ought to agree on the urgency of a few simple measures.
> …
> [Response: See here. – gavin]
That stated the need; lest someone leap on that 1999 statement, it’s worth noting that the new system to address those needs, the CRN, went operational in 2004:
http://www.publicaffairs.noaa.gov/releases2004/jan04/noaa04-004.html
They’re running in parallel; the historical network, the updated historical sites, and the new CRN stations — always smart with new systems: http://www.ncdc.noaa.gov/crn/observations.htm
ZT (87):
“I am surprised to read that the heat capturing effects of asphalt are so easily dismissed as myth – there must be some effect – as anyone who walks across a parking lot in the summer would attest. But perhaps this is indeed vanishingly small. ”
The myth is not that asphalt traps heat (it does), the myth is that asphalt has biased the temperature readings and produced a false warming trend. As the links demonstrated, the dataset handlers are able to properly account for asphalt creap and produce a quality dataset.
The problem with some of the AGW deniers (e.g. those who keep claiming the dataset is biased) is that they seem to think that scientists are too stupid to recognize potential issues like asphalt creap and account for it.
Someone has the idea:
http://www.globalwarmingimages.net/index.html
But where are the pictures?
Here’s an example of what I’m talking about. You can clearly see the before and after.
http://thrivingoceans.org/index.php/2009/05/27/coral-bleaching-and-macroalgae-infiltration/
Another:
http://snf.fra.affrc.go.jp/English/wwwsupl/supl1j1.jpg from http://snf.fra.affrc.go.jp/English/wwwsupl/supl1.htm
Another:
http://sites.google.com/site/vulnerabilityadaptation/_/rsrc/1240428658864/elyce/Coral%20Bleaching.jpg from http://sites.google.com/site/vulnerabilityadaptation/elyce
I found these on Google images (don’t know anything about their possible copyrights: http://tinyurl.com/yfxbxqt
Steve Bloom #81: Nice but the focus doesn’t seem to be quite there.
Giorgio Gilestro #113: “77, 78. I like the idea of the pictures. It may appeal to a lot of people. Is there a collection of beautiful pictures big enough to make a screensaver?
Gavin has a book but AFAIK there’s not an online version.
Would be a fun project for someone (not me), even funded research?
I asked another question on asphalt – was it rejected or inappropriate?
Many thanks for the links on asphalt not producing warming. I am surprised to read that the heat capturing effects of asphalt are so easily dismissed as myth – there must be some effect – as anyone who walks across a parking lot in the summer would attest. But perhaps this is indeed vanishingly small. However, I think that the records do show that stations which are not moved and are not in developed areas (and therefore do not have an increase of asphalt around them per unit time) show a lower warming rate. Has anyone investigated this – or do the statistics not permit a conclusion on such small data sets?
John (114):
“if there is no statistically significant difference between raw and processed data, why process/correct it at all?”
The more data the better. This is generally true in all fields of science, not just climate science. If data, even though it may not be perfect, adds to a better understanding of a field of study it would be foolish to throw it away or ignore it. It would also be foolish to not use valid methods to improve the quality of the data, when there are known issues that can easily be corrected.
In my own experience (not climate related), I’ve had to go back and correct large subsets of measurments because of some mistake in the calibration process. Without doing so the entire experiment (quite expensive) would have been worthless. But using a simple (and demonstratably correct) method, the data was saved and the end result was a successful experiment and increased knowledge about the process.
While excellent approximations of global warming trends can be determined from just 100 well placed measuring stations, such a limited dataset wouldn’t be at all helpful for regional climate study. And such a limited set of measurements would also be attacked by deniars claiming “they’re hiding data that might prove it’s actually cooling”.
Revkin to leave NYT.
http://www.cjr.org/the_observatory/revkin_taking_nyt_buyout.php
You folks should use the WSJ opinion piece by Mike Hulme as one of your blog postings. I know this is off-topic, but then my comments rarely make it through anyway (about 1 of 5), so I thought this might be a good way to communicate my suggestion to you. The articles is at:
http://online.wsj.com/article/SB10001424052748704107104574571613215771336.html
AMac says:
“The post-1990 decline in the count of GHCN stations with long-term temperature series is truly shocking. As the stakes in Climate Change become higher, data quantity nosedives. Huh?!?”
Just a wild guess on my part, but could that not simply be a result of the collapse of the USSR? I believe many, many things in the former Soviet Union states lost much of their funding after that event, presumably Met services were no different?
thanks gavin
“je ne chercbe pas de poil aux oeufs’ I’m not a denialist…(you can catch a glance on my site if you want)
But I prefer your second link.
The pushback continues. Someone needs to go point-by-point through this
http://www.dailyexpress.co.uk/posts/view/146138
[Response: They have. – gavin]
@121,132
Wow, Gavin knows french expressions that we (= french people) don’t ! ;)
[Response: On peut toujours apprende quelque chose de nouveau…. – gavin]
Gavin, in your link to antarctic stations, I noted not a single urban site. What are you guys hiding?
#110 Dale
I agree with Nick O. I’m politically Centrist but personally conservative. Science itself is agnostic. We all live in one world and the development of the polio vaccine was unaffected by the fact that their are two dominant political parties in America.
The polio vaccine was developed by a guy in Pennsylvania pretending he was the polio virus and wondering what in the human body might harm him. It was a thought experiment that did not worry about the politics of left or right but the battlefield within the human body.
RE: Fuller’s article. The Examiner.com is not the San Francisco Examiner. It’s a nationwide vanity opinion site with no editorial input at all and 0 credibility in all regards.
[Response: The point is that individual stations are being cherry picked. An honest assessment would pick sites at random, as we have done. It is of course possible that some stations have problems that CRU didn’t catch. Picking on those isn’t objective.–eric]
At what point does the sample size of “picked cherries” become large enough that it ceases to be “cherry picking?”
[Response: Sure, but this would be repeating what CRU, GISS, etc. have been doing for years and years. If you want to reinvent the wheel, go ahead. Oh, while we’re at it, let’s redo the epidemiology on smoking and cancer. Until that’s done, let’s all take up smoking. After all, who can trust the corrupted peer-reviewed literature in leftist journals like the New England Journal of Medicine?–eric]
Are you implying that the relationship between human-produced CO2 and global climate is identical to the relationship between habitual smoking and cancer/heart disease?
Thank you for your comment Ken. I am absolutely only interested in learning – not denying. I am not an expert in this field – as I am sure that you can tell.
If urbanization is not effecting thermometer readings – why is the land temperature increasing faster than the ocean? Surely, if the global temperature was increasing – would not both land and sea increase at the same rate?
If there is a growing divergence between land and sea (?) that would imply either that the land and sea are not in thermal equilibrium (I would have thought that they are because wind is pretty effective in cooling and warming) or that the land measurements capture additional effects.
Comments?
(This is a question raised by Tom Wigley, of course):
http://www.eastangliaemails.com/emails.php?eid=1067&filename=1257546975.txt
[Response: a) not NASA stations, b) not the only source of data, and c) On peut s’amuser à chercher un poil aux oeufs, mais ca sert à rien. – gavin]
Just out of curiosity, have you actually clicked on those station links? Those are some of the saddest looking graphs I’ve ever seen – multiple years missing, etc.
You originally said:
[And no, the measurements from Antarctic (used by GISTEMP for instance) don’t just come from one station. Why do you automatically believe people who have been shown over and again to be misrepresenting the true situation? – gavin]
Do you think you’ve shown that there are multiple contintental antarctic stations with good data sets?
I haven’t found any on either of these links.
Well there we have it then. Homogenization adds no statistically relevant value in deriving the global mean temperature.
Do the science a favor by presenting a simpler argument using just the raw data.
#95 Doug Bostrom — a most gratifying excoriation of the denier army! Diplomacy and tact are manifestly ineffective on that crowd. Your blunt but elegant language may not have any more effect on your target, but it impressed me mightily. My profound admiration and thanks 8^)!
“so I guess my question now is: if there is no statistically significant difference between raw and processed data, why process/correct it at all? why not leave the data that has no problems alone and only process the ones with clear gaps, errors, etc.?”
There are a couple of answers to this. The simple one is, “Because scientists are obsessive about being as accurate as possible.” It is very common to see data corrections that do not alter the conclusions. I’ve had students ask me, “Why should I have to redo the analysis if you expect that it won’t alter the conclusions?” The answer being, “Because this one you used was wrong; this one is right.”
A more subtle explanation is that you don’t know whether correction/normalization of the data will make a difference until you do it. But it is an error to make your decision of what analysis method to use contingent upon the data. Although we tend to associate cherry-picking of data with denialists, it is an error that all scientists must guard against. If you allow yourself to change your mind about whether to use a particular data manipulation once you have seen the result, or if you apply a correction in some instances but not others, you open the door to the possibility of selecting (perhaps unconsciously) the data manipulation that gives the results that you would like to see. After-the-fact analyses such as the one presented by Eric are routinely done because we like to have an idea of how large the impact of data normalization on our conclusions actually is. While we endeavor to choose the most appropriate analysis method for our data, we’d like the data to be robust enough that the conclusions are not dependent upon how the data is corrected or normalized. If adjustments to data make a critical difference, then we are going to worry a lot more about those correction/normalization methods, or look for alternative data that can validate our conclusions without requiring the same correction/normalization.
[Response: Try reading the post again, slowly. What we did is what we said we did, plain and simple. If you want more details, go to the links we provided, which, yes, are the raw data. You are really trying hard to find fault where there isn’t any.–eric]
Isn’t finding fault what peer review is all about? When the authors use the term “objective”, then set out to prove the “concensus” then it’s not objective. That you find the same answer as CRU only prove’s Pielke’s judgement that Phil Jones’ assertion that CRU isn’t independent of GISS et al false.
Rather than picking a random set and applying the same “concensus” corrections as CRU does, an “objective” analysis would be to pick the most pristine sites (those with the least UHI correction required, those with the least TOB correction required, etc…in otherwords those with the least “Anthropogenic” fiddling). If you got the same answer then, that would “objectively” demonstrate that the “Anthropogenic” biases are not being introduced in the temperature trend.
In your graphs you show the similar trends as CRU, but in the distributons (the figures to the right) the means are close, but the widths of the distributions are not. Why not? Sounds like something a peer reviewer would be quick to ask about.
When do you plan to submit this work to a journal, and when you do, please keep us posted with the peer review comments as it works it’s way through the peer review process.
#139 AC
I think you’re missing the point. Add the total internal forcing + blackbody and relative constants + pre v. post industrial forcing components (i.e. GHG levels and capacities imposing on internal heat trapping of atmosphere) + natural variation (short and long) + atmospheric lifetime of Co2 + etc. = relatively clear picture of current and expected warming which translates to infrastructure and capacity.
By looking at the big picture, one can find ways to see reasonably that data gaps pose an interesting problem but insignificant in consideration of the overarching AGW picture.
Worrying about a few needles in the haystack is less significant when ones main concern is understanding that it is a haystack.
This is totally OT, but check out the comments on this article on the Daily Tech blog (which George Will cites as a “scientific” source)
http://www.dailytech.com/Farmers+Say+Global+Warming+Fix+is+Full+of+Manure+Promise+Big+Cuts/article17153.htm
Truly scary stuff. I can only imagine what you all go through on a daily basis.
Keep up the good work and please BE CAREFUL OUT THERE!