This is a continuation of the last thread which is getting a little unwieldy. The emails cover a 13 year period in which many things happened, and very few people are up to speed on some of the long-buried issues. So to save some time, I’ve pulled a few bits out of the comment thread that shed some light on some of the context which is missing in some of the discussion of various emails.
- Trenberth: You need to read his recent paper on quantifying the current changes in the Earth’s energy budget to realise why he is concerned about our inability currently to track small year-to-year variations in the radiative fluxes.
- Wigley: The concern with sea surface temperatures in the 1940s stems from the paper by Thompson et al (2007) which identified a spurious discontinuity in ocean temperatures. The impact of this has not yet been fully corrected for in the HadSST data set, but people still want to assess what impact it might have on any work that used the original data.
- Climate Research and peer-review: You should read about the issues from the editors (Claire Goodess, Hans von Storch) who resigned because of a breakdown of the peer review process at that journal, that came to light with the particularly egregious (and well-publicised) paper by Soon and Baliunas (2003). The publisher’s assessment is here.
Update: Pulling out some of the common points being raised in the comments.
- HARRY_read_me.txt. This is a 4 year-long work log of Ian (Harry) Harris who was working to upgrade the documentation, metadata and databases associated with the legacy CRU TS 2.1 product, which is not the same as the HadCRUT data (see Mitchell and Jones, 2003 for details). The CSU TS 3.0 is available now (via ClimateExplorer for instance), and so presumably the database problems got fixed. Anyone who has ever worked on constructing a database from dozens of individual, sometimes contradictory and inconsistently formatted datasets will share his evident frustration with how tedious that can be.
- “Redefine the peer-reviewed literature!” . Nobody actually gets to do that, and both papers discussed in that comment – McKitrick and Michaels (2004) and Kalnay and Cai (2003) were both cited and discussed in Chapter
2 of3 the IPCC AR4 report. As an aside, neither has stood the test of time. - “Declines” in the MXD record. This decline was
hiddenwritten up in Nature in 1998 where the authors suggested not using the post 1960 data. Their actual programs (in IDL script), unsurprisingly warn against using post 1960 data. Added: Note that the ‘hide the decline’ comment was made in 1999 – 10 years ago, and has no connection whatsoever to more recent instrumental records. - CRU data accessibility. From the date of the first FOI request to CRU (in 2007), it has been made abundantly clear that the main impediment to releasing the whole CRU archive is the small % of it that was given to CRU on the understanding it wouldn’t be passed on to third parties. Those restrictions are in place because of the originating organisations (the various National Met. Services) around the world and are not CRU’s to break. As of Nov 13, the response to the umpteenth FOI request for the same data met with exactly the same response. This is an unfortunate situation, and pressure should be brought to bear on the National Met Services to release CRU from that obligation. It is not however the fault of CRU. The vast majority of the data in the HadCRU records is publicly available from GHCN (v2.mean.Z).
- Suggestions that FOI-related material be deleted … are ill-advised even if not carried out. What is and is not responsive and deliverable to an FOI request is however a subject that it is very appropriate to discuss.
- Fudge factors (update) IDL code in the some of the attached files calculates and applies an artificial ‘fudge factor’ to the MXD proxies to artificially eliminate the ‘divergence pattern’. This was done for a set of experiments reported in this submitted 2004 draft by Osborn and colleagues but which was never published. Section 4.3 explains the rationale very clearly which was to test the sensitivity of the calibration of the MXD proxies should the divergence end up being anthropogenic. It has nothing to do with any temperature record, has not been used in any published reconstruction and is not the source of any hockey stick blade anywhere.
Further update: This comment from Halldór Björnsson of the Icelandic Met. Service goes right to the heart of the accessibility issue:
Re: CRU data accessibility.
National Meteorological Services (NMSs) have different rules on data exchange. The World Meteorological Organization (WMO) organizes the exchange of “basic data”, i.e. data that are needed for weather forecasts. For details on these see WMO resolution number 40 (see http://bit.ly/8jOjX1).
This document acknowledges that WMO member states can place restrictions on the dissemination of data to third parties “for reasons such as national laws or costs of production”. These restrictions are only supposed to apply to commercial use, the research and education community is supposed to have free access to all the data.
Now, for researchers this sounds open and fine. In practice it hasn’t proved to be so.
Most NMSs also can distribute all sorts of data that are classified as “additional data and products”. Restrictions can be placed on these. These special data and products (which can range from regular weather data from a specific station to maps of rain intensity based on satellite and radar data). Many nations do place restrictions on such data (see link for additional data on above WMO-40 webpage for details).
The reasons for restricting access is often commercial, NMSs are often required by law to have substantial income from commercial sources, in other cases it can be for national security reasons, but in many cases (in my experience) the reasons simply seem to be “because we can”.
What has this got to do with CRU? The data that CRU needs for their data base comes from entities that restrict access to much of their data. And even better, since the UK has submitted an exception for additional data, some nations that otherwise would provide data without question will not provide data to the UK. I know this from experience, since my nation (Iceland) did send in such conditions and for years I had problem getting certain data from the US.
The ideal, that all data should be free and open is unfortunately not adhered to by a large portion of the meteorological community. Probably only a small portion of the CRU data is “locked” but the end effect is that all their data becomes closed. It is not their fault, and I am sure that they dislike them as much as any other researcher who has tried to get access to all data from stations in region X in country Y.
These restrictions end up by wasting resources and hurting everyone. The research community (CRU included) and the public are the victims. If you don’t like it, write to you NMSs and urge them to open all their data.
I can update (further) this if there is demand. Please let me know in the comments, which, as always, should be substantive, non-insulting and on topic.
Comments continue here.
Chris @424 It’s called GLOBAL CLIMATE CHANGE. Hows about we structure the bet a little differently. How about $200 that the next decade will be warmer globally on average than the current one? That is, that the current warming trend will continue. Up for that? Willing to give odds?
Gavin, someone, above, asked whether you were doing any of this moderating during your working hours for the US government.. It’s a fair question, and as you probably know, now the subject of a Freedom of Information Act Request.
Are you?
Neil22 (445) — And yet it warms:
http://news.softpedia.com/news/Fast-Melting-Glaciers-Expose-7-000-Years-Old-Fossil-Forest-69719.shtml
Nature doesn’t care what you think about it.
Since this is the site by and for real climate scientists, would somebody please explain the scientific basis for applying the “fudge factor” in the tree ring data programs below:
Lets take a look at exactly how the FORTRAN (glad to see someone still appreciates the
original programming language) programs briffa_sep98_e.pro & briffa_sep98_d.pro
“****** APPLIES A VERY ARTIFICIAL CORRECTION FOR DECLINE*********”
my comments are within [[[ ]]], otherwise the code snippets are as they appear in the file.
;
; PLOTS ‘ALL’ REGION MXD timeseries from age banded and from hugershoff
; standardised datasets.
; Reads Harry’s regional timeseries and outputs the 1600-1992 portion
; with missing values set appropriately. Uses mxd, and just the
; “all band” timeseries
;****** APPLIES A VERY ARTIFICIAL CORRECTION FOR DECLINE*********
;
yrloc=[1400,findgen(19)*5.+1904]
[[[ this creates 20 consequtive 5 year subsets (possibly averaged) of the tree ring data by date starting in year 1904 ]]]
valadj=[0.,0.,0.,0.,0.,-0.1,-0.25,-0.3,0.,-0.1,0.3,0.8,1.2,1.7,2.5,2.6,2.6,
2.6,2.6,2.6]*0.75 ; fudge factor
[[[ these are the 20 different “fudge factor(s)”-the programmer’s words not mine – to be applied to the 20 different subsets of data, so here are those fudge factors with the corresponding years for the 20 consequtive 5 year periods:
Year Fudge Factor
1904 0
1909 0
1914 0
1919 0
1924 0
1929 -0.1
1934 -0.25
1939 -0.3
1944 0
1949 -0.1
1954 0.3
1959 0.8
1964 1.2
1969 1.7
1974 2.5
1979 2.6
1984 2.6
1989 2.6
1994 2.6
1999 2.6
a little further down the program adjusts the 20 datasets with the corresponding fudge factors: ]]]
;
; APPLY ARTIFICIAL CORRECTION
;
yearlyadj=interpol(valadj,yrloc,x)
densall=densall+yearlyadj
[[[ So, we leave the data alone from 1904-1928, adjust downward for 1929-1943, leave the same
for 1944-1948, adjust down for 1949-1953, and then, whoa, start an exponential fudge upward (guess that would be the “VERY ARTIFICIAL CORRECTION FOR DECLINE” noted by the programmer).
[edit]
[Response: Without any context, you have no basis for your statements. What was this for? what paper did it appear in? was it even ever used? Track that down and then we can talk. – gavin]
RichardC wrote:
“The physics are pretty simple – CO2 reflects infrared. The planet radiates heat as light from the Sun bounces off of it. The more CO2 the more radiated heat stays here and less radiates to space. Result – warming. None of that is undermined by anything in the hacked files. How quickly the planet will heat under what CO2 concentrations is not pinned down yet to absolute certainty yet, but these files neither significantly contribute to nor really undermine the efforts to pin that down”
The physics are not in question and never really has been. Rather, the scientific question orbits around the significance of the anthropogenic effect. The political question is whether we will reorder society in order to counter the [unproven] consequences predicted by computer models.
Climate scientists are reaping this whirlwind because they have allowed the Al Gores of the world to [prematurely?] declare the scientific question closed while using scare tactics to advance their agenda.
caerbannog says: 24 November 2009 at 7:0 PM
… “nonhomogenized” has gone through the emails and has demonstrated that most of the email messages have been subjected to creationist-style quote-mining:
http://www.reddit.com/r/science/comments/a6zdw/summary_of_notable_emails_from_the_cru_hacking
Bump. It’s well done. I hope it gets pulled together into a single reference with index.
Ray Ladbury
“So the discovery that humans are causing climate change is a by-product of climate science. The desire to save the planet is a by-product of being human. Try it sometime.
How about we try this? The discovery that the globe is warming is yours. Let someone else ‘desire’ to save the globe. You may believe that no scientist is or can be truly ‘disinterested’, but that is no excuse to gleefully jump in and become an interested party.
Gavin
“…This attitude is so completely abhorrent to me, that I find it almost incomprehensible that you genuinely think it is a defensible intellectual position.”
It is perfectly defensible.
[edit – OT]
To do nothing is all the more appropriate considering the dubious parallels between the above two examples and what’s going on with AGW. Especially when we know that it is the poor of the world who will pay the price, whether it be because of global warming or the bill to cool the globe.
Whether the globe is warming or not, is decided by a certain set of rules (those of science). What to do about it, or whether at all anything should be done is, by another. It is not watertight, but there should be no revolving door between the two either.
Happy holidays!
[Response: You have changed your argument, from one that professed no care even as a human being for the fate of the planet, to now discussing the best method to prevent harm. These are completely antithetical. If you genuinely believed your first argument, there would have been no hope for you. If instead, you now believe your second argument – that there are decisions to be made on how best to prevent harm, well then we can start talking. Since I made no statements on specific methods required to deal with the risk of climate change above, I’m not sure why you think my work on the science is somehow compromised by my non-expressed opinions. – gavin]
Hello, trying to learn more here, hope you don’t mind.
Reading through the available material, the questions from Dr. Keiller struck me as particularly insightful and that they get right to the heart of the matter in dispute. These questions are:
1) Are the reconstructions sensitive to the removal of either the Yamal data and Strip pine bristlecones, either when present singly or in combination?
[Response: Yes. The further back you go the more important the long tree ring records are, but the general patterns are robust. – gavin]
2) Why these series, when incorporated with white noise as a background, can still produce a Hockey-Stick shaped graph if they have… a low individual weighting?
[Response: Not sure what you are actually talking about here. – gavin]
I seem to be having an awful lot of trouble getting a straight answer to this question, so if there is no other answer on record I’d appreciate if one of the experts here took a stab at it. Thanks!
Thanks for your answer in 406. I think that the more transparent you can be in showing exactly how the peer review process works, the better. Showing the changes made during the process is one way of doing so.
People aren’t accusing you of withholding data. They are accusing CRU. Have you read this post by Willis Eschenbach? It is quite persuasive in arguing that CRU withheld data from him (and others) that should not have been withheld.
Although your indefatigable energy is answering stupid critics is impressive, I would focus your efforts on the most impressive/coherent complaints. How would you answer Eschenbach? This is a complex enough topic that it might merit a new thread.
PS. We enjoyed dinner courtesy of SAC several years ago. I have been following this debate ever since. Kudos to you for all your efforts in increasing code transparency and data-sharing.
[Response: Eschenbach was told in 2007 exactly why they couldn’t release the stuff that included the restricted data from the NMSs but that the vast majority of the data was online already. Nothing has changed except that CRU have been harassed with perhaps 100 vexatious FOI requests for exactly the same thing, and which received (unsurprisingly) exactly the same response. How this is supposed to encourage CRU to work together with these people to get the NMSs to rescind their restrictions, I’m really not sure. Funnily enough, continually threatening people with lawsuits is not something designed to get them to go out of their way to help you. It is instead a recipe for them doing the absolute minimum, and even that grugdingly. Not smart maybe, but eminently understandable. – gavin. (PS. Good dinner!).]
This is a PR disaster and the climate scientists have themselves to blame. What’s amazing to me is the continued arrogance in the face of this growing scandal.
Complete transparency is the only way to recover your damaged reputations. Release everything and explain how you arrive at your conclusions so that nonscientists can undersand.
Rad Ladbury And as has been said REPEATEDLY, the reason the Hadley center cannot release their raw data is because of prior agreements.
“Trust me” doesn’t sound like science. If they can’t release their data the papers should be rejected.
[Response: Let’s start with all the Spencer and Christy papers perhaps? Or can you point me to their source code and raw data? And if not, why not? – gavin]
Hi Gavin
What annoys me about people in this debate are the assumptions that all AGW sceptics think GW believers think the apocalypse is upon us, and that all AGW believers think all AGW sceptics think zero AGW has occured.
The truth, is somewhere in between. This is not a relgious or moralistic debate, as some people are determined to make this out to be.
Personally, I think there has been AGW since 1850, combined with us being in a natural warming period. How much more will temperatures rise? Well given they have risen by 0.76 since the beginning of the industrial revolution, I am thinking another degree, or two at most. (basing this on what has happened already).
The emails found, are not good, and paint a picture of a AGW cult to some degree, in that they would prefer to see warming and catastrophic consequences to be proven right. I am afraid this is not science.
This does not discredit AGW, nor does it mean we should hold off on action in reducing emissions, but what it does show is that people are conspiring to show things in as catastrophic a light as possible to scare people into action, worried that unless they create worst case scenario’s, nothing will be done.
[Response: This is nonsense and there no evidence whatsoever for this statement from the emails. – gavin]
We know that a rise of 1.7-2.7 degrees will cause major problems (though not the end of the world), but to the man on the street it may not seem like much of a big deal, which is why we get these sexed up graphs to try and get the layman on side.
DJK (#392) says: “General circulation models all seem to assign positive values to these feedbacks, yet there is new evidence that they may be closer to zero or even negative.”
What you really mean is that if you can cherry-pick a few papers from the recent literature that make this claim. However, you can find many more papers from the literature, both very recent and less so, that support the positive values to the feedbacks. You don’t just get to ignore all of the evidence that you don’t like and look at only that which you fancy.
Furthermore, the supposed evidence seems pretty weak: If you are referring to Lindzen and Choi, even Roy Spencer is skeptical of their work: http://wattsupwiththat.com/2009/11/03/spencer-on-lindzen-and-choi-climate-feedback-paper/ If you are referring to Roy Spencer himself, he seems to be a moving target with claims of a climate sensitivity of ~ 0.7 C but his latest claim was a climate sensitivity (on a timescale relevant to the issue at hand) is on the order of 1.6 to 2.0 C, which borders the IPCC “likely” range of 2.0 to 4.5 C. And, if you are referring to Christopher Monckton, well,… never mind!
Overall, the body of the evidence still favors the range given by the IPCC. Could it be less? Possibly. Could it be greater (or could there be some nasty surprise of some sort that we have not even contemplated)? Possibly too.
Re: Response to post #392
First, thank you for the response. I’ve included the two points I made in the original post here for reference:
“Based on what I have read, there are only two main points of contention in the debate:
1) The exact sensitivity of the climate to increases in carbon dioxide relative to other forcing mechanisms (i.e. the thrust of the question in post #285.)
2) The state of our current climate relative to the paleoclimate; i.e. are we experiencing unprecedented temperatures relative to, say, the Medieval Warm Period, or is our current climate within a historical range?”
___________________________________________
Now, my comments to your response:
[Response: Don’t talk wet. Point 1) is a valid goal in science and multiple groups all around the world are focussed on it.]
Your comment implies that you agree with me that vigorous debate on this point remains. That’s great, and exactly what I would expect from the scientific community. The question that follows is: If this matter has not been settled scientifically, why are we now in the process of transforming our society on the basis of the unfledged results?
If I’m misreading you on this, then I’d refer you back to the question raised by the poster in #285:
“Where is the link, the data or other evidence, that shows warming is -driven- by CO2 increase – especially when it seems, when looking at longer term temp and CO2 record, that CO2 is re-active to temp change and not leading?”
[Your point 2) is a complete red herring. It is irrelevant whether the MWP was 0.1 deg C warmer than the late 20th C or 0.1 deg C cooler. If you want a time in the past that was warmer than today, try the last interglacial, or the Pliocene, or the Eocene or the Cretaceous. There is no ambiguity there.]
I agree that point #2 is largely irrelevant from a purely scientific standpoint; i.e. the physics today is what it is regardless of what the climate was like in the past. I even commented to the effect that point #1 was the more important of the two. Unfortunately, rather than addressing point #1 in any depth (which is what I was hoping you’d do), you jumped all over point #2. Not sure why.
With that said, I would argue that point #2 has greater importance from a public perception standpoint; i.e. that the public would be more willing to buy into an AGW theory–and its associated policy implications–if it can be shown that temperature fluctuations in the paleoclimate were relatively benign relative to the present, and less so if it were shown to be more dynamic–agree?
[If you want evidence of non-CO2 forcings, look at the 8.2 kyr event or the mid-Holocene.]
Okay.
[Similarly. No conceivable revisions of the medieval temperatures are going to constrain either sensitivity, or solar forcing or internal variability – the differences are just too small to be useful.]
I never claimed that point #2 had any impact on point #1. Why refute a nonexistent assertion?
[As for M&M tackling the MWP, they have done nothing of the sort. They have never published a single reconstruction. Never mind one that goes back to medieval time.]
Again, I never made this claim. I said that they were focused on (not ‘tackling’) the state of our current climate relative to the paleoclimate. I concede that their methodology has been to criticize published reconstructions–most notably MBH 1998 and Briffa 2000, 2008.
[The issue is not one of shutting down debate – there is debate a-plenty at science conferences and workshops across the world. it’s that the ‘skeptics’ keep talking about non-issues – however loudly they declare that they are being repressed. – gavin]
My reading of the released e-mails suggests that “debate” does not take place quite as freely as you claim. I’ll grant you that some skeptics may deal in “non-issues”, but some–Richard Lindzen, for example–are busy exploring point #1, which you acknowledge is a valid goal.
My apologies if I continue to “talk wet”–whatever that means.
Hi there,
Is the validity of climate models tested by comparisons to the historical temperature record? In particular, the historical temperature record that is derived solely by proxy studies?
[Response: Sure. See fig 6.13 in AR4 Chapter 6 (bottom panel). – gavin]
As well, could someone point me to a historical reconstruction that does not graft instrumental temperature data? All I’m looking for is a study that does not suffer the pitfalls of divergence. Thanks
[Response: Any of them. See the NOAA list. – gavin]
#422 – Jim, so you think the Arctic will be ice free soon? Let’s start with looking at a average temperature of the Arctic since 1950. Go to http://www.rimfrost.no/
and click on the
Country/Region –Arctic
Weatherstation – RIMFROST – AVERAGE
This shows an initial drop in temps from 1950 to about 1970.A gradual rise from 1970 to ~1995, and a more noticeable increase from 1995 to ~2007, and now what appears to be a drop in temps. So depending on your point of view, one could say there is a gradual rise of about 1 deg., in the past 60 years, or there is part of a periodic oscillation with a period of about 50-60 years. This 50-60 year cycle seems to show up in a number of places. The figure below shows the E. England data from 1659-2008 with a 40 year Fourier filter. Again this 50-60 year oscillation shows up. In fact the pattern from 1950 forward, resembles the Rimfrost Arctic average.
http://www.imagenerd.com/uploads/t_est_28-bGGxs.gif
Looking at the Arctic sea ice information, while you references are interesting, I prefer looking at a “strip” chart presentation. I think you can get a lot more info, and it’s probably a carry over from using analog computers a few decades ago. The figure below is from the Cryosphere sea ice anomaly time history from 1979.
http://arctic.atmos.uiuc.edu/cryosphere/IMAGES/sea.ice.anomaly.timeseries.jpg
From the chart, it would appear the arctic sea ice remained relatively stable from 1979 to about 1995, after which there was a decline until about 2007. After that it appears, to increase. So if I were to draw a straight line from the fall of 1979 (0. mkm) to the fall of 2008 (-0.75 mkm), I would be hard pressed to say Arctic will disappear very soon. This is in spite of the increase of CO2. Seems to be a contradiction here.
#429 Brian, if CO2 is increasing so much, why is the Arctic sea ice “recovering”?
Anand Rajan KD said @ 1:19 PM “Legislation should impact on palpable realities, not things like ‘future of the planet’, ‘planet’s well-being’ and their ilk.”
I see, so if you became aware through observation that a certain disaster is likely to happen fairly soom, let’s say like, oh I don’t know, how about the imminent collapse of a major bridge (really one can think of any number of scenarios), your philosophy is that you would say nothing about it to anyone because it hasn’t actually happened yet right, it’s still in the future?
You also apparently believe that there should be no legislation protecting the environment. WOW! What does one say to that? Lucky for you though that others with more concern for their fellow man and creature have fought, and continue to fight, to ensure that you and yours have safe water to drink and clean air to breathe. That dumping of toxic waste in the vacant lot next to you is illegal because it could cause you harm. That you have as robust an EPA and FDA as possible. That you have national parks to visit should you ever need to get away from it all. That other species besides humans are allowed to exist, which interestingly enough ensures you own survival, etc.
“Wanting to care for the whole planet is just overarching hubris and power hunger in disguise.”
So you feel not an iota of appreciation or respect for the planet that gives you existance? Not an ounce of awe for the complex and diverse web of life that has evolved over the last three and a half billion years? Not a smidgen of feeling for the majesty and beauty of the only living planet that we know of and possibly the only one we may ever be able to inhabit (without protective gear)? You have no sense of of the wholey avoidable trajedy of the current sixth extinction? Of millions of years of evolution down the drain? Again, wow.
———–
Before I flew I was already aware of how small and vulnerable our planet is; but only when I saw it from space, in all its ineffable beauty and fragility, did I realize that human kind’s most urgent task is to cherish and preserve it for future generations. –Sigmund Jähn, Astronaut German Democratic Republic
Some new climate modelers have presented a newer simpler model looking at raw, untreated temperatures only, going back 150 years or so. They use a day over day moving average much like stock traders use (this metric has proved one of the most accurate historical indicators of stock performance). Their model’s conclusion, is categorically opposite to the warming predictions we regularly hear, that being that temperatures changes have not been extreme or drastic, and forecasting a century out look to stay the same.
Perfectly modeling every conceivable system that drives climate is an impossibility, I don’t care what you think you can do. Furthermore attempting to do so to predict future TEMPERATURES seems extremely foolish, when temperatures alone can be modeled much more accurately than the entire Earths energy profile.
So why all the complication? And can you refute the methodology of these new modelers (in principle, obviously you don’t have all the specifics) and why the complicated (and highly suspicious) way is better?
[Response: Nothing is more suspicious than people claiming to have a new perfect model without showing any details, linking to a publication, and apparently having the ability to look into the future without paying any heed to the scenarios and then demanding that we refute their (likely non-existent) efforts. Right. Show us the data! ;-) – gavin]
“We are predicting the extinction of about two-thirds of all bird, mammal, butterfly and plant species by the end of the next century, based on current trends.” –Peter H. Raven (1999), former President of AAAS, the American Association for the Advancement of Science.
Neil22:Guys. In terms of PR…..it’s, er, like…..game over…. Enter the word ‘climategate’ on Googletrends…
Uh huh. Did you try entering it along with *any other* news item search term? It turns into a speck of sand.
The deniers are excited because they’re having free reign in the private conversations of the climate scientists they’d love to demonize. This is so exciting that they’ve failed to notice that there’s very close to nothing there. A few quotes you can pull out and read into what you wish was there – but weak tea.
The fact is this will turn into an urban legend within the extreme right – but the majority of people don’t care.
This is because what it confirms is the opposite of what you wanted to see, what the majority of educated readers have already noticed – you pulled away the curtain and there is no conspiracy. No hoax. No falsification. Just hardworking scientists, as human as any of us, in a few cases losing their cool in private. The goons of the right broke into their server and found very little of note.
The balloon burst here is the last of the credibility of the conspiricy theory crowd.
As noted above by another poster, the true similarity to watergate, is that someone among the reactionaries stooped to committing a criminal offence. My vote for the role of Nixon in this case is Marc Morano, of swift-boating fame. It’s his style, and he works at that level of morality.
A comment to the scientists behind RealClimate, if you read this refuse:
In last-ditch activist efforts on both the right and the left a common tactic, sometimes even spoken of openly, is to just “keep the pressure on” as the euphimism goes. Frivolous lawsuits, personal attacks, harassing family members, throwing mud. The goal is *not* to win via debate, nor even to cause delay. The goal is to push your opponent to the point of a stress breakdown. Please don’t let them do that, and know when to disengage from their “debate”. Your work is too important, and well appreciated by the sane majority, no matter what its conclusions are.
I am perplexed to how anyone here can claim there is no issue at all here. I mean really? You all think everything in these emails is completely above board?
Thanks for the ad-hominem attack Gavin. The data is being made available, but that’s wasn’t the point anyway. I was hoping you could offer a scientific viewpoint as to how/why one method of modeling would/could be predictably more accurate than the other. To the extent that modeling is even scientific anyway. Maybe something in terms of math or physics if those terms mean anything to you.
Apparently you’re not in that business. Excuse me for assuming you were…
[Response: (Look up ‘ad hom’ – you have no idea what it means). But still, you haven’t provided a reference. Without one, nothing can be said other than if you aren’t basing your model on physics, it will have no useful predictability at all. But since no one knows what your model is or does, even that statement is moot. Please supply a reference before bothering to further engage. – gavin]
Give the savior role to activists and politicians? are you serious?
As a long time proponent of anthropogenic climate change, and someone who has put my own credibility on the line many times to argue for the science, I have to say that I am really appalled at what these emails reveal.
Yes, I understand that they appear to have been obtained by theft and I condemn that. But what it does show me is a cabal of scientists deliberately conspiring to stifle the transfer of data and knowledge, spinning data to justify preconceived notions, and generally just subverting scientific method.
Ironically, it appears your efforts will have the opposite effect from what you intended, and justifiably so, in my opinion.
I think you folks–you know who you are–need to come forward and apologize. I can understand that perhaps you acted on nobler motives, but all the same it appears that you subjected good scientific practice to political expediency.
George Monbiot reflects my feelings well, and does so much more articulately than I could. I hope you read his piece.
[Response: I think that you, like Monbiot and many others, are jumping to conclusions without thinking very much. You might try reading this for a bit of perspective.–eric]
383 – 0.o “The kind of person that doesn’t have a stake in the planet’s well-being is the kind of person who minds his own business”
The kind of person you refer to is nothing more then a leech on society.
“Climate scientists have taken it upon themselves to ‘care for the globe’. You refuse to see through this. Any manipulation, twisting of the facts, lies, deception would seem noble in such a high stakes game. The science most certainly would seem honorable. Is it difficult to see that a scientist who sets himself up to save the world would stop at anything? I see the CRU crusading evangelists to fit this mould. You do the science to save the world and the opponents to the science look like haters of humanity. Now, is that a tricky situation for a scientist to be in or what?”
I bet you trusted the science and its predictions when you started your car, turned your lights on, typed on your computer, put on clothing, and used every other technology that has been built by scientists. When your health is in danger, I bet you put your life into the hands of doctors and machines.
I think you are confusing mystics with scientists.
#429 Brian, if CO2 is increasing so much, why is the Arctic sea ice “recovering”?
Comment by J. Bob
Maybe because it is getting on to winter?
And as for sea-ice “recovering”, it is clear that you haven’t kept up to date with the data. Sea ice coverage for this time of year is nip-and-tuck with the 2007 all-time lows: http://nsidc.org/data/seaice_index/images/daily_images/N_timeseries.png
Stolen material, information not gathered legally is not admissible in any credible court that I know of,
yet the US repubs, will comb through what should be banned in court… Says volumes about their tactics, cheating is acceptable if it serves their political goals.
Gavin,
Thanks again for all your replies. This long-time question of mine relates tangentially to the contentious email from Dr. Trenberth and to your previous responses to it. To me, that email just looked like good scientific enquiry from Trenberth, so I take no issue with it.
My question is instead, when are we going to see an updated plot of the temperature record compared to the model record, such that the model record is broken down into the various model contributions (CO2, methane, aerosols, etc.)?
That particular plot is great for the lay person because it shows us in detail where you climate scientists think the temperature rise over, e.g., the past 50 years has come from.
I have been unable to find a more recent version of this plot than the outdated one from Wikipedia Commons. The latest IPCC report only seems to have 1) a timeline plot without a breakdown into components, as well as 2) a bar graph of each of the contributions with error bars, where each contribution is integrated over time.
You must routinely make such plots for yourselves when you run the models, no? Do you perhaps feel (a la Trenberth) that your understanding is insufficient for a ‘model average’ of such a plot to be usefully made for public release?
I would like to see such a plot anyway, even if the error bands on the individual terms are large and highly correlated.
Thanks, s. wing
[Response: For individual contributions, look at Hansen et al 2007. It takes a lot of time to do those kinds of experiments (14 individual effects, 5 simulations each, 150 years, plus a few sensitivity tests = ~11,000 model years, assuming 10 runs simultaneously is roughly 250 days continuous computation, plus a few months to process all the output). Oh, and just in case anyone cares, the output is here, and the code is here. – gavin]
Gavin, I’ve been following the thread here but haven’t posted yet. But I have posted some thoughts over at DotEarth including this proposition: For the sake of argument, let’s ignore any papers associated with the scientists whose emails are at issue. Sureley there are many, many (thousands and thousands?) of other peer-reviewed papers by scientists not associated with those of the CRU group who have independent data sets and models not associated with the data and models related to the CRU group that point to the conclusion that human activities are causing global climate change. (Not that I am questioning their validitgy but just for the sake of argument.) Is it possible to point us to a few of the main ones so that people can see how scientific consensus does not depend on one group of scientists and a select few papers? I believe this might help illustrate a broader point that is relevant to the “debate.” Thank you for remaining a voice of reasoned critical thinking through all the madness.
#474 – caerbannog. Don’t forget nip & tuck with 03 & 05, plus above 06, for this time of year.
http://www.ijis.iarc.uaf.edu/en/home/seaice_extent.htm
472
“I was hoping you could offer a scientific viewpoint as to how/why one method of modeling would/could be predictably more accurate than the other. To the extent that modeling is even scientific anyway. Maybe something in terms of math or physics if those terms mean anything to you.”
If you are asking a general question about models, I’ll explain some to you.
First and foremost, scientific theories are mathematical models. From a mathematical perspective, models do not have to have physical limitations, but from a scientific perspective, models are required to have physical limitations. For example, the square root of 25 will result in +5 and -5 (A square root has 2 answers). The two answers are fine as far as mathematics is concerned; however, both answers may not be suitable in a physical problem. If the square root of 25 results in the length of a computer monitor, a -5 does not make sense. How could you have a computer monitor that had a -5 inch side? So the value is ignored in the computation because of a physical limitation.
Scientists do observations of gravity, electricity, climate, heat, cold, and other physical phenomenon so that they can find limitations and place them upon models. Physical theories are models that predict the outcome of events given certain conditions. I think a lot of people are confusing “prediction” with some kind of mystical ability.
Re 468 Petey
“Some new climate modelers have presented a newer simpler model looking at raw, untreated temperatures only, going back 150 years or so. They use a day over day moving average much like stock traders use (this metric has proved one of the most accurate historical indicators of stock performance).”
Why would the climate system behave so much like the stock market? Sure, there are similarities and potential analogies between any two things, but…
Why would looking only at temperatures over 150 years tell you what will happen in the future depending on what external forcings transpire?
“Perfectly modeling every conceivable system that drives climate is an impossibility, I don’t care what you think you can do.”
Imperfect models are used with great benifit.
“Furthermore attempting to do so to predict future TEMPERATURES seems extremely foolish, when temperatures alone can be modeled much more accurately than the entire Earths energy profile.”
You need to know energy fluxes in order to predict temperature changes.
And the temperature change is only part of what has scientific and practical importance.
“So why all the complication?”
Because the physics gets complicated! But you can simplify the model if you’re willing to settle for a lower-order approximation.
“And can you refute the methodology of these new modelers (in principle, obviously you don’t have all the specifics) and why the complicated (and highly suspicious) way is better?”
It sounds like this new model is a statistical model. Those have their place, but not so much in modelling climate changes.
“To the extent that modeling is even scientific anyway. Maybe something in terms of math or physics if those terms mean anything to you.”
Yes, a model based on math and physics would be in order. Thankfully, there are some.
prince_prospero,
Here’s a short (not out of context) snippet from one of the papers that Mann and Co. got all worked up about. (linky: http://climatedebatedaily.com/southern_oscillation.pdf)
To remove the noise, the absolute values were
replaced with derivative values based on variations.
Does the use of a derivative operator to “reduce noise” in temperature data not raise your eyebrows a bit? Especially temperature data where you are looking for the influence (or non-influence) of a long-term trend?
How is it the GISP ice core temperature data shows like in Al Gore’s movie temperature in the N.H. has been relatively flat for 800 years with a .5 C spike the last 100 years. However why do they stop at 800 or so years because the rest of the data show drastic temperature change as much as 2.5 degrees C in less than 200 years. For example it shows 1 degree C higher 1000 yrs ago, 2 degree C higher 2000 yrs ago and 3 degree C higher 3300 yrs ago with many ups and downs in between. Oh Al all the polar bears are still here and the earth did not flood 3300 years ago.
I know the proxy data tries to dampen these spikes shown in the GISP ice core but most of that data is highly flawed after 1000 years and running them through a 40 year filter is a nice “Trick”
PS – all the data from the southern hemisphere or Volstok ice core shows no extended time period of flat temps like the NH the last 800 years, matter of fact the last 800 years it has had 3 major temp changes over 2.5 degrees C. So how is it CO2 only effects the Northern Hemisphere and not the Southern Hemisphere?
I just have to say that Rob is obviously not a professional software developer. If he was, the first SCM systems he would have mentioned would have been CVS and Subversion.
Personally, I hate CVS and use a commercially sourced system called Perforce, which works much better than CVS and handles directories better than SVN. The reason for the question was just to throw mud on the quality of the code.
[[[ So, we leave the data alone from 1904-1928, adjust downward for 1929-1943, leave the same
for 1944-1948, adjust down for 1949-1953, and then, whoa, start an exponential fudge upward (guess that would be the “VERY ARTIFICIAL CORRECTION FOR DECLINE” noted by the programmer).
[edit]
[Response: Without any context, you have no basis for your statements. What was this for? what paper did it appear in? was it even ever used? Track that down and then we can talk. – gavin]
Gavin, while I can appreciate the desire for context in order to comment in a meaningful way, what explains your pounding down on Mark S. (“no basis for your statements.”) If you will read carefully, Mark made no statements! (apart from describing the working of the code…)
Why so defensive? That was uncalled for.
Wayne Davidson:
Were the Pentagon Papers stolen? What did you think about the fact that the Supreme Court deemed them printable?
Baloney.
The U.S. Supreme Court has held that the press is well within its First Amendment protections when it publishes information obtained by a third party, even obtained illegally.
Remember when the NY Times felt duty to leak CIA memo’s on the Bush admin?
Says volumes indeed watching all the phony self righteousness indignation now.
>Says volumes about their tactics, cheating is acceptable if it serves their political goals.
This is off topic, but if you want to understand the american right, read swimming with sharks, an article about how republicans train their young. Sourcewatch’s entry about Marc Morano is also quite eye-opening.
Okay, I’m in the process of addressing these claims to some denialists. What’s the deal with this excerpt:
“I can’t see either of these papers being in the next IPCC report,” Jones writes. “Kevin and I will keep them out somehow — even if we have to redefine what the peer-review literature is!”
[Response: Both papers were cited in the report – MM either refers to McIntyre and McKitrick (2005, E&E) (Ch6), or more likely McKitrick and Michaels (2004) (Ch. 2), and the other paper was Kalnay and Cai (2003), also cited (Ch. 2). No redefinition apparently took place, which is unsurprising. – gavin]
“#429 Brian, if CO2 is increasing so much, why is the Arctic sea ice “recovering”?”
The snarky answer is that Poe’s Law is alive and well, and propagating into global warming skepticism. I guess I should have put double quotes around “recovery” or a smiley after the conflation of “recovery” with “below the OLS projection of declining ice”. Would I have had to put a smiley up if I had said “the patient has made a remarkable recovery; he’s still dying, but he was almost dead”?
A more relevant answer is that most of the multiyear ice has already disappeared from the Arctic. http://www.newsdaily.com/stories/tre59s3lt-us-climate-canada-arctic/
The remaining ice is so thin that the summer minimum and trajectory by which it is reached has become responsive to wind, cloud cover, humidity, and other weather effects – this year, the rate of melt dropped 50% in about 10 days in mid July, when the weather changed. There have been other short “recoveries”in the past – 60-63, 67-69, 81-83,90-92, but they didn’t last, and each successive peak has been lower. It’s just short term fluctuations, internal variability, not real trends.
Please explain this (if you can…)
http://www.americanthinker.com/2009/11/crus_source_code_climategate_r.html
[Response: What do you want explained? That post is a litany of incorrect assumptions, mis-attributions and full of fury that people actually do what they’ve said they’ve done in the literature. The apparent visceral need for some people to blame Mike Mann for anything they don’t understand in someone else’s work entirely is as puzzling as it is stupid. – gavin]
charles [edit – don’t vent]
It was pretty clear from the beginning that McI was not part of this (although he may have been cheered by it, much as PJ found news of the passing of Daly cheering).
Of course, RC, The Air Vent and CA pretty much all got links at the same time, all apparently from a variety of anonymous proxy servers. Hours reading the emails because you wouldn’t forward copies of the text you were reading?
[edit]
Sorry Gavin, self-aggrandizing stupidity is something I can’t deal with. Sort of like Prince_Prospero, who obviously does not share the same dialect of English that the rest of us do. [edit]
Finally on a more serious and less insulting note. The bozo at the Spectator is filing suit against GISS for what? Correcting an error found by McI and acknowledging that? Oh yeah, the correction was huge (according to McI) and completely rewrote the temperature history of the planet. Oh wait. You mean it moved the anomaly for the lower 48 for 1998 from a statistical tie with 1934 to 1934 being in a statistical tie with 1998? And it had no effect whatsoever on the global record? That’s obviously actionable!
This is obviously phase two. It is clear that harassing scientists with FOIA requests has pissed them off and to some extent kept them from doing “dangerous” “research” which might show that we are “up shit creek”. However, ill-formed FOIA requests are too easy to blow off. But a lawsuit? Unless the judge, as he should, laughs this off, GISS will be brought to a halt by disclosure proceedings. This is not a pretty sight, even though the lawsuit will eventually be tossed.
caerbannog (474) is wrong.
Go back and check the NSIDC figures for Arctic sea ice extent (at September end of melting season).
2007 4.29 million sq. km.
2008 4.67 (+9%)
2009 5.36 (+25%)
The record shows that there has been a significant recovery.
Will it last? Will it increase? Who knows?
[edit]
[sea ice trends are OT – go talk about this somewhere else]
I find this response interesting. Possibly telling.
Gavin leaves us with the impression that reports claiming global warming was going to dramatically increase from 10 years ago were ‘irrelevant’, which I assume means ‘incorrect’. That would certainly explain why the data used to come up with the flawed reports has not been made publicly available. Who wants to be shown to be telling falsehoods? But the ‘telling’ part is that the data used to determine CURRENT global warming beliefs is also not being made public. Why so? There is only one conclusion that we can draw. You should allow anyone access to the data so that we can determine whether or not global warming is the snake oil that so many scientists are claiming that it is.
Will this post get tossed or delayed by Gavin as the data from the FOI emails suggest? It will be interesting to find out!
[Response: Yawn. If you want raw data go to GHCN. If you want current temperatures from exclusively public-domain sources go to GISTEMP. If you want all the raw data that went into those figures go to NOAA Paleoclimate. If you just want to rag on scientists, go somewhere else. – gavin]
“473. Give the savior role to activists and politicians? are you serious?”
Scientists are only meant to report the data. Whatever it says. It’s not their business to do anything else. Telling people what to do for the “good of mankind” is church business.
Do you know how serious particle physicists do their work, for example? They keep a few parameters of their experiment unknown to everyone in the collaboration. That way no one can do any pre-fitting to insure that the results will agree, or that they are even on the right track. They do all the calculations, and only when that is done, they check the final parameters. And there’s no turning back. If you get a wrong answer, that’s what you get. Needless to say, you won’t publish it.
That’s science. Cutting the last years to make the average look warmer is not science. It’s s**t.
[Response: Agreed. All the data up to and including the last months’ number is visible at GISTEMP. – gavin]
Anand Rajan 383:
“The kind of person that doesn’t have a stake in the planet’s well-being is the kind of person who minds his own business.”
Setting aside the moment the moral bankruptcy of your stance, which Gavin already noted — isn’t it obvious that the well-being of the planet is everybody’s business? If you take an action that harms the planet, you are harming me and my children. How can that possibly not be my business? This point simply makes no sense, whatever your moral stance.
You stated in an earlier comment that you think climate scientists have an agenda a priori. I think you have a basic misunderstanding of the philosophy of science (shared also by other commenters on the issue of “saving the world”). Science is based on the philosophy of Empiricism, in which knowledge is held to come from evidence and experimentation, not a priori.
Therefore it would be counterintuitive for a scientist to approach a question with an a priori agenda and then distort the science to fit that agenda. You appear to think that the “bias” of scientists arises from genuine concern for the well-being of the planet. However, to a scientist, the only way to determine what constitutes the well-being of the planet is through accurate science — through empirical evidence obtained from experimentation and observation, and logical theory based on that evidence.
To put this another way, how can a scientist possibly know what is best for the planet if they have distorted the science to serve an a priori agenda? In that case, the information obtained from the science would be unreliable, and so the scientist would have no way of knowing what would be best for the planet.
Therefore the motivation of a scientist who is genuinely concerned for the well-being of the planet (which you appear to think Gavin and others are) would be to make sure the science is as accurate as possible, so as to be able to determine correctly what would constitute the planet’s well-being.
It makes no sense to say “scientists distorted the evidence because they wanted to save the world” when to a scientist the only way to find out how to save the world is to have accurate evidence.
The only way I can see that you can logically support the idea that there is an a priori agenda underlying the science is if you assume that the scientists involved are not genuinely concerned with the well-being of the planet, and instead have a particular political goal that they want to achieve irrespective of the effect on the planet. If that is what you think, you should say that and we can discuss it. But that is not what you said, and without that assumption your argument makes no sense.
As somebody who has done distinguished research in a different field for years, your spin on the emails, saying it lacks context, is disappointing to say the least. Entire email threads in their full glory are available in some instances. The language is explicitly clear in many instances, even if you didnt know the first thing about the science.
1. Here’s a quote from one of the emails : I got a paper to review (submitted to the Journal of Agricultural, Biological, and Environmental Sciences), written by a Korean guy and someone from Berkeley, that claims that the method of reconstruction that we use in dendroclimatology (reverse regression) is wrong, biased, lousy, horrible, etc. … If published as is, this paper could really do some damage … It won’t be easy to dismiss out of hand as the math appears to be correct theoretically (…) I am really sorry but I have to nag about that review — Confidentially I now need a hard and if required extensive case for rejecting.” Im assuming this is about MBH98 or 99? Even if it is not, Pls dont try to spin this one. It is an obvious attempt at suppressing a paper critical of one of their work, at the review stage, by the folks whose work he is finding fault with. The reviewer, it seems actually doesnt have a case for rejecting but rather would like other non-reviewers to make one up. Instead he should have actually accepted the paper pointing out whatever issues he uncovered for corrections. Calling this malfeasance doesnt do justice to the word malfeasance.
[Response: You are missing context. The first is from someone in the middle of a review asking a colleague whether he has the right data to do a test of a particular method in the un-named, and un-sent paper he was reviewing. One of those emails is from the editor asking his reviewer to get a move on with a review for a paper the other reviewers have suggested rejecting. Calling this malfeasance indeed, doesn’t do justice to the word malfeasance (i.e. it isn’t malfeasance). – gavin]
2. Heres another quote: ““The two MMs [Canadian skeptics Steve McIntyre and Ross McKitrick] have been after the CRU station data for years. If they ever hear there is a Freedom of Information Act now in the UK, I think I’ll delete the file rather than send to anyone.” Why so? if these guys are not good climate scientists, then you should be able to supply them the data you have and if they come up with bogus objections to your claims, then you should be able to show them up for who they are by pointing out the flaws in their analysis. While at it, you could have other genuine climate scientists publically analyse the data and confirm your claims. Why dodge the data request, if you knew your data can withstand the scrutiny? i have shared data with others who wanted to replicate my claims as is std practice for many branches of science. MBH98/99 being critical paper for some of the earlier IPCC findings, why would you resist scrutiny, if you know the data is solid?
[Response: Please read the background on this. The FOI requests started in 2007 and they were turned down then (and again as recently as Nov 13) not because there is anything wrong or embarrassing about the data, but because some of it is restricted by agreements with third parties. The public domain source data is available to all and sundry and has been for years (GHCN) and is the basis of the other temperature records (i.e at GISTEMP). As for MBH98/99, the data and code is all available, and again, has been for years. This is also true for the more recent and comprehensive reconstructions. – gavin]
Here’s another quote: “I think we have to stop considering Climate Research as a legitimate peer-reviewed journal. Perhaps we should encourage our colleagues in the climate research community to no longer submit to, or cite papers in, this journal. We would also need to consider what we tell or request of our more reasonable colleagues who currently sit on the editorial board”
what the ..? Who died and made these guys kings and king makers? If a journal is of such a bad quality, let the majority researchers in the field conclude such after a) pointing to flaws in the papers they publish b) and seeing that the journal doesnt take action over a period of time, even after their publications have been shown to be bad.
[Response: Sure. That is indeed what happened. But talking about the quality of a journal and informing your colleagues of your feelings is not unethical. – gavin]
Science is all about dissenting voices and hard scrutiny. These guys were clealry attempting to get rid of all dissenting voices and scrutiny of their models and data, which forms a key foundation of a lot of the IPCC claims. Imagine Einstein and many others like him trying to silence the quantum mechanics crowd, because he didnt believe god played dice. This is not even some esoteric n-dimentional math that avg joe has no use for. This is something that governments are going to take hard action on, that affects everybody. THe science should go through the toughest scrutiny we can put on it, before billions of people are affected, either due to the climate change or due to the actions taken by govts to deal with climate change. If the science and the scientists on your camp are superior, why dodge the scrutiny and attempt to get rid of people critical of your models/data?
[Response: You are projecting here. The criticism was of very specific (and bad) papers, not everyone who is critical. Lot’s of critical articles get published all the time, but in the cases discussed here – maybe 4 or 5 papers – the studies weren’t just critical, they were fundamentatlly flawed. Soon and Baliunas (2003), Douglass et al (2008), McKitrick and Michaels (2004), McLean et al (2009) etc. Where does anyone say that no critical papers should be published? Where? – gavin]
“Something is rotten in the state of denmark” indeed.
Hello again and thank you for the previous response to #458.
Follow-up questions to [Response: Yes. The further back you go the more important the long tree ring records are, but the general patterns are robust. – gavin], if I may.
a) How much weight are the long tree ring records given in the reconstructions, in the different time periods?
b) How many individual trees’ samples were directly included in the reconstructions, does this vary by time period and/or affect weighting in the reconstruction?
c) What establishes those records as valid temperature proxies?
d) In the absence of tree ring histories, what alternatives could be used to reconstruct historical temperatures over a similar timeframe?
e) What precisely is the technical definition of the word ‘robust’ in this case? Is there a specific, commonly used threshold for the standard of robust with respect to this speciality? Can that term be quantified?
f) Has verification of the trends established by the reconstructions been performed using independently-sourced data?
[Response: This recent review covers most of this. – gavin]
While some of this particular “scandal” looks like a storm in a teacup to me, it has highlighted one important factor:
If our governments are going to be making important decisions on future economic ground, eg for taxation, carbon credits etc, then we have to view the problem as a mix of scientific and financial.
It is prudent to submit the underlying data to the same sort of stringent auditing process applied in an accounting context. There needs to be a very clear and transparent “audit trail” for all the datasets – and appropriate penalties applied for any that have been found to be falsified, hidden etc.
This does not mean I think that any data *has* been supplied in a wrong fashion, but I think it is important that we have safety checks in place to ensure we’re basing our decisions on the highest-quality and most-reliable information.