This is a continuation of the last thread which is getting a little unwieldy. The emails cover a 13 year period in which many things happened, and very few people are up to speed on some of the long-buried issues. So to save some time, I’ve pulled a few bits out of the comment thread that shed some light on some of the context which is missing in some of the discussion of various emails.
- Trenberth: You need to read his recent paper on quantifying the current changes in the Earth’s energy budget to realise why he is concerned about our inability currently to track small year-to-year variations in the radiative fluxes.
- Wigley: The concern with sea surface temperatures in the 1940s stems from the paper by Thompson et al (2007) which identified a spurious discontinuity in ocean temperatures. The impact of this has not yet been fully corrected for in the HadSST data set, but people still want to assess what impact it might have on any work that used the original data.
- Climate Research and peer-review: You should read about the issues from the editors (Claire Goodess, Hans von Storch) who resigned because of a breakdown of the peer review process at that journal, that came to light with the particularly egregious (and well-publicised) paper by Soon and Baliunas (2003). The publisher’s assessment is here.
Update: Pulling out some of the common points being raised in the comments.
- HARRY_read_me.txt. This is a 4 year-long work log of Ian (Harry) Harris who was working to upgrade the documentation, metadata and databases associated with the legacy CRU TS 2.1 product, which is not the same as the HadCRUT data (see Mitchell and Jones, 2003 for details). The CSU TS 3.0 is available now (via ClimateExplorer for instance), and so presumably the database problems got fixed. Anyone who has ever worked on constructing a database from dozens of individual, sometimes contradictory and inconsistently formatted datasets will share his evident frustration with how tedious that can be.
- “Redefine the peer-reviewed literature!” . Nobody actually gets to do that, and both papers discussed in that comment – McKitrick and Michaels (2004) and Kalnay and Cai (2003) were both cited and discussed in Chapter
2 of3 the IPCC AR4 report. As an aside, neither has stood the test of time. - “Declines” in the MXD record. This decline was
hiddenwritten up in Nature in 1998 where the authors suggested not using the post 1960 data. Their actual programs (in IDL script), unsurprisingly warn against using post 1960 data. Added: Note that the ‘hide the decline’ comment was made in 1999 – 10 years ago, and has no connection whatsoever to more recent instrumental records. - CRU data accessibility. From the date of the first FOI request to CRU (in 2007), it has been made abundantly clear that the main impediment to releasing the whole CRU archive is the small % of it that was given to CRU on the understanding it wouldn’t be passed on to third parties. Those restrictions are in place because of the originating organisations (the various National Met. Services) around the world and are not CRU’s to break. As of Nov 13, the response to the umpteenth FOI request for the same data met with exactly the same response. This is an unfortunate situation, and pressure should be brought to bear on the National Met Services to release CRU from that obligation. It is not however the fault of CRU. The vast majority of the data in the HadCRU records is publicly available from GHCN (v2.mean.Z).
- Suggestions that FOI-related material be deleted … are ill-advised even if not carried out. What is and is not responsive and deliverable to an FOI request is however a subject that it is very appropriate to discuss.
- Fudge factors (update) IDL code in the some of the attached files calculates and applies an artificial ‘fudge factor’ to the MXD proxies to artificially eliminate the ‘divergence pattern’. This was done for a set of experiments reported in this submitted 2004 draft by Osborn and colleagues but which was never published. Section 4.3 explains the rationale very clearly which was to test the sensitivity of the calibration of the MXD proxies should the divergence end up being anthropogenic. It has nothing to do with any temperature record, has not been used in any published reconstruction and is not the source of any hockey stick blade anywhere.
Further update: This comment from Halldór Björnsson of the Icelandic Met. Service goes right to the heart of the accessibility issue:
Re: CRU data accessibility.
National Meteorological Services (NMSs) have different rules on data exchange. The World Meteorological Organization (WMO) organizes the exchange of “basic data”, i.e. data that are needed for weather forecasts. For details on these see WMO resolution number 40 (see http://bit.ly/8jOjX1).
This document acknowledges that WMO member states can place restrictions on the dissemination of data to third parties “for reasons such as national laws or costs of production”. These restrictions are only supposed to apply to commercial use, the research and education community is supposed to have free access to all the data.
Now, for researchers this sounds open and fine. In practice it hasn’t proved to be so.
Most NMSs also can distribute all sorts of data that are classified as “additional data and products”. Restrictions can be placed on these. These special data and products (which can range from regular weather data from a specific station to maps of rain intensity based on satellite and radar data). Many nations do place restrictions on such data (see link for additional data on above WMO-40 webpage for details).
The reasons for restricting access is often commercial, NMSs are often required by law to have substantial income from commercial sources, in other cases it can be for national security reasons, but in many cases (in my experience) the reasons simply seem to be “because we can”.
What has this got to do with CRU? The data that CRU needs for their data base comes from entities that restrict access to much of their data. And even better, since the UK has submitted an exception for additional data, some nations that otherwise would provide data without question will not provide data to the UK. I know this from experience, since my nation (Iceland) did send in such conditions and for years I had problem getting certain data from the US.
The ideal, that all data should be free and open is unfortunately not adhered to by a large portion of the meteorological community. Probably only a small portion of the CRU data is “locked” but the end effect is that all their data becomes closed. It is not their fault, and I am sure that they dislike them as much as any other researcher who has tried to get access to all data from stations in region X in country Y.
These restrictions end up by wasting resources and hurting everyone. The research community (CRU included) and the public are the victims. If you don’t like it, write to you NMSs and urge them to open all their data.
I can update (further) this if there is demand. Please let me know in the comments, which, as always, should be substantive, non-insulting and on topic.
Comments continue here.
Speaking of the “slow-down in warming this decade”:
Dr. Mojib Latif, who was, until the present matter blew up, the climate scientist most often taken out of context, was interviewed on NPR Sunday.
http://www.npr.org/templates/story/story.php?storyId=120668812&ft=1&f=1007
NPR interviewer Guy Raz: . . .Do you think your work is being misused?
Dr. LATIF: Yes. It is misused. I must say this, unfortunately, because these changes we are talking about, these short-term changes, you know, their amplitudes are much smaller than the long-term warming trends. . .
BAU, denialist style.
@Pete Ridley, #76. Thank you for posting that excellent paper on water vapor in the atmosphere. It contributes to climate researcher’s understanding of the many feedback mechanisms involved in climate research. However, you miss the overall point of that article, which is that while everyone knows water vapor is an important GHG, and water is more likely to evaporate in warmer conditions when the air can hold more moisture, but quantifying the rate at which this occurs with temperatures changes, in their research, is not monotonic, meaning it can’t be plotted in a straight line on a graph. It does nothing to dispute ACC – it’s shedding light on a well known piece of the puzzle, opening up some of its lesser-known nuances. It’s good science and many climate researchers no doubt incorporate such ideas (and others as they come along) where pertinent. You also claim they make no mention of ACC, but on the first page:
“For instance, there is evidence that the width of the Hadley circulation has increased over the past decades [e.g., Hu and Fu, 2007; Seidel and Randel, 2007; Seidel et al., 2008], and it also increases in many simulations of climate change in response to increased concentrations of greenhouse gases [e.g., Kushner et al., 2001; Lu et al., 2007; Previdi and Liepert, 2007; Johanson and Fu, 2009]. This widening of the Hadley circulation is often linked to the decrease in the moist adiabatic temperature lapse rate with increasing surface temperature, which results in an increased tropical static stability and can lead to a widening of the Hadley circulation, at least in dry atmospheres…”
so you seem guilty yourself of picking out some data and throwing the rest away. I’m glad you posted it though, as I’ve been too busy with school work (and today, this distraction) to catch up on all of my AGU (and GSA) reading. Well done.
#53: Ray Ladbury says:
23 November 2009 at 6:12 AM
Quote: The basic problem with the denialist arguments is that the proponents are so utterly ignorant of the science. For God’s sake, we now have people contending that the rise in CO2 is not due to human activity! Folks, come on. Humans have produced more CO2 than can be accounted for by the increase in the atmosphere and we have acidification of the oceans that accounts for the rest. We know the carbon is from a fossil source by the fact that it is enriched in light carbon isotopes. We KNOW these things, and to claim we do not is not skepticism but ignorance at best. /Quote
That’s a thing that has long fascinated me.
We burn ~84,000,000 barrels of oil a day. Colossal tonnages of coal and volumes of gas. The figures are huge, mind-bogglingly so. One would think that burning hydrocarbons in air would produce oxides of hydrogen and carbon. You folks with me so far?? (/sarcasm)
I wonder where all that carbon dioxide is supposed to go. Perhaps there is an invisible black hole into which it all disappears? Because in the world of the denialist, it must end up in a parallel dimension somewhere!
Keep up the good work, Gavin!
Cheers – John
Gavin: Judy Curry has some kind comments about you, here: http://www.climateaudit.org/?p=7826#more-7826
Her take on some of the data availability:
“The HADCRU surface climate dataset and the paleoclimate dataset that has gone into the various “hockeystick” analyses stand out as lacking such transparency. Much of the paleoclimate data and metadata has become available only because of continued public pressure from Steve McIntyre. Datasets that were processed and developed decades ago and that are now regarded as essential elements of the climate data record often contain elements whose raw data or metadata were not preserved (this appears to be the case with HADCRUT). The HADCRU surface climate dataset needs public documentation that details the time period and location of individual station measurements used in the data set, statistical adjustments to the data, how the data were analyzed to produce the climatology, and what measurements were omitted and why. If these data and metadata are unavailable, I would argue that the data set needs to be reprocessed (presumably the original raw data is available from the original sources). “
The comment was ill-advised because, taken at face value, it is suggesting that people ought to break the law.
In itself, though, it does not prove that the law was actually broken. Perhaps people responded saying, “no, think this through, we can’t do that” and those who feloniously purloined the e-mail didn’t include this response. Perhaps Jones, on further thought, agreed and retracted his suggestion and you’re not being shown that by the agenda-driven hackers.
Or, perhaps they did break the laws.
The point is that from the files, we don’t know. So saying “the comment was ill-advised” is certainly accurate. Claims that Jones broke the law go beyond what we currently know.
If he did, no sympathy on my part, he should know better. If he didn’t, the e-mail still makes him look foolish.
Either way:
It has no effect on the science whatsoever. CO2 isn’t going to suddenly stop absorbing long-range infrared radiation out of embarrassment over Phil Jones’ e-mail.
It also has no bearing on the obvious fact that someone committed a felony in breaking into CRU’s server and distributing the contents.
“That’s no good advice to give a scientist. Let the scientists establish the science, but leave the savior role to activists and politicians.”
Yes, I considered that angle, but decided on balance it is better to give encouragement and try to establish a sense of perspective.
Obviously the best thing a scientist can do to promote the well-being of the species is to provide accurate data — otherwise we will be making decisions based on faulty information. Keep in mind that it cuts both ways: if scientists can prove that AGW is false, they will save us from massive wasted effort. Either way they have provided an invaluable service to society. I think Gavin understands that.
However, the role of providing accurate data is crucially important. It’s easy to get caught up in the petty details of the argument. I was trying to call attention to the big picture. What these scientists are doing is incredibly important regardless of the results of their research, and they are being attacked by opponents with no scruples or intellectual honesty. They should stand proud and remember that they are doing humanity a great service.
Lawrence Coleman says: 23 novembre 2009 at 10:21 AM
[huge paragraph full of statements without sources] then
> Do I have to go on or can you begin to see a picture happening here!
> You can ask gavin et-al for all the hard scientific facts.
C’mon, Lawrence. Posting a collection of assertions without cites, then telling people they can bother Gavin about getting the facts?? This is rude.
If you take each sentence in that huge dump and look it up, I’d guess you can support half those claims or less. Some of them are incoherent, like
> rate of coral polyp and inverebrate deformation due to weak Calcium
> carbonate matrices is now common in most parts of the world
> caused by ocean acidification.
Please, you can do much better than asserting things and expecting others to find the evidence for them. That’s just wasting everyone’s time.
I think there are a few other groups whose email correspondence would be worth viewing…
1) American Petroleum Institute
The budget for their 1998 latter project included $600,000, partially to “identify, recruit and train five independent scientists to participate in media outreach.”
If you want to find out who those 5 media-oriented tobacco scientists are, just check Andy Revkin’s blog at the NYT – he usually cites one or two of them in every post. That’s an important point – the tobacco scientists need access to media, and guess who is there to provide it? Can we see a collection of emails sent to Andy Revkin?
In fact if the NYT email accounts were hacked and put on file somewhere, the NYT would probably refuse to discuss it, citing the need for journalists to protect their sources!
API shadiness has few limits – recall the 2006 story on how the National Science Teachers Association refused to distribute free copies of An Inconvenient Truth? Here’s an API blurb on that from their 1998 memo:
Organize under the GCSDC a “Science Education Task Group” that will serve as the point of outreach to the National Science Teachers Association (NSTA) and other influential science education organizations. Work with NSTA to develop school materials that present a credible, balanced picture of climate science for use in classrooms nationwide.
The scope is quite breathtaking – but with millions of dollars to spend, that’s what happens.
2) American Coalition for Clean Coal Electricity.
Bonner Associates forged letters from various public groups on ACCCE’s behalf, but there was no legal response – ACCCE was not censured or banned from lobbying or fined. For more on this dishonest coal front group:
http://www.huffingtonpost.com/pete-altman/accce-the-american-chambe_b_322677.html
“ACCCE gets its turn in the barrel later today when it will enjoy the spotlight of a Congressional hearings on letters that were forged on its behalf this summer and sent to members of Congress.
Yes, and Andy Revkin at the NYT was all over that story – multiple blog posts, front page headlines – right? Wasn’t he? Actually – no, he didn’t. Not an issue, those forged letters… but climate scientist emails, oh, that’s big news, we gotta run with that, it’s very “sinister.”
I’ve sent a letter to the NYT ombudsman on this, I suggest others do so as well.
3) FutureGen Industrial Alliance, Inc.
FutureGen is a public-private partnership to design, build, and operate the world’s first coal-fueled, near-zero emissions power plant, at an estimated net project cost of US $1.5 billion. The commercial-scale plant will prove the technical and economic feasibility of producing low-cost electricity and hydrogen from coal while nearly eliminating emissions.
Reporters might want to submit FOI requests to the DOE on this Sept 2009 story:
The FutureGen Alliance and the U.S. Department of Energy today signed a Cooperative Agreement allowing for the continued development of a cutting edge, low emissions coal-fueled power plant in Mattoon, Ill.
In particular, reporters should be asking if what is really going is the construction of a coal-to-gasoline plant under the guise of a zero-emissions coal plant.
The reason? It’s all coal gasification technology, and the component parts of FutureGen are to be auctioned off to Illinois coal producers after the $2 billion “demonstration project” is completed. That is to say, there is no plan to keep FutureGen operating as a zero-emissions coal plant – which makes sense, since the technology doesn’t work – like fusion, it’s “fifteen years in the future.”
What happens to coal-to-gasoline plans under binding emission limits? They evaporate.
4) The CO2 Capture Project.
The CO2 Capture Project (CCP) is a partnership of the world’s leading energy companies, working with academic institutions and government organisations to research and develop technologies to help make CO2 capture and geological storage (CCS) a practical reality for reducing global CO2 emissions and tackling climate change.
Here you have your tar sand syncrude and liquified natural gas concerns, and since both are emissions-intensive, a PR effort has been mounted (as with coal) to portray them as “clean” and to promote the myth that the emissions have been captured, so tar sand imports can go forward. Interestingly enough, this was the same rationale that the State Department used when granting a recent tar-sand import permit to Canada.
A complete list of all correspondence from these institutions – particularly that involving their DOE partners and their press partners – would make for far more interesting reading than the CRU email, I think…
I always get fascinated by the conclusion some of the AGW-skeptic/denier/whatever group:
“We do not understand climate, therefore we are safe to continue CO2 emission without limit”
Maybe I am extremely risk averse, but for me this is more logical:
“We do not understand climate, therefore we should not mess with it”
Thanks to RC for putting this light breeze in a small teacup to bed.
But, Gavin, you may be buzy over the next few days. Knowing how some of these things work – that’s the opinion management people (=PR)- there may be more selected leaks coming out in the next few days, now the story has international traction.
Holy moly! It’s like something parodied on the The Simpsons:
It’s amazing how hard people will work to dumb themselves down for the thrill of being mean.
“Ah, there’s no justice like angry mob justice.”
Arthur Dent says in #86 : “Ill advised is a very strange term to use. It is a criminal offence in the UK to destroy information that is the subject of an FOI request.”
However, if the data had already been deleted before the FOI request then there is no issue.
If the data cannot be passed on due to binding legal agreements with the IPR owners of the data, the party making the FOI request can obviously approach the same agencies who are the owners of the IPR rights for that data, and no party has been coerced into exposing themselves to litigation.
If appropriate civil servants of the UK goverment review the FOI requests and judge that the requests are unnecessary, the requestors are free to appeal.
Re #s 80 and 86: The email advising deletion of confidential emails seems problematic only if taken out of context. Note that it starts with the FOI officer advising one of the scientists that correspondence (specifically with Caspar Ammann and Mike Mann) that is considered by CA and MM to be confidential is exempt from the British FOI law. Presumably CA and MM did say so, as a consequence of which the deletion was perfectly legal. That it would have also been appropriate given concerns about the ultimate security of emails kept on the UAE server has been made obvious by recent events.
It’s worth noting that many of the released emails would fall under the same exemption.
Some of the inequalities in Trenberth can be explained by melting “anchor ice” on the fringe of Antarctica.
Anchor ice is sea water that is rapidly cooled so that it freezes without expelling it the salt in the water; has a density greater than sea water; and, a melting point below 0C. At one time, there was a good deal of such anchor ice. Now, there is less.
Southern Ocean currents can deliver large amounts of heat to subsurface ice, the melting of which could absorb large amounts of heat with minimal change in the temperature, volume, or density of the surrounding waters. This is not a process you will find — unless you are looking for it. However, the melting of the anchor ice could absorb large amounts of heat, and by the by, may protect the base of the WAIS from warm currents.
I have no evidence except what my slide rule tells me, but that slide rule has always been as reliable as a virgin with a mouth full of fresh bay leaves. It is just a matter of asking the right question – and believing in the oracle. Most will not like this answer, and therefore will not believe it.
Dan E. Bloom: This is why we are going to need polar cities in 500 years.
BPL: There won’t be a North Pole in 500 years.
Antarctica will still be covered with ice. You need farms to feed cities.
I want to thank RC for fighting this fight (among all the others that you find yourself fighting), and extend sympathy and solidarity to the researchers whose privacy was violated and who find themselves having to defend their professionalism due to this illegal action.
In the long term, I think that the shrill nature of the deniers is doing them far more harm than good. They keep crying wolf and RC keeps, calmly and rationally, showing them that, well, not a wolf.
Keep it up.
Saying these are “private emails” and the normal banter of colleagues (I agree with that for 98% of the emails) omits that the University of East Anglia’s written Freedom of Information Act policy states that all email should be considered public and staff should always strive to be “clear and professional” in their email correspondence.
The best approach now is for the participants themselves to ask for an independent investigation of the statements made in the emails – the comments in the source codes and other documents. If there is nothing to hide, this will be easily resolved.
Dr. Trenberth says these emails are just a few selected to make people look bad. An easy solution, then, is to release all or most of the email correspondence since the rest will presumably present researchers in a better light. Per the UEA FOI policy, the rest of the email is now likely to be requested via FOIA anyway.
In the future, proceed with as much transparency and openness as possible (per Dr. Judith Curry and commonsense) – release the data and code where ever possible. Open source has shown that many eyes help produce more reliable code – and help find the truth.
Refrain from any appearance of “stacking” the peer review process in the future. Again, focus on openness and transparency.
Climate scientists have suffered a huge black eye. The best way to re-establish credibility and reputations is to change the mindset from secrecy and tribal behavior to openness.
Max: they believe that the complex system of radiative forcings and feedbacks doesn’t easily have an amplification of much more than 1. They want to calm down the debate to a more rational and sensible level,
BPL: Why is it either rational or sensible to use a figure that isn’t correct? That’s like Woody Allen’s line about a guy who thought humans would never progress until they lowered their body temperature from 98.6, which he felt was unreasonable.
It might be helpful to focus on the basics of the science underpinning the idea of AGW to counter the skeptics. The greenhouse effect is a real phenomenon and human activity releases GHGs into the atmosphere. No amount of statistical nit-picking about tree ring proxies can change those two facts. Human activity causes the earth to warm because it has to warm without violating really fundamental physics.
Chris K: I may yet be proven wrong, and a link between CO2 and global average temperature may exist, but I doubt it
BPL: Try here:
http://BartonPaulLevenson.com/Correlation.html
My guess is that many of them will continue to deny that humans are affecting climate change, arguing its a natural phenomena and thus we don’t have to do anything. Those that do realize it probably just won’t speak up – “victory has a thousand fathers, but failure is an orphan”, that type of thing.
I have a question for the ‘AGW supporters’ (let’s use that shorthand for now)
Why do you want the globe not to warm up?
[Response: That is a reasonable question and is based on the fact that society is adapted to the climate we have (though not perfectly of course). Where we build cities, where we grow crops, how we live are all based on assumptions about the climate – the temperature, the rainfall, the tides, the expected storm surges, the variability etc. Changes to that background will likely require shifts to how we do all of those things and some changes will be hugely expensive in terms of people and infrastructure to adapt to – even if that is possible. Coupled with the uncertainty of exactly a planet that would be warmer (say 3ºC) than at any time in the last 3 million years would look like (and note that sea levels were ~60 ft higher then!), I don’t want to take the risk. YMMV. – gavin]
Let me also, address the converse side of the question. If you believe your data and research indicates that there is anthropogenic global warming, why do you want ‘Copenhagen to succeed’?
Why do you want to see immediate, concrete, legislative or normative action taken on the basis of your scientific findings?
[Response: See above. – gavin]
For a AGW-supporting climatologist, failure of Kyoto-ish protocols and laws taking root in governments around the globe is considered a personal/professional failure. Why?
[Response: Not sure about that. I am not involved in drafting legislation or treaties and so I have no personal stake in their passage, other than my hope as one citizen among many that politicians might, occassionally, be able to look forward beyond the next election cycle. – gavin]
WDS — 23 November 2009 @ 5:00 AM:
I am an academic (biological science, retired) and I disagree with you. If an incident like that in CR occurred in a journal I published in, I would have acted the same. Scientists should be very upset when the review process fails so catastrophically. Read the CSI piece Gavin linked to above (issues) where this, and another incident in my area of interest are discussed.
Steve
Wow…Ray Ladbury is still posting…so much for non-insulting.
Gavin…I appreciate your efforts in a very daunting task. For what it’s worth, I have spent years working for a state environmental agency and finally gave it up after seeing my sections mission change from working for the tax payer to becoming predatory towards the tax payer.On top of that, having to deal with people like Mr. Ladbury on a daily basis made for a very hostile work environment. I also realize that there are two sides to every debate. The main problem I see with the continuance of this discussion both here and in public is the attitude exhibited by people like Mr. Ladbury, who demand that everyone agree with him or face his wrath. The scientific community can learn from this little hacking incident. If you don’t want it hacked…get rid of it. Dropping dimes in any hackers chat room can provide amazing results.
If all you want to do is insult people with legitimate questions Mr. Ladbury, have Gavin give you my email…you’ll be toast by noon.
Re 87: There’s nothing at all anti-science about some deniers (although you get your fair share of nuts in both camps I’m sure).
In fact the main problem I have is that, as #81 so fortuitously provides in his ill-advised comment, most climate change believers feel that the science is looking to prove a trend “they already know is there”. His words, not mine.
Now, as a scientist (we just need a science degree to call ourselves that these days don’t we!! :)) I would say that this is equivalent to the logic behind intelligent design devotees. They see proof where there is none because they already know what they want to see.
I saw a programme once about some idiot who believed the temples at Angkor Wat were a massive early star map. He saw patterns and trends of temple placement which he felt fitted with the stars…because that’s what humans do. They pick out patterns and trends in order to try to understand the things they see around them. Of course he was talking utter rubbish and his claims were a total nonsense.
Just as I have a problem with creationists and the like hijacking science’s reputation by claiming some pseudo-science basis for their own folly, I also have a problem with man-made global warming believers using unproven science to drive massively debilitating and expensive political policy that will affect everyone.
Perhaps global warming is man-made. I’ve said it before..I doubt it is…but all I am waiting for is better proof before banging my fists and demanding punitive taxation as the answer.
[Response: No data has been lost or destroyed. – gavin]
[Response: No. But I am not party either to those FOIA requests, nor the timing and nor do I know what happened or what the scope was. – gavin]
Given your second response, or first is a guess, not what you know. Presenting unknowns as known facts is what got you in this mess.
In our paper’s
supplementary information, we state that we used the HadCRUT2
temperatures for this purpose, which combines land air temperatures
with SST observations. In fact, we used the CRUTEM2 land-only
temperature data set for this purpose. These should be identical
where the proxy locations are not coastal. For these correlations,
we did not filter the data, nor did we detrend it, and we used the
*full* period of overlap between the proxy record and the available
instrumental record.
We excluded records that did not show a *positive* correlation with
their local temperatures.
“So WHY!? state that you used HadCRUT2 when you in fact used CRUTEM2”
[Response: I assume that you have never made a mistake? And if you did, you never corrected it? – gavin]
This is why I do not believe we will come to some consensus to try and ward off some of the worst effects of global warming. We have individuals with no science background, and no basic scientific understanding, looking for one quick soundbite that will confirm that AGW is a lie. Context not needed. A large majority of this country will never believe because the science looks at temperatures over 6,000 years ago. In their mind, anything further back is a lie by the devil because the earth is only 6,000 years old. There is no explaining to them. Unfortunately, this group is a majority in the US. They get their “science” from the bible and Glen Beck. The day Stephen Schneider gets invited on the GB show to explain how peer review works, the scientific method, and why we must do something now, I will start believing in a god. [Please note: I will not start believing if this invitation occurs after the evidence becomes so overwhelming that you would have to belong to the flat earth or gravity denying groups to think otherwise].
I’d be interested in any comments you may have on this – 1201724331.txt – an email from Caspar Ammann to P. Jones:
“Fodder for the critics: all these modelers, they always put in too much energy – no wonder it was warming – and now they want to reduce the natural component?”
It seems it’s not only contrarians who harbour doubts the validity and predictions of climate models.
Additionally, I’ve yet to see convincing evidence that the start of the supposed current warming period being coincident with the start of the industrial era, is anything but a coincidence.
[Response: The previous line is helpful: “And regarding TSI, it looks like that 1361 or 1362 (+/-) are going to be the new consensus. All I hear is that this seems to be quite robust. “. He is talking about the uncertainty that exists about the absolute calibration for the solar ‘constant’ (see the PMOD site for more info). His comment above is tongue in cheek because a change in the base TSI level has very little impact on the sensitivity of the models to changes in GHGs – everything else gets changed slightly to get the model into equilibrium. This would only be fodder for sceptics who didn’t know what the issues were, and I’m sure there aren’t any of those. – gavin]
Can anyone deny that politics is involved on BOTH sides? I for one would like to know the true facts. Skeptics looking for the truth should not be put down as “deniers” any more than proponents of the theory should be praised as defenders of “settled” science. All of this is theory on both sides. None of this is settled science. I think that both sides should be open with their findings and methods, and transparency should replace accusations and posturing. I have not seen any proof that greenhouse gases are the cause of global warming, and I have not seen any proof that they are harmless. We need more science and less politics.
Why cant they just ged rid of those bad apple scientists, and get on with the work? copenhagen is waiting, there are thousands of scientsist, who cares if a couple of them made som bad decisions?
One of the hacked emails appear to address a recent, narrow controversy. Perhaps Dr. Schmidt or another RC moderator could comment?
Lakebed sediments (“varves”) can be informative about paleoclimate. The varve records from Lake Korttajarvi in Finland were described by Tiljander et al. (Boreas, 2003); they have been used as proxies in recent long-term temperature reconstructions. Some of them were used in an “upside-down” orientation by Mann et al. (PNAS, 2008). Kaufman et al. (Science, 2009) initially made the same mistake, then corrected it upon online publication, when the error was brought to their attention.
This topic has been discussed at a number of science-oriented blogs, notably Stoat. Also ClimateAudit, Roger Pielke Jr’s blog, Delayed Oscillator, and Cruel Mistress.
The original email (with two replies) is coded 1252154659.txt, reproduced as Comment #28 at this thread at Pielke’s blog. It appears to show Kaufman and a coauthor candidly conferring with CRU colleagues about how to handle the upside-down Lake Korttajarvi proxy that Steve McIntyre originally identified.
Is this email genuine, do you think?
If it is, then all parties seem to have concluded that the use of certain Lake Korttajarvi proxies in building the climate reconstructions in Mann et al (PNAS, 2008) was mistaken.
If this is the case, should Dr. Mann and the other authors of the peer-reviewed PNAS article issue a notice that reflects this consensus? Should PNAS’ editors request such a correction?
[Response: This issue was discussed ad nausem at Stoat – bring it up there. However, note that Kaufmann’s study and Mann’s study were different studies with different methodologies. Kaufmann’s used a straight up average of a priori selected and normalised temperature proxies – in that case there is no calibration step nor a valdiation step as there is the method used by Mann et al. In that case, the calibration of the records to the instrumental record is needed and that determines how a specific proxy fits in the scheme. If the varved proxy is contaminated by new non-climatic issues over the calibration period, it can’t be used in the Mann methodolgy (though the truncated version could still be useful for Kaufmann). Thus it is important to test whether the Mann results were robust to the non-inclusion of the potentially problematic proxies. Which he did. There is no other possible reconstruction that would use the proxy in another orientation. It is either in the way it was, or it isn’t included at all. Both options were published together in the PNAS paper. No correction needed. For Kaufmann the issues are different and he does have a choice about how to enter it in the process. Hence the correction in Science. – gavin]
How do you guys avoid overfitting your computer models to historical data ? I know there are a number of techniques. How have the computer models from years earlier predicted the temperatures in the years that follow ?
[Response: Good question. Some answers in our FAQ. And yes, look at the Hansen et al results. – gavin]
My questions is this,
With the computer models, would it be fair to say that they model all of the natural variations and mechanisms that we know of, i.e. Solar Energy, Volcanos, etc.?
[Response: Yes. There are details that make this a little more than straightforward. – gavin]
Also, if they don’t lead to the warming that we are evidencing that there must be another forcing of the climate other than what is already accounted for?
[Response: Pretty much. Just using these changes creates a large and growing divergence in surface, stratospheric temperatures, water vapour etc. – gavin]
And finally, humans are attributed as the forcing that is unaccounted for in the model? If there is a forcing that we are not taking into account in our models currently that is not attributable to humans, i.e. Cosmic Rays, Sun-Cloud connection, ocean current variability, wouldn’t that look the same in the model?
[Response: Not really. We put in the human forcings that we can quantify and see whether this better explains the changes (and they do). These experiments suggest tests and observations that can be used to distinguish this hypothesis from others. Other changes (such as CR effects) can be explored to see whether they would also help improve the match (they don’t). However, you are only left with plausible explanations of what has happened, not proof that this is an exact match. Different models and different groups come up with equally plausible explanations and the collective predictions from this group of models can be examined to see what is robust. -gavin]
From what I understand about models, and correct me if I am wrong, is that they reflect the inputs they are given and show the conditions that would exist with said inputs. Then we attempt to attribute any variations that we see which differ from what the model predicts. If it is warmer than the model suggest then it must be because of an external forcing. Why does this forcing have to be greenhouse gas levels? This would help satisfy a lot of questions I have. Thanks Gavin.
There is nothing inherently (intrinsicly) wrong with the scientific method and its entire literature of which climate science is a part is all treated equally by each and every science that uses its method. Science is septical and shooting down orthodoxy (AGW) is what it is all about so if orthodoxy is not shot down by the peer review then it stands and stand it does.
These leaked emails just show some of the politics of the practitioners of science I am sure but say little of the masses of data collected by CRU/GISS (seperate records)and hence say nothing tangiable about climate change.
Science is not suffering here. A few of its practitioners might be feeling a little uncomfortable perhaps as quotes and work is taken out of context to try and make them look like they are doing something wrong or colluding together to make the believe something it should not be.
So Gavin now claims that “trick” means “a good way to deal with a problem.” That’s odd. Back in a November 9, 2006 post entitled “Cuckoo Science” criticizing Christopher Moncton, “trick” meant “absurdities that occasionally pass for serious ’science’ on the web and in the media” and “concepts are being mangled, logic is being thrown to the winds, and completetly unjustified conclusions are being drawn.”
Gavin said in November 2006 that:
See https://www.realclimate.org/index.php/archives/2006/11/cuckoo-science/
[Response: Your theory is that once someone has a used a word with multiple meanings in one way, they are forever barred from using it in any other context or sense? Now keeping track of that would be a neat trick! – gavin]
I also want to express my thanks to the RealClimate team for their hard work in what is, I’m sure, a difficult time. A perspective on this debate from the vantage point of anthropology and science studies may not be welcome right now, but FWIW I’ve blogged about this issue here: http://borderlandsofscience.typepad.com/rhetoric-on-the-borderlan/2009/11/the-cru-hack-controversy.html
Good luck and thanks for fighting the good fight.
To be fair, “trick” is sometimes used by RealClimate in a non-nefarious context:
“The trick is to do this carefully so that as much climate information is retained while the growth function is discarded, and dendroclimatologists know how to do this quite well.”
So since all this stuff is supposedly cherry picked to have things taken out of context the only way you can prove everybody wrong is to open up your data and make whatever you have that the hackers missed open and public. I’m sure you’ll have a million excuses not to do so that of course have nothing to do with hiding anything but it’s the only way I’ll ever have any respect for this website or the CRU. Although I half believe it’s just all taken out of context I find it hard to believe there are so many things that sound so suspicious, I can tell you for a fact that not one of my e-mails sounds remotely that sinister.
Then again 70% or more of meaning is lost in a written message due to lack of voice inflection and body language, but now you’ve got 2 things to prove, human caused climate change and your own intentions/integrity.
Take a look at Figure 2 of this article:
http://www.skepticalscience.com/Why-is-Greenlands-ice-loss-accelerating.html
Basically, before 2000, Greenland wasn’t really losing much ice mass. However, SINCE 2000, Greenlands mass has dropped off the deep end. It’s really quite a remarkable difference: pre-2000 and post-2000. Perhaps this graph is reproduced for Antarctica, etc. (I don’t know). My question is: if, in the last 10 years, we’ve seen an acceleration in global ice-melting, wouldn’t that as a simple matter of physics HALT global warming over the same 10 years? Systems do not change temperature while constituents within them are changing phase: its a thermodynamic no-no (I believe its Gibbs Law, or something).
As to the email ‘controversy’: I guess I understand why RC feels a need to summarize and explain the 13 years of pilfered ‘private’ conversations, but I feel personally soiled to read any of it. As I indicated yesterday, everyone now knows that if you tortured someone, the denier community would be OK reading his confession, as long as you splashed it all over the internet. That’s just where their heads are at. But it doesn’t surprise all of us that they are just that morally bankrupt.
As an active researcher in the medical field I have a few general comments to the whole climate science story. My main concern is that there is too little caution applied to the interpretation of the ongoing science.
While I agree with you Gavin that the peer review process is necessary and helpful, I think that anyone who has conducted and tried to report contradicting science in any field has run into problems with getting their message out. This problem tends to increase when the field is prestigious and gets lots of attention. This is mainly due to the fact that influential researchers control the whole process of peer review and they have a strong incentive to propagate their own views, both due to financial interests and plain vanity. The same people usually control the review process at the funding agency as well, which amplifies the problem. These problems always create an exaggerated picture of consensus within a field. This is a fact and if you do conduct science at a certain level you will know about this. Hence it is not impossible that a whole field will be flawed for a long time and this has happened a lot of times before. Paradigm shifts due to the emergence of new technologies also occur at a regular interval and to be sure about anything in science is to expose yourself to later ridicule.
It seems to me that the science that is being conducted within the climate field is extremely difficult. To simplify it the whole field seems to be about correlations. Causality is theoretically extremely hard to dig out. You have no hard endpoints, no real experimental models where you can test your theories (apart from mathematical ones) and the timeframe you are working with is completely un-human. It is like trying to catch the influence of tobacco smoking on the development of lung cancer during 5 minutes of smoking.
The current controversies makes for interesting reading though and as has been pointed out previously there are a number of good reasons to move away from fossil fuels apart from the possibility of the AGW theories to be accurate. Just to be able to halt the current monetary enrichment of Norway would in many peoples eyes be reason enough=).
Was there an actual attempt to “hack” RealClimate or was there just someone attempting to post (as was done at other blogs)? If an actual hack attempt, has it been reported to the police? Details?
[Response: Yes. No. Here. – gavin]
Re:
27
Ron R. says:
23 November 2009 at 2:53 AM
Dave Mc said Nov 22 @6:44PM: “Experts from across the the world in Climate should be invited to take part in peer review, irrespective of their standpoint.”
Not if their standpoint has been bought and paid for by Big Oil and King Coal.
“The opinion of an AGW ’sceptic’ is as valuable as that of a AGW ‘believer’.”
Again, not if their standpoint has been bought and paid for by Big Oil and King Coal.
***************
Ron, are you more comfortable with a standpoint paid for by Big Government? Keep in mind government has access to a lot more of your money than Big Oil/Coal, is always looking for ways to get more of it, and for expanding its control of everything (lots and lots of cushy jobs at stake here). Anyway, let’s get out of the search for motivation (at best it’s a saw-off) , accept that a source of information is no guarantee of its truth and let the best science available go about its proper business of discovering truth wherever it can be found.
Ron
Gavin,
Nothing like this has ever happened before in the history of science. (Well, some of Marie Curie’s love letters with a married man were published in a newspaper and she had to hide in the countryside to escape reporters, but that’s a mere anecdote.) I hope your institution is supporting you whole-heartedly in this work — people may not realize how important it is. Aside from weird cranks like Velikovsky and their coteries of crankish supporters, never before has an entire scientific community been accused of dishonesty. And yes, you represent the scientific community, not just the handful of folks named in the stolen emails. The logic of the attack is to undermine the personal credibiilty of each of the thousands of researchers who participated in the statements of the IPCC, National Academy of Sciences, American Physical Society, etc.
The real issue is trust, as many of the comments have pointed out. Social scientists will tell you this has become key in all science-technology-society relations. After all, government leaders are planning deep changes in the world economy based, essentially, on trust in the scientific community. They sure can’t run climate models themselves.
Attacks on the integrity of climate scientists contribute to a broader suspicion of scientists in general. This suspicion has enormous potential for harm; consider for example the resistance to vaccination. So Gavin, you are on the front lines in a battle of the highest significance.
How best to conduct it? We don’t understand very well how to respond to such attacks, as politicians on both left and right who lost elections thanks to calumnies can tell you. One thing we know is that it’s important to answer criticism immediately with facts — and immediately means within 24 hours or less. So your labors are of the very first importance. We could wish that many more scientists would join the effort, but people turn to science because they desire the objective, impersonal, painstaking pursuit of the truth; most are temperamentally unfit for such controversies and prefer to stick to their work.
One bit of gratuitous advice. Resist the temptation to answer, for the Nth time, any query that the questioner can quickly resolve by searching realclimate.org or a dozen other places. You just don’t have the time for such lazy people. Concentrate on detailed answers to the specific criticisms raised by the emails. And resist the temptation to repeat your answers if you can do it more quickly by saying, “Please see response to #xxx” or, “Please search this thread for the words ‘zzzz’.” This can not only save you time and emotional/intellectual energy, but will help readers notice the extent of repetition and the failure of many to even bother to search the thread.
Say, do any of you readers want to help by suggesting additions or revisions to the listing at the top where responses to common attacks can be found, whether on realclimate or elsewhere? I’d do it myself but I’ve got this paper to write…
Cheers,
Spencer
Well, I’ve run out of social science explanations. This hacking, and all the other frenetic denialist pranks and tactics in the run up to Copenhagen appears to be the work of the devil himself. And I’d really like to pin it on the culprits, but…
A great TWILIGHT ZONE episode, “The Howling Man” (see: http://en.wikipedia.org/wiki/The_Howling_Man ), explained how we all do bad things due to our human nature, but when REALLY bad things happen (like world wars, or the denialist war on climate science), it’s the devil who’s gotten loose and gone a rampage.
Re 79&81
Martin Vermeer has pretty well nailed it- the basis of debate has moved beyond hypothesis testing to the “It’s the physics, stupid!” stage , but that does not mean we are out of the woods on parameter estimation.
A threefold range ( 1.5-4.5 K) was the best 4AR could do on climate sensitivity , and until the historical spectrum of peer reviewed estimates stops rattling around in a multi-K range , and starts to converge on a two decimal place ‘consensus’ value, policy analysts may continue to view the 90% rhetoric with condign skepticism.
It matters not whether the letters were purloined from the Climate Research Unit, the American Petroleum Institute-, or their respective PR firms.
That both sides in the present controversy should stoop to reading other peoples’ mail reflects as little credit on the honor of the scientific profession as some of the emails in question.
From one of the E mails.
One prominent IPCC researcher concerned about this gap in knowledge is Simon Tett from the Hadley Centre for climate modelling at the Meteorological Office, home to one of the world’s five leading global circulation models, capable of recreating a mathematical version of how the atmosphere works and of running simulations of climatic changes over decades or even centuries. He says that “in the past, our estimates of natural variability have been based on climate models.” But this autumn [date?], he says, those estimates have been thrown into turmoil by a paper published in the journal The Holocene. In it, Tim Barnett of the Scripps Institution of Oceanography, part of the University of California at San Diego, compared model estimates of natural temperature fluctuations over the past 400 years with the best evidence from the real world — from instruments in the past century and “proxy data”, such as Briffa’s tree rings, from before that.
The result was bad news for the modellers. The two models examined — one German, the other American — generated a natural variability of around 0.1 degree C per century. This was less than half that revealed in the proxy data. “Of course we don’t have to believe the proxy data. They certainly have problems attached to them. But my belief is that they both models, and proxy data too, underestimate real variability,” says Barnett
The models’ error was not, perhaps, too surprising. As Barnett points out, they do not include vital “forcing” mechanisms that alter temperature, such as solar cycles and volcanic eruptions. Nor can they yet mimic the strength of the largest year-on-year variability in the natural system, the El Nino oscillation in the Pacific Ocean, which has a global impact on climate.
Nonetheless, the findings should serve as a warning, Barnett says, that “the current models cannot be used in rigorous tests for anthropogenic signals in the real world”. If they are they “might lead us to believe that an anthropogenic signal had been found when, in fact, that may not be the case.”
Explain please.
[Response: This is a piece by Fred Pearce, for New Scientist, written in….. 1996. He still has a compuserve email address! Things have changed (just a little) since then, and you can read about that in the IPCC AR4 (2007) report. – gavin]
I read the email with this comment from Phil Jones in (1189722851.txt). I think it’s interesting that he suggests backdating documents.
——–
Ammann/Wahl – try and change the Received date! Don’t give those skeptics something to amuse themselves with.
Cheers
Phil
[Response: Context is all. The published Amman and Wahl paper had (has?) a typo in the “Received By” date, saying that it was received on 22 August 2000, when that was actually received in 2006. Amused? ;) – gavin]
Hi there,
I’m kinda right down the middle on climate change. I think the planet’s climate is slowly changing, but I’m not positive its all because of man. When this story broke, I wasn’t really sure what to believe because I know people will spin this in any direction they can. But I have the following questions:
1. What effect does solar radiation have on the climate of the planet, and how do climate models take this into account? How is solar radiation measured?
[Response: See some of our papers on the topic i.e Shindell et al (2006) or Schmidt et al (2004). – gavin]
2. What effect does geothermal activity have on the climate of the planet, and how do climate models take this into account? My thought here is that how do we measure for changes in temperature and pressure below the earth’s crust?
[Response: Generally this is ignored in the global models. It is a very small term 0.07 W/m2 on average compared to 240 W/m2 absorbed from the sun. In specific applications – deep sea vent communities, borehole analysis, permafrost stability, it is important, but it isn’t thought to be relevant to climate change. – gavin]
3. As someone who builds financial models, I know there are a great number of assumptions that have to be made in order for them to work. What are some standard assumptions made in climate models, and could these assumption lead to a materially different result if they were changed?
[Response: See our FAQ on the topic. – gavin]
Thanks for responding if you get a chance.
There has been a lively exchange here and on other sites about the legal ramifications of the leak of illegally hacked personal correspondence of scientists working directly or indirectly for the public domain, and possibly themselves on shaky legal ground WRT the FOIA.
The legal aspects are really not that pertinent, as it is highly unlikely that anyone on either side is going to be prosecuted.
The suggestion has been made that the timing of this leak was politically orchestrated to present an already shaky Copenhagen conference even greater difficulties in arriving at any binding decisions to stop man-made global warming.
I agree that the fallout here will be political, rather than legal.
There is no question that Copenhagen will suffer from the exposé that a relatively small group of highly influential publicly employed scientists who control and interpret the scientific data being used by the IPCC to support the AGW premise, who peer review each other’s reports and influence what does and does not get published in scientific journals, are colluding in their own interest and as political activists.
And even more damning is the evidence, which shows that they are doing so rather arrogantly with an apparent intolerance for dissent or differing scientific opinion.
It has been suggested that this leaked personal correspondence only shows that scientists are human, like everyone else.
I believe it goes much further than that, and that these scientists have hurt the very cause, which they are championing by their ill-advised and arrogant behavior.
This is quite apart from the validity of the science supporting (a) potentially alarming AGW, (b) minor AGW with no alarming consequences or (c) no appreciable AGW at all, which is a totally independent question
But I would welcome others’ opinions on this (particularly those of Gavin).
Max