Currently, a few errors –and supposed errors– in the last IPCC report (“AR4”) are making the media rounds – together with a lot of distortion and professional spin by parties interested in discrediting climate science. Time for us to sort the wheat from the chaff: which of these putative errors are real, and which not? And what does it all mean, for the IPCC in particular, and for climate science more broadly?
Let’s start with a few basic facts about the IPCC. The IPCC is not, as many people seem to think, a large organization. In fact, it has only 10 full-time staff in its secretariat at the World Meteorological Organization in Geneva, plus a few staff in four technical support units that help the chairs of the three IPCC working groups and the national greenhouse gas inventories group. The actual work of the IPCC is done by unpaid volunteers – thousands of scientists at universities and research institutes around the world who contribute as authors or reviewers to the completion of the IPCC reports. A large fraction of the relevant scientific community is thus involved in the effort. The three working groups are:
Working Group 1 (WG1), which deals with the physical climate science basis, as assessed by the climatologists, including several of the Realclimate authors.
Working Group 2 (WG2), which deals with impacts of climate change on society and ecosystems, as assessed by social scientists, ecologists, etc.
Working Group 3 (WG3) , which deals with mitigation options for limiting global warming, as assessed by energy experts, economists, etc.
Assessment reports are published every six or seven years and writing them takes about three years. Each working group publishes one of the three volumes of each assessment. The focus of the recent allegations is the Fourth Assessment Report (AR4), which was published in 2007. Its three volumes are almost a thousand pages each, in small print. They were written by over 450 lead authors and 800 contributing authors; most were not previous IPCC authors. There are three stages of review involving more than 2,500 expert reviewers who collectively submitted 90,000 review comments on the drafts. These, together with the authors’ responses to them, are all in the public record (see here and here for WG1 and WG2 respectively).
Errors in the IPCC Fourth Assessment Report (AR4)
As far as we’re aware, so far only one–or at most two–legitimate errors have been found in the AR4:
Himalayan glaciers: In a regional chapter on Asia in Volume 2, written by authors from the region, it was erroneously stated that 80% of Himalayan glacier area would very likely be gone by 2035. This is of course not the proper IPCC projection of future glacier decline, which is found in Volume 1 of the report. There we find a 45-page, perfectly valid chapter on glaciers, snow and ice (Chapter 4), with the authors including leading glacier experts (such as our colleague Georg Kaser from Austria, who first discovered the Himalaya error in the WG2 report). There are also several pages on future glacier decline in Chapter 10 (“Global Climate Projections”), where the proper projections are used e.g. to estimate future sea level rise. So the problem here is not that the IPCC’s glacier experts made an incorrect prediction. The problem is that a WG2 chapter, instead of relying on the proper IPCC projections from their WG1 colleagues, cited an unreliable outside source in one place. Fixing this error involves deleting two sentences on page 493 of the WG2 report.
Sea level in the Netherlands: The WG2 report states that “The Netherlands is an example of a country highly susceptible to both sea-level rise and river flooding because 55% of its territory is below sea level”. This sentence was provided by a Dutch government agency – the Netherlands Environmental Assessment Agency, which has now published a correction stating that the sentence should have read “55 per cent of the Netherlands is at risk of flooding; 26 per cent of the country is below sea level, and 29 per cent is susceptible to river flooding”. It surely will go down as one of the more ironic episodes in its history when the Dutch parliament last Monday derided the IPCC, in a heated debate, for printing information provided by … the Dutch government. In addition, the IPCC notes that there are several definitions of the area below sea level. The Dutch Ministry of Transport uses the figure 60% (below high water level during storms), while others use 30% (below mean sea level). Needless to say, the actual number mentioned in the report has no bearing on any IPCC conclusions and has nothing to do with climate science, and it is questionable whether it should even be counted as an IPCC error.
Some other issues
African crop yields: The IPCC Synthesis Report states: “By 2020, in some countries, yields from rain-fed agriculture could be reduced by up to 50%.” This is properly referenced back to chapter 9.4 of WG2, which says: “In other countries, additional risks that could be exacerbated by climate change include greater erosion, deficiencies in yields from rain-fed agriculture of up to 50% during the 2000-2020 period, and reductions in crop growth period (Agoumi, 2003).” The Agoumi reference is correct and reported correctly. The Sunday Times, in an article by Jonathan Leake, labels this issue “Africagate” – the main criticism being that Agoumi (2003) is not a peer-reviewed study (see below for our comments on “gray” literature), but a report from the International Institute for Sustainable Development and the Climate Change Knowledge Network, funded by the US Agency for International Development. The report, written by Morroccan climate expert Professor Ali Agoumi, is a summary of technical studies and research conducted to inform Initial National Communications from three countries (Morocco, Algeria and Tunisia) to the United Nations Framework Convention on Climate Change, and is a perfectly legitimate IPCC reference.
It is noteworthy that chapter 9.4 continues with “However, there is the possibility that adaptation could reduce these negative effects (Benhin, 2006).” Some examples thereof follow, and then it states: “However, not all changes in climate and climate variability will be negative, as agriculture and the growing seasons in certain areas (for example, parts of the Ethiopian highlands and parts of southern Africa such as Mozambique), may lengthen under climate change, due to a combination of increased temperature and rainfall changes (Thornton et al., 2006). Mild climate scenarios project further benefits across African croplands for irrigated and, especially, dryland farms.” (Incidentally, the Benhin and Thornton references are also “gray”, but nobody has complained about them. Could there be double standards amongst the IPCC’s critics?)
Chapter 9.4 to us sounds like a balanced discussion of potential risks and benefits, based on the evidence available at the time–hardly the stuff for shrill “Africagate!” cries. If the IPCC can be criticized here, it is that in condensing these results for its Synthesis Report, important nuance and qualification were lost – especially the point that the risk of drought (defined as a 50% downturn in rainfall) “could be exacerbated by climate change”, as chapter 9.4 wrote – rather than being outright caused by climate change.
Trends in disaster losses: Jonathan Leake (again) in The Sunday Times accused the IPCC of wrongly linking global warming to natural disasters. The IPCC in a statement points out errors in Leake’s “misleading and baseless story”, and maintains that the IPCC provided “a balanced treatment of a complicated and important issue”. While we agree with the IPCC here, WG2 did include a debatable graph provided by Robert Muir-Wood (although not in the main report but only as Supplementary Material). It cited a paper by Muir-Wood as its source although that paper doesn’t include the graph, only the analysis that it is based on. Muir-Wood himself has gone on record to say that the IPCC has fairly represented his research findings and that it was appropriate to include them in the report. In our view there is no IPCC error here; at best there is a difference of opinion. Obviously, not every scientist will always agree with assessments made by the IPCC author teams.
Amazon forest dieback: Leake (yet again), with “research” by skeptic Richard North, has also promoted “Amazongate” with a story regarding a WG2 statement on the future of Amazonian forests under a drying climate. The contested IPCC statement reads: “Up to 40% of the Amazonian forests could react drastically to even a slight reduction in precipitation; this means that the tropical vegetation, hydrology and climate system in South America could change very rapidly to another steady state, not necessarily producing gradual changes between the current and the future situation (Rowell and Moore, 2000).” Leake’s problem is with the Rowell and Moore reference, a WWF report.
The roots of the story are in two blog pieces by North, in which he first claims that the IPCC assertions attributed to the WWF report are not actually in that report. Since this claim was immediately shown to be false, North then argued that the WWF report’s basis for their statement (a 1999 Nature article by Nepstad et al.) dealt only with the effects of logging and fire –not drought– on Amazonian forests. To these various claims Nepstad has now responded, noting that the IPCC statement is in fact correct. The only issue is that the IPCC cited the WWF report rather than the underlying peer-reviewed papers by Nepstad et al. These studies actually provide the basis for the IPCC’s estimate on Amazonian sensitivity to drought. Investigations of the correspondence between Leake, scientists, and a BBC reporter (see here and here and here) show that Leake ignored or misrepresented explanatory information given to him by Nepstad and another expert, Simon Lewis, and published his incorrect story anyway. This “issue” is thus completely without merit.
Gray literature: The IPCC cites 18,000 references in the AR4; the vast majority of these are peer-reviewed scientific journal papers. The IPCC maintains a clear guideline on the responsible use of so-called “gray” literature, which are typically reports by other organizations or governments. Especially for Working Groups 2 and 3 (but in some cases also for 1) it is indispensable to use gray sources, since many valuable data are published in them: reports by government statistics offices, the International Energy Agency, World Bank, UNEP and so on. This is particularly true when it comes to regional impacts in the least developed countries, where knowledgeable local experts exist who have little chance, or impetus, to publish in international science journals.
Reports by non-governmental organizations like the WWF can be used (as in the Himalaya glacier and Amazon forest cases) but any information from them needs to be carefully checked (this guideline was not followed in the former case). After all, the role of the IPCC is to assess information, not just compile anything it finds. Assessment involves a level of critical judgment, double-checking, weighing supporting and conflicting pieces of evidence, and a critical appreciation of the methodology used to obtain the results. That is why leading researchers need to write the assessment reports – rather than say, hiring graduate students to compile a comprehensive literature review.
Media distortions
To those familiar with the science and the IPCC’s work, the current media discussion is in large part simply absurd and surreal. Journalists who have never even peeked into the IPCC report are now outraged that one wrong number appears on page 493 of Volume 2. We’ve met TV teams coming to film a report on the IPCC reports’ errors, who were astonished when they held one of the heavy volumes in hand, having never even seen it. They told us frankly that they had no way to make their own judgment; they could only report what they were being told about it. And there are well-organized lobby forces with proper PR skills that make sure these journalists are being told the “right” story. That explains why some media stories about what is supposedly said in the IPCC reports can easily be falsified simply by opening the report and reading. Unfortunately, as a broad-based volunteer effort with only minimal organizational structure the IPCC is not in a good position to rapidly counter misinformation.
One near-universal meme of the media stories on the Himalaya mistake was that this was “one of the most central predictions of the IPCC” – apparently in order to make the error look more serious than it was. However, this prediction does not appear in any of the IPCC Summaries for Policy Makers, nor in the Synthesis Report (which at least partly explains why it went unnoticed for years). None of the media reports that we saw properly explained that Volume 1 (which is where projections of physical climate changes belong) has an extensive and entirely valid discussion of glacier loss.
What apparently has happened is that interested quarters, after the Himalyan glacier story broke, have sifted through the IPCC volumes with a fine-toothed comb, hoping to find more embarrassing errors. They have actually found precious little, but the little they did find was promptly hyped into Seagate, Africagate, Amazongate and so on. This has some similarity to the CRU email theft, where precious little was discovered from among thousands of emails, but a few sentences were plucked out of context, deliberately misinterpreted (like “hide the decline”) and then hyped into “Climategate”.
As lucidly analysed by Tim Holmes, there appear to be a few active leaders of this misinformation parade in the media. Jonathan Leake is carrying the ball on this, but his stories contain multiple errors, misrepresentations and misquotes. There also is a sizeable contingent of me-too journalism that is simply repeating the stories but not taking the time to form a well-founded view on the topics. Typically they report on various “allegations”, such as these against the IPCC, similar to reporting that the CRU email hack lead to “allegations of data manipulation”. Technically it isn’t even wrong that there were such allegations. But isn’t it the responsibility of the media to actually investigate whether allegations have any merit before they decide to repeat them?
Leake incidentally attacked the scientific work of one of us (Stefan) in a Sunday Times article in January. This article was rather biased and contained some factual errors that Stefan asked to be corrected. He has received no response, nor was any correction made. Two British scientists quoted by Leake – Jonathan Gregory and Simon Holgate – independently wrote to Stefan after the article appeared to say they had been badly misquoted. One of them wrote that the experience with Leake had made him “reluctant to speak to any journalist about any subject at all”.
Does the IPCC need to change?
The IPCC has done a very good job so far, but certainly there is room for improvement. The review procedures could be organized better, for example. Until now, anyone has been allowed to review any part of the IPCC drafts they liked, but there was no coordination in the sense that say, a glacier expert was specifically assigned to double-check parts of the WG2 chapter on Asia. Such a practice would likely have caught the Himalayan glacier mistake. Another problem has been that reports of all three working groups had to be completed nearly at the same time, making it hard for WG2 to properly base their discussions on the conclusions and projections from WG1. This has already been improved on for the AR5, for which the WG2 report can be completed six months after the WG1 report.
Also, these errors revealed that the IPCC had no mechanism to publish errata. Since a few errors will inevitably turn up in a 2800-page report, obviously an avenue is needed to publish errata as soon as errors are identified.
Is climate science sound?
In some media reports the impression has been given that even the fundamental results of climate change science are now in question, such as whether humans are in fact changing the climate, causing glacier melt, sea level rise and so on. The IPCC does not carry out primary research, and hence any mistakes in the IPCC reports do not imply that any climate research itself is wrong. A reference to a poor report or an editorial lapse by IPCC authors obviously does not undermine climate science. Doubting basic results of climate science based on the recent claims against the IPCC is particularly ironic since none of the real or supposed errors being discussed are even in the Working Group 1 report, where the climate science basis is laid out.
To be fair to our colleagues from WG2 and WG3, climate scientists do have a much simpler task. The system we study is ruled by the well-known laws of physics, there is plenty of hard data and peer-reviewed studies, and the science is relatively mature. The greenhouse effect was discovered in 1824 by Fourier, the heat trapping properties of CO2 and other gases were first measured by Tyndall in 1859, the climate sensitivity to CO2 was first computed in 1896 by Arrhenius, and by the 1950s the scientific foundations were pretty much understood.
Do the above issues suggest “politicized science”, deliberate deceptions or a tendency towards alarmism on the part of IPCC? We do not think there is any factual basis for such allegations. To the contrary, large groups of (inherently cautious) scientists attempting to reach a consensus in a societally important collaborative document is a prescription for reaching generally “conservative” conclusions. And indeed, before the recent media flash broke out, the real discussion amongst experts was about the AR4 having underestimated, not exaggerated, certain aspects of climate change. These include such important topics as sea level rise and sea ice decline (see the sea ice and sea level chapters of the Copenhagen Diagnosis), where the data show that things are changing faster than the IPCC expected.
Overall then, the IPCC assessment reports reflect the state of scientific knowledge very well. There have been a few isolated errors, and these have been acknowledged and corrected. What is seriously amiss is something else: the public perception of the IPCC, and of climate science in general, has been massively distorted by the recent media storm. All of these various “gates” – Climategate, Amazongate, Seagate, Africagate, etc., do not represent scandals of the IPCC or of climate science. Rather, they are the embarrassing battle-cries of a media scandal, in which a few journalists have misled the public with grossly overblown or entirely fabricated pseudogates, and many others have naively and willingly followed along without seeing through the scam. It is not up to us as climate scientists to clear up this mess – it is up to the media world itself to put this right again, e.g. by publishing proper analysis pieces like the one of Tim Holmes and by issuing formal corrections of their mistaken reporting. We will follow with great interest whether the media world has the professional and moral integrity to correct its own errors.
PS. A new book by Realclimate-authors David Archer and Stefan Rahmstorf critically discussing the main findings of the AR4 (all three volumes) is just out: The Climate Crisis. None of the real or alleged errors are in this book, since none of those contentious statements plucked from the thousands of pages appeared to be “main findings” that needed to be discussed in a 250-page summary.
PPS. Same thing for Mike’s book Dire Predictions: Understanding Global Warming, which bills itself as “The illustrated guide to the findings of the IPCC”. Or Gavin’s “Climate Change: Picturing the Science” – which does include a few pictures of disappearing glaciers though!
Update 24 March: Simon Lewis has made an official complaint to the Press Complaints Commission about Leake’s Amazon story.
Update 29 March: IPCC Chairman Rajendra Pachauri has published an interesting article in the Guardian.
Has anyone here read this paper and have any comment on it? It ever so slightly raised my eyebrows.
http://www.pnas.org/content/107/2/576
Atmospheric CO2 concentrations during ancient greenhouse climates were similar to those predicted for A.D. 2100
Breeker, Sharp, McFadden (2009)
PNAS 2010 107:576-580; published online before print December 28, 2009, doi:10.1073/pnas.0902323106
Summary at Nature Reports: http://www.nature.com/climate/2010/1002/full/climate.2010.03.html
Insights from earth.
“…They found that calcite forms in soils only during the hottest and driest times of year, rather than year-round. Using this information, the team recalculated atmospheric CO2 concentrations over the past 400 million years. While previous studies point to atmospheric CO2 concentrations of 3,000–4,000 parts per million during ancient greenhouse events, Breecker’s team revises this down to around 1,000 parts per million.
Their findings are in line with estimates from plant fossils, which have previously been regarded as controversial. The study suggests that a hothouse world may be closer to present-day reality than once believed.”
Ferrocious says “The kind of scenarios discussed in AR4 and the kind of recommendations that come out of it need to be in the 99.9% likely range to be justified because the consequences are so large.”
Bullshit! The costs of mitigating climate change amount to 3-5% of the global economy for a period of 30 years or so. We have squandered similar amounts to bail out billionaires and wage war against nations that posed no threat to us! What is more, most of the expenditure would be needed in any case to develop the new energy economy we will need to cope with depletion of fossil fuels–also likely to occur this century.
And what you are proposing is in effect a sliding scale for scientific truth depending on whether we like the consequences. I would absolutely love to see you calculate the expectation value for a game of rou-l-ette according to those rules. “Hmm, let’s see if I win, that’s good. I’ll only require 60% CL for that, but I’ll take the lower bound of the 99.9% CL about whether the ball lands on green…”
You are confusing the two stages of the risk calculus. The standards for science do not vary. The standards for risk depend on the severity of the consequences and the cost–that is where different threats demand different levels of confidence. However, given that we are talking about changing the climate of the only habitable planet we know of at a time when global population will be climbing to 9-10 billion people, I don’t think you’ll like the answer.
Dude, learn what the hell you are talking about!
J Bowers (551) — I read the PNAS paper, but I’m an amateur at this. At a minimum we may conclude that we don’t know very well ancient CO2 levels above, say, 1000 ppm and so ought to work harder at avoiding further increases in atmospheric CO2.
Scary.
BFJ says: 21 February 2010 at 3:57 PM
Hey, “BFJ”, why don’t you publish here some specific allegations of fraud against specific individuals, along with your own real name and address? Superficially you sound very confident, but it takes no rocket scientist to notice you mention no proper names in your post, your own or any others. So you’re not confident? What’s the deal?
If you don’t deliver any facts, you have no credibility.
Loser.
Ferrocious says “The kind of scenarios discussed in AR4 and the kind of recommendations that come out of it need to be in the 99.9% likely range to be justified because the consequences are so large.”
I was diagnosed with Stage IIb lung cancer last year. The surgery to remove my right lung cost $85k – of which I paid about $5k in deductibles and copays. I had no disability insurance, so I had to take four months off work without pay and another month half time – that’s another $24k.
The five year survival rate with the surgery is 50%. Without surgery, I would have had a 5% chance of surviving for five years.
According to ferocious, I was a fool to waste 1/3 of my annual income to have a 50% chance of surviving the next five years. I should have done nothing until there was a treatment with a 99.9% chance of working.
OK, so I read the “Africagate” link given above. The piece says that the essential problem is that Agoumi 2003 “doesn’t say” what the IPCC claims.
So I went to Agoumi 2003 and found this:
“Studies on the future of vital agriculture in the region
have shown the following risks, which are linked to
climate change:
• greater erosion, leading to widespread soil degradation;
• deficient yields from rain-based agriculture of up
to 50 per cent during the 2000–2020 period;”
A direct cut-and-paste, folks. (That’s where the cute little bullet points came from; I wouldn’t have bothered.)
Can you say “Richard Northgate?” Boy, I can’t wait for the fame and glory that comes with revealing that a reporter LIED.
Oops, sorry for the sarcasm.
553 David B. Benson: “Scary”
You’re the second person to say something on those lines. I’m actually hoping someone will say it’s rubbish. Maybe I should post it at WUWT or CA ;)
Re: Africagate.
Regardless of the provenance of the quote, it is pretty obvious — since we’re halfway into the 2000 – 2020 period — that the prediction is not true. To my knowledge, crop yields in Africa haven’t fallen 25% in the last decade. So unless there is expectation of massively accelerated decline in the next decade, that 50% figure won’t hold up.
If you want a more detailed and reasonable analysis of the threat to agriculture in Africa, here is a useful report from FAO:
ftp://ftp.fao.org/docrep/fao/012/ak915e/ak915e00.pdf
Potentially 50% loss in yield by 2100. Lack of funding for adaptation is the urgent issue.
Don Shor, it’s not “Africa.” It’s “North Africa.” And it wasn’t 50%, it was “up to 50%.”
See also:
http://www.newscientist.com/article/mg19526144.000-global-warming-is-changing-the-worlds-rain.html
http://www.africanagricultureblog.com/2008/03/north-africa-to-develop-drought-salt.html
http://www.bilateralchamber.org/newsletter/2009/dec/agriculture.html
I’d like to see a study on what has happened since 2003.
Nice job moving the goalpost, though.
Let me move it back: the IPCC did not lie, Richard North did. That was what was being discussed on the subthread.
“558
Don Shor says:
22 February 2010 at 12:50 PM
Re: Africagate.
Regardless of the provenance of the quote, it is pretty obvious — since we’re halfway into the 2000 – 2020 period — that the prediction is not true. ”
It is?
Please explain the lost yields in Africa, then and how you have worked out the sources of their loss (and the gains that should have occurred by farming practices and the whole GM-crop-will-solve-third-world-hunger bounty that hasn’t appeared).
The Bugle episode 103 (link to mp3 download) had a fine discussion on climate denialism. They compare the use of the errors in AR 4 to discredit all of climate science to Hillary and Tenzing finding a snowman on top of Everest and declaring “I can’t believe it! Someone told us this was a mountain, and it’s just a snowman with a 25,000 foot skirt!”
Could someone at RealClimate.org help me?
You write:
” Himalayan glaciers: : …There are also several pages on future glacier decline in Chapter 10 (“Global Climate Projections”), where the proper projections are used e.g. to estimate future sea level rise”
I opened the PDF file, 100 pages in size, you linked to for Chapter 10. Not having time to sift through 100 pages I used the internal search feature. Trying several times I got no matches for “Himalayan”. Should I use another keyword?
And my second question (still intersted in the first, but this even more pressing) what is the actual current best estimate range for projections? I found one online talk overheads (I think linked to from realclimate.org) a few weeks ago, I think the “how long before they would be mostly (not entirely) melted” was in the range of 150 years. Is that right?
Third and last, is this statement by a 2004 article by Monbiot true, “the Himalayan glaciers which feed the Ganges, the Bramaputra, the Mekong, the Yangtze and the other great Asian rivers are likely to disappear within 40 years.”?
Thanks in advance.
How come all errors in these reports seem to show the problem as being bigger than it really is? If these were truly accidents in data entry, wouldn’t there be errors that didn’t support their position?
may as well ask, geoff, why EVERY SINGLE THING that is unknown or poorly understood in climage science is, when picked up by the denialcrowd, ALWAYS going to cap global warming thereby making it nothing we have to do anything about.
But you neglect to note that not everything in the IPCC report IS turning out to say it’s worse than really eventuates:
http://www.ossfoundation.us/projects/environment/global-warming/myths/images/arctic/arctic_sea_ice_extent6_800pxW.jpg/view
So now what to your theory of “if it were truly accidents”?
Here is an error that doesn’t support a position YOU have labeled them with of “AGW is going to be worse than we said”.
After asking for comments on, ‘Atmospheric CO2 concentrations during ancient greenhouse climates were similar to those predicted for A.D. 2100. Breeker, Sharp, McFadden (2009).’ another paleoclimate paper on CO2 and temperature cropped up:
Palaeoclimate: Global warmth with little extra CO2. Birgit Schneider & Ralph Schneider (2010).
http://www.nature.com/ngeo/journal/v3/n1/full/ngeo736.html
Nature Geoscience 3, 6–7 (1 January 2010) | doi:10.1038/ngeo736
Clive Hamilton at Australia’s ABC says: “A new study concludes that an average warming of 3-4°C (which means 7-8°C on land), previously thought to be associated with carbon dioxide concentrations of 500-600 ppmv, is now believed to be associated with concentrations of only 360-420 ppmv, a range that covers the current concentration of 385 ppmv, rising at 2 ppmv per annum. If confirmed by further research, the implications of this are terrifying.”
http://www.abc.net.au/unleashed/stories/s2829295.htm
Any comments? Anyone read the full paper? Can someone please debunk it…
geoff,
There are such errors, but they don’t get pointed out by the denialists.
Re: 565 (J Bowers)
I’ve just made similar observations following the artice “Good news for the earth’s climate system?”; see comment 377 .
I hadn’t actually noticed the Schneiders’ paper, but it does seem consistent with other recent papers on the Pliocene which I discussed. Tripati et al’s paper is even scarier, since they reconstruct CO2 at only 350ppm during the mid-Pliocene.
I find it rather concerning that, while all these dramatic papers are hitting the journals, the front page of “Real Climate” is entirely devoted to media storms and rebuttals; including rebutting Fred Pearce in “The Guardian” of all sources! Something’s going horribly, horribly wrong.
Dr Nick Bone, thanks for pointing out the other work …. I think ;)
Dr. Stephen Schneider, on NPR’s Science Friday (http://www.npr.org/templates/story/story.php?storyId=123892162): “The IPCC, the Intergovernmental Panel on Climate Change, indeed is a human institution. You know, it’s 200 scientists. It goes through multiple rounds of review. It’s people from all over the world. It’s got to have mistakes. That’s not the problem.
The problem is when it makes a few mistakes, and these mistakes were largely discovered by the well-oiled, multimillion-dollar disinformation engines and the Competitive Enterprise Institute and other things, they pointed it out. You know what? They have a right, and they should have.
IPCC, I think, badly responded by calling the ones who pointed it out voodoo scientists. That’s ridiculous. What you have to say is thank you, we’ll check it. When we checked it, two or three of those errors were really bad.
What they didn’t say, and this is the fraudulent part, is how can you report just a few mistakes and not report the overall track record of the group? It’s a man-bites-dog story. So here’s the temple of, you know, intellectual science caught in a few errors, and indeed they are, but not to say that when there’s a thousand conclusions, and no matter how hard the guys search from the other side, they’ve only found three wrong, my view is: Give me a crack team, I could probably find 10.”
Nick: “IPCC, I think, badly responded by calling the ones who pointed it out voodoo scientists.”
Nick, you’re assigning that comment a context it doesn’t have.
The “Voodoo Science” claims are for those who say that “It’s the Iris Effect!” or “CO2 can’t do it because greenhouse gas theory breaks the laws of thermodynamics!”
These ARE Voodoo Science.
And that is where the claim for voodoo science comes from.
For other Voodoo Science complaints that denialists (who claim “skepticism”), see the non-exhaustive list here:
http://www.skepticalscience.com/argument.php
I would like to inform you that this statement of yours are wrong:
“Gray literature: The IPCC cites 18,000 references in the AR4; the vast majority of these are peer-reviewed scientific journal papers.”
There are thousands of references to gray literature in the IPCC reports. I assume that you (realclimate) as hardworking serious scientists are interested to nuance your statement.
See the post here for a through anaylysis of gray literature in the IPCC:
http://rogerpielkejr.blogspot.com/2010/03/gray-literature-in-ipcc-tar-guest-post.html
[Response: Actually, simple logic dictates that you can’t prove a statement about AR4 wrong by looking at TAR. Even in TAR however, a large majority of references were from peer-reviewed journals, and in WG1, it clearly is the ‘vast majority’ (84% according to you). I have no reason to suspect that this is dramatically different in AR4, and indeed, I might expect a higher proportion because of the increased attention that AR4 got from the reviewers. But as we stated above, the grey literature – particularly in WG2 and WG3 – is essential for assessing impacts and responses. No-one has ever denied this. – gavin]
For anyone reading who’s new to this whole area, the acronyms refer to the time of publication of the edition.
TAR: “Third Assessment Report (TAR) of the IPCC, published in 2001.”
“The IPCC has started work on the preparation of its Fifth Assessment Report … The Fourth Assessment Report (AR4) was released in 2007.”
http://www.ipcc.ch/
Kids, when writing school papers, always check for a newer edition of whatever source you found on the shelves. Don’t accidentally rely on outdated sources.
Gavin, I “extrapolated” my results, under the assumtion, as you also say, “I have no reason to suspect that this is dramatically different in AR4”. TAR and AR4 are most likely similar in basic bibliometric data.
Are you guys going to be doing a post on the Science article on Arctic methane?
[Response: Yes, Patience! – gavin]
Re 548 debunking Einstein
I went to the web site that is supposed to debunk the theory of relativity. These are all statements and opinions. There is no science nor math to back them up, and there is no real logical sequence to the statements. It is just crap. He doesn’t even understand that the speed of light is not relative to any thing. It is constant measured against any reference point.
@Andreas: I’m calling your bluff. I did a quick check of the references to several chapters of AR4 WG1, and got nowhere near the 1 in 6 non-journal references that you claimed for TAR (I didn’t check that). 1 in 10 non-journal references would already be an overestimation.
Moreover, your table only distinguished journal vs non-journal. Earlier IPCC reports are also frequently referenced in the AR4. They are heavily peer reviewed, but you consider them “grey” just because they are not journals. Same goes for the many books that ARE peer reviewed. “Grey” literature in your evaluation. To make things worse, NO book or report published by a genuine publishing agency is actually considered “grey” literature, regardless of whether there is any peer review or not (!).
Andreas, I think you owe the people here at RC an apology, and I think you should ask Roger to significantly alter your claims on “grey” literature on his webside. Remember, you demanded RC to correct itself. Will you do so yourself?
Econdemocracy #562,
Try the brief section 10.7.4.2 for the long-term, global view, and an important note on the modelling requirements. But I don’t see any mention of Himalayas in ch. 10, either. Section 4.5.3 describes current developments as follows.
Be wary of anyone claiming to know how fast the Himalayan glaciers will melt over the next centuries. (Fools, rushing in where glacier experts fear to thread, may try extrapolating wildly from various current rates cited in the Kargel backgrounder.)
The Monbiot statement from 2004 that you cite sounds like it comes from the same tainted source as the unfortunate WG2 paragraph.
#571
I see nothing wrong with references to the “gray literature” as long as the cited papers pass muster with the reviewers.
Re: references in AR4,
I’ve long been curious about the mechanics of hundreds and authors writing and referencing ~3000 pages in a uniform way. Might the entire bibliography of AR4 reside in a monster BibTex file somewhere? If so, it should be an easy lunch break’s work to break it down into publication types and do a frequency count of journals.
576 Marco,
Please post the results from your count (on the chapters of WG1) at Pielkes blog. I am interested. Also explain your method.
If you want me to change anything, please be specific on what you claim is wrong. What you say is already things I have explained in my text, more or less.
First, the RC statement concerns specifically journal articles, so I am right about that. “The IPCC cites 18,000 references in the AR4; the vast majority of these are peer-reviewed scientific journal papers
Second, I explain my method and I don´t consider anything grey that is not grey. There is a difference between my technical term non journal articles and gray literature. I don´t give any exact numbers on gray literature, since I don’t have that.
Earlier IPCC reports don’t change much, since there can me a maximun of 6 references to earlier reports in each TAR chapter (3 parts of first and second assessment report). Perhaps you count in the text and not the reference list as I do?
579 CM,
CM, for TAR there was no such monster BibTex or database of references. I had to copy and paste from PFD-files from the IPCC homepage. When that is done, break it down in pulication type etc. That is a couple of month full time work. The authors of the different chapters of IPCC seems to work rather independenty with the references since the abbreviations of journal names differ between chapters. And the text is actually not that uniform, although they started with “uncertainty police” in TAR (read Schneiders new book).
575:Sir said: “I went to the web site that is supposed to debunk the theory of relativity.”
I’m sorry.
Andreas in what way is your instruction:
“Please post the results from your count (on the chapters of WG1) at Pielkes blog. I am interested. Also explain your method.”
Address the answer to your assertions from Marco?
You have the WG1 papers.
YOU stated something about them, Marco has responded with his research and if YOU know what you stated about them, then YOU should know where and what he’s missed.
Pielke’s not going to be able to explain it, unless you’re parroting him.
Andreas #581,
Don’t get me wrong, I didn’t mean to belittle your efforts; I rather imagined you had to analyze the TAR by hand. I was just speculating if there was a quicker route to doing similar work on AR4. For a collaborative work, updated at regular intervals, with 14,000+ references, reference-management software and a shared bibliographic database would make sense. But perhaps getting hundreds of people with ingrained writing habits to use it is more trouble than it’s worth.
… You didn’t happen to make a database yourself during those months, did you?
CFU, I don´t know what methods Marco used, that´s why I asked him. Before I know that, I can´t say much of the results. That´s how science works. I also explained why Marco is mistaken on some of his claims, e.g. that the RC quote concerns journal articles (hence, no excuse is needed) and that Marco is wrong regarding how I treat gray literature. CFU, please read MY text (at Pielkes blog) before you comment.
CM, yes, I understand, but I don´t think there is a quicker way for AR4. I think each working group work rather independently. I have excel-files and a bunch of different matrixes, but no database.
“586
Andreas Bjurström says:
7 March 2010 at 2:32 PM
CFU, I don´t know what methods Marco used, that´s why I asked him”
But you didn’t ask him. You asked him to tell someone else.
Neither was Marco hiding his methods. What you’re hiding are YOURS.
Reversing the position, Andreas. Marco asked YOU what YOUR method was, showed HIS workings and now YOU say you want HIS methods because he didn’t say???
Pathetic.
> Earlier IPCC reports are also frequently referenced in the AR4.
> … but you consider them “grey” …. Same goes for the many books
> that ARE peer reviewed. “Grey” literature in your evaluation. …
> NO book or report published by a genuine publishing agency is actually
> considered “grey” literature, regardless of whether there is any peer review
Andreas, do you agree with this description of your criteria?
CFU, please, you have not even read my original text (Gray Literature in the IPCC TAR, A guest post by Andreas Bjurström, at Pielkes blog). Marco understand what I am saying becayse he have been reading my text.
Hank Roberts, no, I don´t agree with that. I quantify journal articles. Everything else is “non journal references” but I don´t say that all of these are gray since they are not. That is why I am cautious to not give any numbers at all of gray literature. I only do some rough qualitative analysis and statements on that. My aim is to give a general outline, I don´t claim great precision and my general conclusions don´t need that.
“589
Andreas Bjurström says:
8 March 2010 at 3:33 AM
CFU, please, you have not even read my original text ”
And you haven’t read Marco’s.
He showed his investigation. They do not show what you claim is there. Either he is right in which case say so, he has missed something, in which case show how.
Neither case requires that Marco print his investigations on Pielke’s website unless he and you are working together.
CFU,
Well, a guest post means that we are working together, and where I posted my text is the appropriate place for in-depth discussion (this was the last time I replied to your ignorant questions on this matter).
“Well, a guest post means that we are working together,”
So you are working with Pielke?
And even if you’re working with him, you don’t need Marco to post there, you seem to be posting a lot just fine on here.
If anyone’s still reading this thread, I’ve been trying my hand at a simple Perl script to parse AR4 references and do a rough type count, in case it could shed some light on the discussion. After a day’s work, it still obviously doesn’t parse WG3 well and isn’t proven on WG2, so I don’t expect it to settle anything.
But summarizing 19,040 references from 44 chapters in 16 seconds is fun, and it did OK analyzing a WG1 chapter (ch. 4, snow and ice), which I also hand-counted for verification. The script correctly found 257 references and suggested 82-86% of these might be journal articles (against 84% in my count – including the non-peer-reviewed but tried and tested Stinemann (1980) in Creative Computing…).
Neither Andreas nor Gavin saw any reason why AR4 would be dramatically different from TAR with regard to journal references, though Gavin expected an increase. My very preliminary results agree. They suggest 90-92% journal articles in AR4 WG1, and the lower end of a 61-67% range for WG2 (cf. Andreas’s TAR figures of 84% and 59%, respectively). I’m not taking any bets on WG3 yet. If I manage to narrow down the errors, I’ll post what I find, though the anti-IPCC crowd will probably have hand-counted every chapter by then already (good, keep them busy).
With regard to “gray” literature, Marco’s and Gavin’s points are clearly the more important. As I said, 84% of the references in AR WG1 ch.4 (the snow and ice chapter) turn out to be journal articles, which is less than in the other AR4 chapters bar the first one.
I have classed the remaining 41 references as follows (hand count):
– 14 proceedings papers,
– 16 chapters in edited collections,
– 5 books, and
– 6 technical reports.
Generally these were published by serious-sounding scientific bodies or publishers, so I assume they have been accepted, edited, and in some cases refereed, by experts. The percentage of references that have passed scrutiny equivalent to peer review could well be in the high nineties. I agree with Marco: The journal article count alone does not tell the whole story.
That bears repeating, because Andreas started out (#571) referring to his “thorough analysis of gray literature” on Pielke’s blog, claiming “(t)here are thousands of references to gray literature in the IPCC reports”. He later backtracked, admitting he had no numbers on “gray” literature as such, only on journal and non-journal publications (#580), but still claimed to have scored a point against RealClimate on a technicality, tch tch.
594 CM,
I´m impressed by the computer coding and the rather good results it gives. The numbers are very interesting and similar to mine. If you like to, please e-mail me the results or just say a few words on how you did it, and we can discuss it some more (you can find my adress with google, or through the link by my blog post at Pielke). Actually, if you develop the computer coding a bit more, and are able to group references to different articles, it would be extremely valuable for me. If you are a researcher, that is a good opportunity for both of us, I would say, to write an article on that. Since I have 2 articles from TAR and lots of material and matrices and different categorications etc.
I agree that my wording (#571) was wrong. My analysis is a through analysis of journal references. Gray literature was the aim, to say something on that, but I slided from the one to the other in my wording.
However, this is an carefully worded estimation “There are thousands of references to gray literature in the IPCC reports.” I based that on the fact that I know that there are about 5.000 non-journal references in TAR, many of these are gray, and many are not. “Thousands” are inbetween 1 and 5 thousand … And I do still believe that RC mix WG1 with the rest when they claim that the “vast majority” are journal articles. I lead author that write on insurance issues or economic aspects may have the opposite view, from the localised context, that hardly any references are to journals, that is true of some WG3 chapters and perhaps a few WG2 chapters as well.
CM, very good posting, very good critique, sharp and fair and very useful!
Andreas,
The script opens each of the 44 reference lists in the IPCC web edition of AR4. It looks for various patterns in the text and formatting to take each reference, break it down into author, year, title, and publication data, and then looks for clues as to what kind of document it is: If the title is italicized, it’s a book; else, if it has a boldface volume number or a DOI, it’s an article; if it contains certain keywords, it’s a technical report or proceedings paper; else, if it has editors (“Ed(s).”) in the publication data part, it’s a chapter in a collection; else, if it has an italicized title in the publication data, it may be an article. The script counts them all up, and in the end reports a low and a high percentage estimate of journal articles. The low estimate is the number of fairly certain articles divided by the total reference count, the high estimate includes also the “maybe” articles from the last step, and is divided by only as many references as the script has managed to classify.
Fairly simple, linear logic so far, in code too messy to share, but it sort of works. It put the journal articles in WG2 ch. 16 at 56-62%; my count: 60%.
CM,
On this Andreas and I agree: Very cool!
Andreas, Ray,
Thanks. I need to catch up with work, but if Andreas wants to do some counting and check my results so far, here’s the output for WG1 (I’ve done ch. 4). If it doesn’t display properly, copy-paste into a text editor with monospace font.
| AR4 chap. | References | Journ. articles |
|-----------+-------+---------+-----------+-------|
| Chapter | Total | Unclear | (abs.) | (%) |
|-----------+-------+---------+-----------+-------|
| wg1-ch1 | 264 | 2 | 200-206 | 76-79 |
| wg1-ch2 | 759 | 30 | 695-695 | 92-95 |
| wg1-ch3 | 802 | 7 | 747-752 | 93-95 |
| wg1-ch4 | 257 | 6 | 210-215 | 82-86 |
| wg1-ch5 | 289 | 1 | 259-262 | 90-91 |
| wg1-ch6 | 609 | 34 | 546 | 90-95 |
| wg1-ch7 | 870 | 15 | 787-797 | 90-93 |
| wg1-ch8 | 687 | 2 | 613-616 | 89-90 |
| wg1-ch9 | 535 | 1 | 494-496 | 92-93 |
| wg1-ch10 | 546 | 2 | 508-513 | 93-94 |
| wg1-ch11 | 609 | 6 | 527-529 | 87-88 |
|-----------+-------+---------+-----------+-------|
| WG1 | 6227 | 106 | 5586-5627 | 90-92 |
|-----------+-------+---------+-----------+-------|
The “unclear” references (1.7%) either did not parse at all, or could not be classified.
The script doesn’t know if a journal is peer-reviewed, a popular magazine or an industry bulletin, so for present purposes, anything that looks like a journal article should be counted as one. I include only the journal-article count here, but I’m interested in getting the other types right too (I’ve used “book”, “in collection” (book chapter), “in proceedings”, “technical report”, but you may have better suggestions).
PS.
> copy-paste into a text editor with monospace font.
… and add spaces so the columns line up again.
> The script doesn’t know if a journal is peer-reviewed
…But we can tell it. That’s the next step, after we know it’s reliable.
Cool. Very good!
From your description, I´m sure that it is possible to write a script that suits my purposes (that is not primarily gray literature). If I will do the AR4 report in future, I will work with a computer programmer. And when that is done, combine my TAR matrices with AR4 to search for trends (in disciplinary bias and interdisciplinary structures).