by Michael Mann and Gavin Schmidt
On this site we emphasize conclusions that are supported by “peer-reviewed” climate research. That is, research that has been published by one or more scientists in a scholarly scientific journal after review by one or more experts in the scientists’ same field (‘peers’) for accuracy and validity. What is so important about “Peer Review”? As Chris Mooney has lucidly put it:
[Peer Review] is an undisputed cornerstone of modern science. Central to the competitive clash of ideas that moves knowledge forward, peer review enjoys so much renown in the scientific community that studies lacking its imprimatur meet with automatic skepticism. Academic reputations hinge on an ability to get work through peer review and into leading journals; university presses employ peer review to decide which books they’re willing to publish; and federal agencies like the National Institutes of Health use peer review to weigh the merits of applications for federal research grants.
Put simply, peer review is supposed to weed out poor science. However, it is not foolproof — a deeply flawed paper can end up being published under a number of different potential circumstances: (i) the work is submitted to a journal outside the relevant field (e.g. a paper on paleoclimate submitted to a social science journal) where the reviewers are likely to be chosen from a pool of individuals lacking the expertise to properly review the paper, (ii) too few or too unqualified a set of reviewers are chosen by the editor, (iii) the reviewers or editor (or both) have agendas, and overlook flaws that invalidate the paper’s conclusions, and (iv) the journal may process and publish so many papers that individual manuscripts occasionally do not get the editorial attention they deserve.
Thus, while un-peer-reviewed claims should not be given much credence, just because a particular paper has passed through peer review does not absolutely ensure that the conclusions are correct or scientifically valid. The “leaks” in the system outlined above unfortunately allow some less-than-ideal work to be published in peer-reviewed journals. This should therefore be a concern when the results of any one particular study are promoted over the conclusions of a larger body of past published work (especially if it is a new study that has not been fully absorbed or assessed by the community). Indeed, this is why scientific assessments such as the Arctic Climate Impact Assessment (ACIA), or the Intergovernmental Panel on Climate Change (IPCC) reports, and the independent reports by the National Academy of Sciences, are so important in giving a balanced overview of the state of knowledge in the scientific research community.
There have been several recent cases of putatively peer-reviewed studies in the scientific literature that produced unjustified or invalid conclusions. Curiously, many of these publications have been accompanied by heavy publicity campaigns, often declaring that this one paper completely refutes the scientific consensus. An excellent account of some of these examples is provided here by Dr. Stephen Schneider (Stanford University).
Perhaps the most publicized recent example was the publication of a study by astronomer Willie Soon of the Harvard University-affiliated Harvard-Smithsonian Center for Astrophysics and co-authors, claiming to demonstrate that 20th century global warmth was not unusual in comparison with conditions during Medieval times. Indeed, this study serves as a prime example of one of the “myths” that we have debunked elsewhere on this site. The study was summarily discredited in articles by teams of climate scientists (including several of the scientists here at RealClimate), in the American Geophysical Union (AGU) journal Eos and in Science. However, it took some time the rebuttals to work their way through the slow process of the scientific peer review. In the meantime the study was quickly seized upon by those seeking to sow doubt in the validity behind the scientific consensus concerning the evidence for human-induced climate change (see news articles in the New York Times, and Wall Street Journal). The publication of the study had wider reverberations throughout the academic and scientific institutions connected with it. The association of the study with the “Harvard” name caused some notable unease among members of the Harvard University community (see here and here) and the reputation of the journal publishing the study was seriously tarnished in the process. The editor at Climate Research that handled the Soon et al paper, Dr. Chris de Frietas, has a controversial record of past editorial practices (see this ‘sidebar’ to an article in Scientific American by science journalist David Appell). In an unprecedented (to our knowledge) act of protest, chief editor Hans von Storch and 3 additional editors subsequently resigned from Climate Research in response to the fundamental documented failures of the editorial process at the journal. A detailed account of these events are provided by Chris Mooney in the Skeptical Inquirer and The American Prospect, by David Appell in Scientific American, and in a news brief in Nature. The journal’s publisher himself (Otto Kline) eventually stated that “[the conclusions drawn] cannot be concluded convincingly from the evidence provided in the paper”.
Another journal which (quite oddly) also published the Soon et al study, “Energy and Environment”, is not actually a scientific journal at all but a social science journal. The editor, Sonja Boehmer-Christensen, in defending the publication of the Soon et al study, was quoted by science journalist Richard Monastersky in the Chronicle of Higher Education somewhat remarkably confessing “I’m following my political agenda — a bit, anyway. But isn’t that the right of the editor?”.
Shaviv and Veizer (2003) published a paper in the journal GSA Today, where the authors claimed to establish a correlation between cosmic ray flux (CRF) and temperature evolution over hundreds of millions of years, concluding that climate sensitivity to carbon dioxide was much smaller than currently accepted. The paper was accompanied by a press release entitled “Global Warming not a Man-made Phenomenon”, in which Shaviv was quoted as stating,“The operative significance of our research is that a significant reduction of the release of greenhouse gases will not significantly lower the global temperature, since only about a third of the warming over the past century should be attributed to man”. However, in the paper the authors actually stated that “our conclusion about the dominance of the CRF over climate variability is valid only on multimillion-year time scales”. Unsurprisingly, there was a public relations offensive using the seriously flawed conclusions expressed in the press release to once again try to cast doubt on the scientific consensus that humans are influencing climate. These claims were subsequently disputed in an article in Eos (Rahmstorf et al, 2004) by an international team of scientists and geologists (including some of us here at RealClimate), who suggested that Shaviv and Veizer’s analyses were based on unreliable and poorly replicated estimates, selective adjustments of the data (shifting the data, in one case by 40 million years) and drew untenable conclusions, particularly with regard to the influence of anthropogenic greenhouse gas concentrations on recent warming (see for example the exchange between the two sets of authors). However, by the time this came out the misleading conclusions had already been publicized widely.
Next, we discuss the first of three so-called “bombshell” papers that supposedly “knock the stuffing out of” the findings of the IPCC. Patrick Michaels and associates billed his own paper (McKitrick and Michaels, 2004) (co-authored by Ross McKitrick ), this way:
After four years of one of the most rigorous peer reviews ever, Canadian Ross McKitrick and another of us (Michaels) published a paper searching for “economic” signals in the temperature record. …The research showed that somewhere around one-half of the warming in the U.N. surface record was explained by economic factors, which can be changes in land use, quality of instrumentation, or upkeep of records.
It strikes us as odd, to say the least, that, after one of the “most rigorous peer reviews ever”, nobody involved (neither editor, nor reviewers, nor authors) seems to have caught the egregious basic error that the authors mistakenly used degrees rather than the required radians in calculating the cosine functions used to spatially weight their estimates**. This mistake rendered every calculation in the paper incorrect, and the conclusions invalid — to our knowledge, however, the paper has not yet been retracted. Remarkably, there were still other independent and equally fundamental errors in the paper that would have rendered it entirely invalid anyway. To the journals credit, they published a criticism of the paper by Benestad (2004) to this effect. It may come as no surprise that McKitrick and Michaels (2004) was published in Climate Research and was handled by none other than Chris de Frietas.
The other two “bombshell” papers were published in the AGU journal Geophysical Research Letters (GRL) which publishes over 1500 papers per year. It can be conservatively estimated that they publish no more than 70% of the papers received, and thus probably process over 2000 papers per year. That gives each of the typically 8 or so editors of the journal almost a paper per day to evaluate. While GRL publishes many excellent papers and provides an important forum to the research community for rapid publication of important results, occasionally, poor papers slip through the net. These two papers were authored by Douglass and collaborators (Douglass et al, 2004a;2004b) the first with Fred Singer as a co-author and the second with both Singer and Michaels. Both papers*** argue that recent atmospheric temperatures have been cooling, rather than warming, based on the analysis of data over a selective (1979-1996) time interval that eliminates periods of significant warming both before and after, and using a controversial satellite-derived temperature record whose robustness has been called into question by other teams analysing the data. An excellent discussion of both papers is provided by Tim Lambert.
Another relevant GRL paper was the article by Legates and Davis (1997) which criticized the use of “centered correlations” common to numerous “Detection and Attribution” studies supporting the detection of human influence on recent climate change. They argued that correlations could increase while observed and simulated global means diverge. However, as pointed out in the chapter on Detection and Attribution in IPCC (2001)*, centered correlations were introduced for precisely this reason: to provide an indicator that was statistically independent of global mean temperature changes. As noted by the IPCC, “if both global mean changes and centered pattern correlations point towards the same explanation of observed temperature changes, it provides more compelling evidence than either of these indicators in isolation”. Again, a basic logical flaw in the authors’ criticism of past work was not caught in peer review.
Next, we consider the paper by Soon et al (2004) published in GRL which criticized the way temperature data series had been smoothed in the IPCC report and elsewhere. True to form, contrarians immediately sold the results as ‘invalidating’ the conclusions of the IPCC, with the lead author Willie Soon himself writing an opinion piece to this effect. Once again, a few short months later, a followup article was published by one of us (Mann, 2004) that invalidated the Soon et al (2004) conclusions, demonstrating (with links to supporting Matlab source codes and data) how (a) the authors had, in an undisclosed manner, inappropriately compared trends calculated over differing time intervals and (b) had not used standard, objective statistical criteria to determine how data series should be treated near the beginning and end of the data. It is unfortunate that a followup paper even had to be published, as the flaws in the original study were so severe as to have rendered the study of essentially no scientific value.
There are other examples of studies that have even been published in high quality venues that were heavily publicized at the time, but in retrospect were flawed (though not as egregiously as the examples above). For instance, Fan et al (1998), on the size of the carbon sink in the continental US, rebutted by Schimel et al. (2000). Or the solar-cycle length/climate correlation described by Friis-Christensen and Lassen (1991) whose seeemingly impressive correlation for the latter half of the 20th Century disappears if you don’t change the averaging scheme half way along (Laut, 2003; Damon and Laut, 2004).
The current thinking of scientists on climate change is based on thousands of studies (Google Scholar gives 19,000 scientific articles for the full search phrase “global climate change”). Any new study will be one small grain of evidence that adds to this big pile, and it will shift the thinking of scientists slightly. Science proceeds like this in a slow, incremental way. It is extremely unlikely that any new study will immediately overthrow all the past knowledge. So even if the conclusions of the Shaviv and Veizer (2003) study discussed earlier, for instance, had been correct, this would be one small piece of evidence pitted against hundreds of others which contradict it. Scientists would find the apparent contradiction interesting and worthy of further investigation, and would devote further study to isolating the source of the contradiction. They would not suddenly throw out all previous results. Yet, one often gets the impression that scientific progress consists of a series of revolutions where scientists discard all their past thinking each time a new result gets published. This is often because only a small handful of high-profile studies in a given field are known by the wider public and media, and thus unrealistic weight is attached to those studies. New results are often over-emphasised (sometimes by the authors, sometimes by lobby groups) to make them sound important enough to have news value. Thus “bombshells” usually end up being duds.
However, as demonstrated above, even when it initially breaks down, the process of peer-review does usually work in the end. But sometimes it can take a while. Observers would thus be well advised to be extremely skeptical of any claims in the media or elsewhere of some new “bombshell” or “revolution” that has not yet been fully vetted by the scientific community.
_______________
*Note added 1/21/05: It has come to our attention that Legates and Davis (1997) were similarly rebutted in a separate publication by Wigley et al (2000).
**Note added 1/21/05: McKitrick and Michaels have published an errata correcting the degrees/radians error in CR 27, 265-268 which now shows that latitude correlates much better with temperature trends than any economic statisitic.
***Note added 1.25.05: Chip Knappenberger correctly points out that the the second Douglass et al paper doesn’t actually make the claim that the atmosphere is cooling. We therefore withdraw that specific comment, but note that the comment concerning the selective use of data series and time periods stands.
References:
Benestad, R.E., Are temperature trends affected by economic activity? Comment on McKitrick & Michaels., Climate Research, 27, 171-173, 2004.
Damon, P. E. and P. Laut, Pattern of Strange Errors Plagues Solar Activity and Terrestrial Climate Data, Eos, 85, p. 370. 2004
Douglass, D. H., Pearson, B.D., and S.F.Singer, Altitude dependence of atmospheric temperature trends: Climate models versus observation, Geophys. Res. Lett., 31, L13208, doi:10.1029/2004GL020103, 2004.
Douglass, D. H., Pearson, B.D., and S.F.Singer, Knappenberg, P.C., and P.J. Michaels, Disparity of tropospheric and surface temperature trends: New evidence, Geophys. Res. Lett., 31, L13207, doi:10.1029/2004GL020212, 2004, 2004.
Fan, S., Gloor, M., Mahlman, J., Pacala, S., Sarmiento, J., Takahashi, T., Tans, P. A Large Terrestrial Carbon Sink in North America Implied by Atmospheric and Oceanic Carbon Dioxide Data and Models, Science 282: 442-446, 1998.
Friis-Christensen, E., and K. Lassen, Length of the Solar Cycle: An indicator of Solar Activity Closely Associated with Climate, Science 254, 698-700, (1991).
Legates, D. R. and R. E. Davis, The continuing search for an anthropogenic climate change signal: limitations of correlation based approaches, Geophys. Res. Lett., 24, 2319-2322, 1997.
Laut, P., Solar activity and terrestrial climate: An analysis of some purported correlations, J.Atmos. Solar-Terr.Phys.,65, 801-812. 2003
Mann, M.E., On Smoothing Potentially Non-Stationary Climate Time Series, Geophys. Res. Lett., 31, 2319-2322, L07214, doi: 10.1029/2004GL019569, 2004.
McKitrick, R., and Michaels, P.J., A test of corrections for extraneous signals in gridded surface temperature data., Climate Research, 26, 159-173, 2004.
Rahmstorf, S., D. Archer, D.S. Ebel, O. Eugster, J. Jouzel, D. Maraun, G.A. Schmidt, J. Severinghaus, A.J. Weaver, and J. Zachos, Cosmic rays, carbon dioxide, and climate, Eos, 85, , 38,41, 2004.
Schimel, D., Melillo, J., Tian, H., McGuire, A. D., Kicklighter, D., Kittel, T., Rosenbloom, N., Running, S., Thornton, P., Ojima, D., Parton, W., Kelly, R., Sykes, M., Neilson, R. and Rizzo, B., Contribution of Increasing CO2 and Climate to Carbon Storage by Ecosystems in the United States, Science 287: 2004-2006, 2000
Shaviv, N, and J. Veizer, Celestial driver of Phanerozoic climate?, GSA Today, 13, , 4-10, 2004.
Soon, W., D. R. Legates, and S. L. Baliunas, Estimation and representation of long-term (>40 year) trends of Northern-Hemisphere gridded surface temperature: A note of caution, Geophys. Res. Lett., 31, , L03209, doi:10.1029/2003GL019141, 2004.
Soon, W., and S. Baliunas, Proxy climatic and environmental changes over the past 1000 years, Climate Research, 23, 89-110, 2003.
Soon, W., S. Baliunas, C, Idso, S. Idso and D.R. Legates, Reconstructing climatic and environmental changes of the past 1000 years, Energy and Environment, 14, 233-296, 2003.
Wigley, T.M.L, Santer, B.D and K.E. Taylor, K.E., Correlation approaches to detection, Geophys. Res. Lett.,, 27, 2973-2976, 2000.
Yes, this essay demonstrates the value of RealClimate. It has been realized on this entry. Well done.
Best,
D
Nice op-ed piece but lacks substance and is downgraded by use of references liked disinfopedia.
Could you elaborate on the ways in which you thought this long piece “lacks substance”, Bill? What would you have liked to see in the way of further substance, that you could not find elsewhere on this site or by clicking on some of the links provided?
Thanks for this discussion and for its important applicability to all the rest of science. You note “cases of putatively peer-reviewed studies in the scientific literature that produced unjustified or invalid conclusions” and you add that “[c]uriously, many of these publications have been accompanied by heavy publicity campaigns.” That calls to mind the physicist Robert L. Park’s Chronicle of Higher Education article “The Seven Warning Signs of Bogus Science” (31 January 2003). Park’s first warning sign: “The discoverer pitches the claim directly to the media.”
Thanks for another great posting. Another claim frequently made about peer review is that it keeps out opposing views, not just on climate change but also things like alternative medicine, creationism, all sorts of pseudo stuff. Perhaps some of you could address this claim.
There’s a broken link for
Rahmstorf, S., D. Archer, D.S. Ebel, O. Eugster, J. Jouzel, D. Maraun, G.A. Schmidt, J. Severinghaus, A.J. Weaver, and J. Zachos, Cosmic rays, carbon dioxide, and climate, Eos, 85, , 38,41, 2004.
I was interested in looking at that article.
[Response: Thanks for alerting us to this. We’ve fixed it. -mike]
The most interesting part is this paragraph:
No single paper will create a “paradigm shift” in climate science. Of course, grasping at straws is what skeptics do.
Another way to put all of this is that “peer review” that gets a paper published is only the first step. After publication a study is still getting “reviewed” in a manner of speaking. Flawed studies that get through the formal process will soon be forgotten, unless of course, they have a legion of quacks who want to use their flawed results to advance their religious, philosphical, political, or economic agendas.
—
Anti-spam: Replace “user” with “harlequin2”
You may be interested in my recent experience with the social science journal, Energy & Environment (E&E). In 2004 (Vol. 15, No. 3) E&E published a paper on sea level rise at Tuvalu by Willis Eschenbach, an amateur scientist and “Construction Manager” for the Taunovo Bay Resort in Fiji. The paper was entitled “Tuvalu not Experiencing Increased Sea Level Rise” which gives a general idea of the content. While most readers would assume that the paper had been peer-reviewed, on closer inspection it appears that the paper is what the Journal calls a “Viewpoint Piece”. The Editorial at the beginning of the Journal, also notes:
“A fascinating story by a local resident, engineer and private scholar, Eschenbach offers a convincing and well documented explanation of the problems facing many Pacific islands. As we could not find any reviewer for his paper, we hope that it will attract responses from those who still believe that the compensation demanded by Tuvalu (with the help of Greenpeace and environmental lawyers) for damage caused by “global warming”, is indeed unjustified.”
This of course begs a number of questions:
1. What qualifications did the Editor (Sonja Boehmer-Christiansen) have to claim that the paper was “a convincing and well documented explanation of the problems facing many Pacific islands”? In a later exchange, the Editor remarked to me that the paper “was reviewed by a few people he selected himself, ….. and also by me”.
2. Is the general reader expected to read the Editorial of the journal just to check which papers have been peer-reviewed?
3. Why on earth could the Editor not find a reviewer for this paper? (or could she just not find one sympathetic to her own views?)
4. Isn’t the Editorial clearly soliciting comments ONLY from those who have one specific political view of sea level rise at Tuvalu?
Boehmer-Christiansen further perverted the process by later stating in a paper at an international conference (see http://www.hwwa.de/Projects/Res_Programmes/RP/Klimapolitik/Papers%20Workshop/Boehmer-Christiansen.pdf):
“I just happen to be publishing an article by a scientist who lives on Tuvalu and who shows that the real problems already being experienced by people there (salination, sinking because of sand excavation) while ascribed by politicians seeking aid to global warming, are in fact due to over population, natural local causes and above (sic) development on what is little more than a floating patch of sand in the Pacific.”
which cites Eschenbach’s paper in E&E. So a paper, which has no more authority than a letter published in a local newspaper was now being cited at an international conference as “an article by a scientist”– with the natural implication that it had been peer-reviewed.
There are two pieces of good news to this story: (a) the Editor subsequently published a comment by myself on the original paper (E&E, 2004, Vol.15, No. 5; this was also not peer-reviewed, even though I requested it to be) and (b) thankfully, if you do a Google on (Eschenbach Tuvalu “sea level rise”) you only get three hits, so Eschenbach’s paper was virtually ignored.
“Another claim frequently made about peer review is that it keeps out opposing views […]”. Well, sometimes it does. When genuinely new ideas come along, they can be kept outside scientific consensus for a very long time–Wegener’s theory of continental drift took some 50 years to be accepted, for instance.
I cannot stress enough that science is a human and uncertain project.
That said, people who claim to have discovered such things are often wrong; most seminal discoveries are reluctantly accepted, even by their discoverers. Big loud claims about how “revolutionary” a work is are usually the sign of someone who doesn’t have a case.
[Response: Wegeners theory did take a long time to be accepted (though there are details in there that aren’t part of the common folklore; his mechanism was wrong, for example). But… did W have any trouble getting his papers published? This is the point at issue here, after all: does PR suppress ideas? In this case, it would seem not: after all, Wegeners theory was disputed (which implies that he presented it in papers/books/conferences?) for 50 years: it wasn’t ignored or unknown for that time – William]
At the risk of sounding like a fan-boi, this is a great piece of writing for people like me who have some knowledge of climate science, but don’t always fully understand how seriously to take the contrarians (so I really look forward to your take on Richard Lindzen).
I really hope that RealClimate is able to serve as an effective conduit to general media; it’d be nice if the facts, served straight, served some role in the popular debate on greenhouse gas emissions.
I understand that peer review is a necessary condition.
However, I have a question, which I explain and ask as follows.
I wrote a paper titled: Earlier in the Year Snowmelt Runoff and Increasing Dewpoints for Rivers in Minnesota, Wisconsin and North Dakota for presentation at the NOAA Climate Prediction Center and Desert Research Institute conference, Oct 20-22, 2003, Reno, NV.
The paper can be viewed at the Minnesotans for Sustainability Climate Change website, at: http://www.mnforsustain.org/climate_change.htm
… by clicking on [Snowmelt & Dewpoints in Minnesota, Wisconsin, and North Dakota] … at the bottom of the webpage.
I submitted my final draft for review by my supervisors and the scientists within the agency I’m employed by. I followed the time lines and agency procedures.
However, the final draft of my paper was not acted upon by supervisors and scientists, although I was told verbally by my supervisor, much later, that regional and national directors, and the chief administrator for climate change, that my paper by itself was not the problem.
A picture of me giving my presentation in October 0f 2003 is at: http://profiles.yahoo.com/patneuman2000
Ten months later, I had a chance to correspond via email with the administrator for the agency. After I explained to him what took place, and what did not take place (no action on my review of my paper), he wrote that [now that I have had a chance to review your paper, which appears to me as a noteworthy and scholarly addition to the body of scientific knowledge — however, remember that peer reviewers are really the ones to go to for critical comments — …]
In my case, I don’t think I could have gone for peer reviewers, because the regional office did not take any action on my paper. I have no idea who peer reviewers would have been. Does anyone have any idea who peer reviewers might have been for my paper? What went wrong? If time, please review my paper… url above.
Many climate scientists are probably finding attacks on their credibility by the greenhouse contrarians both tedious and time-wasting. Sure — scientific papers are not perfect and it is easy to find flaws in almost any paper. But science is a self-correcting process — it cannot go way off the track for very long because (a) other scientists will eventually realise something is amiss (and pointing out the mistakes of others does earn a certain kudos) and (b) science is a CONSISTENT SYSTEM, so one branch of science cannot go off on its own track independent of other (“adjacent”) branches of science — these other branches of science will soon notice that something is wrong.
(As an aside, it is tempting to speculate that other branches of human endeavour such as “social” science and economics (from where a number of the more vocal contrarians come) are not nearly so robustly constrained. It is probably not totally unfair to suggest that “wrong” theories can survive significantly longer in these disciplines than in the “true” sciences.)
The question remains: would science (or any other discipline) progress more satisfactorily and rapidly if significantly more time were spent on “auditing” other people’s work (e.g. if referees demanded to always see all the original data, and used it to repeat all the analyses of every paper)? I suspect not — two people doing identical work on the same problem would effectively halve the number of active scientists.
I also suspect that those contrarians involved in their own branches of research might protest a little less loudly if they realised that it is quite within the bounds of possibility that demands be made for all THEIR original data and analysis software to come under public scrutiny.
William, as far as I know Wegener presented most of his ideas and evidence in a book, rather than in peer-reviewed periodicals–at least the book is what’s cited in all the materials I can easily find on the web. He did, in other words, just what the crackpots do. A big difference between Wegener and the crackpots, however, is that Wegener did other major scientific work that was much valued and respected; this isn’t true of most of the crackpots.
[Response: Just because work is published in a book doesnt mean it wasn’t peer-reviewed. I have edited a number of books and the chapters were all peer-reviewed and appropriately revised before publication. That’s the responsibility of the editor(s). I have also written books in which (for my own reassurance) I ask colleagues to review chapters, and then revise them, if necessary, before they are published. I can tell you that I’ve had some pretty devastating reviews of chapters that have caused me to go “back to the drawing board” and start all over again, which was rather discouraging but in the end led to a much better product (I think). A good publisher will also ask for an independent scholar to review an entire book manuscript (for balance and omissions etc) before it is published. Of course, this does not always happen, so I can’t provide you with a hard and fast rule. But I believe that most scientists publishing books or book chapters do appreciate getting feedback on their work from reviewers. –Ray]
Having just had an argument about the value of Soon’s work, I must add my voice. The paper really was quite bad, even to someone not in the field (like me). For example as part of their methodology they state Table 1 and Figs. 1 to 3 summarize the answers to the questions posed here about local climatic anomalies. For Questions (1) and (2), we answered Yes?? if the proxy record showed a period longer than 50 yr of cooling, wetness or dryness during the Little Ice Age, and similarly for a period of 50 yr or longer for warming, wetness or dryness during the Medieval Warm Period?? And sure enough he did follow this methodology and you can find wet periods being reported as evidence for the LIA, MWP and even the transition time inbetween.
I did some homework on Wegener & continental drift; there’s a book on his theories and the responses to them: Drifting Continents and Shifting Theories by H.E. Le Grand (Cambridge University Press, 1988) & I spent a few minutes with it this afternoon at Powell’s. Wegener did publish a conference paper before he published his book, and his theories were debated for decades. They formed a speculative backdrop to geology for many years; my impression is that at least some papers were published. His ideas inspired such a vehement reaction, though, that publication would have been hit or miss; someone unlucky enough to get the wrong reviewer would probably not have been published.
There were a number of problems that kept a consensus from forming around Wegener’s ideas: he wrote a broadly synthetic work which did, indeed, contain many errors; he also used other people’s conclusions as part of his argument, which US geologists of the period regarded as invalid methodology. About 40 years after the first edition of The Origin of Continents and Oceans was published, the first evidence of rock magnetism and sea floor spreading emerged, and a new generation of geologists, who had grown up outside of the old debates, began to accept the theory of continental drift. It’s not, I think, a coincidence that it took two generations for the ideas to gain wide acceptance. Despite all claims of objectivity, scientists can be as stubborn as the rest of humanity, and the wide acceptance of truly radical theories often awaits the emergence of a new generation of scientists.
As for a parallel with the consensus on global climate change, I’d suggest James E. Hansen as the figure who first started the controversy and I note that Seitz, Michaels, and Singer are all over 60. Roughly, I’d guess the debates over global climate change took place largely between 1981 and 1995; a good bit shorter than the debates over continental drift, but then there was less radical about the idea of global climate change–it was already known that the planet’s climate had changed in the past, so the idea that it might be changing in the present was less radical than the idea that the vast continents might, in fact, be drifting like huge floating islands.
This post provides excellent insight into the realities (imperfections) of the checks and balances that the scientific peer review system intends to impose on papers which reach the public. It is the best system yet devised to assure credibility in the discourse among scientists.
Odd that in a post on this subject, some quite obviously *not* peer reviewed links are provided. I’m saddened to see links such as disinfopedia attached to the names of your adversaries, even though I’m sure I dislike their “concerted campaigns of misinformation” as much as you clearly do. No matter how deliberately flawed and/or one sided we know the tactics of our adversaries to be, I strongly feel that it does not serve well to take their bait and play that same game. The example of peer review provides a standard of credibility to which I sincerely hope this web site could try to adhere.
It may be unrealistic for me to expect this (and I certainly have no right to influence policy here), but I feel that any argument worth making about the REALity of Climate, as it is best described by those of us who on the front lines of understanding it, is to keep the debate to the standards of scientific peer review, rather than to concede to the tactics of a political campaign.
— Peter J. Wetzel, Ph.D. Atmospheric Science, 1978, Colorado State University. Specializing in the parameterization of land-atmosphere exchange for use in Global Climate, Regional Mesoscale, and Local Cloud-Resolving numerical weather prediction models.
Re: Comments by Peter J. Wetzel (#16) and others. You need to seperate science, which needs to be peer reviewed, from background information on the debate, which does not. It is hard to see where you could find a peer reviewed biography of Willie Soon, or any of the other participants in the debate.
Re: #16 and #17
I think it is also important to point out that much of the drive for acceptance of these questionable papers comes, not from the scientific community, but from interest groups and the media. That being said, it is awfully difficult to counter the specious claims being made by focusing attention solely within the confines of the peer-review. Look at it this way, there is an entire constituency within the public that firmly believes that there is a grand division among the scientific community on the validity of anthropogenic climate change. The reality is that there are a couple of handfuls of very vocal skeptics out there and the grand division doesn’t exist. If we insist on confining our efforts solely to the peer review this perception cannot be addressed.
What I am hoping is that the *tone* of discussion here, related to both the science and the background, be kept on a level which would pass muster if it was to be peer reviewed. I know I’ve personally failed to meet this standard myself a few times, but I’m trying.
The model I’m hoping would be followed here, is that commentary is written as if it were required to be “accepted” by a handful of arbitrary reviewers (including some who may be hostile to your thoughts) before it would be allowed to appear.
In fact the blog owners have that purview, and can effectively act as a self-governing review committee.
[Response: Indeed, you describe our current process. Any posting you see on “RealClimate” has been independently reviewed by several members of the group by the time you see it. – Mike]
Thanks for the analysis, you did a great job explaining how peer reviews don’t catch everything and that it is a long term process where any recent snapshot may not give an accurate picture of the current consensus.
However, I find it quite hard to believe that those who seek to dispute global warming are the only ones guilty of bad science. Given my experience in the academic world I find it utterly preposterous that only one side of the subject would get emotional or make mistakes in their papers. While one should be careful to make it clear that individual errors don’t debunk the theory I think it would aid the appearence of objectivity greatly to also mention some of these instances.
Now I don’t suggest falling into the trap that both sides need to be given equal time, but i do think it is quite reasonable to spend an amount of time proportional to the number of unsound papers on each side. Perhaps I have missed previous articles where you have done this and I don’t want to suggest not mentioning these makes for a poor article but since for most people the global warming question is an issue of trust this would probably help convince many people that you are objective.
[Response: Not all of the papers criticized by us in this piece express skepticism regarding anthropogenic impacts on climate (e.g. the Fan et al paper). However, it happens that many of the most egregious papers (particularly, those accompanied by especially aggressive publicity campaigns) do. We remind our readers that the two previous RealClimate postings on “Global Dimming” actually criticized instances in which we felt that likely anthropogenic impacts on climate have been overstated, not understated. That having been said, we caution our readers that the notion of “balance” that is often emphasized in the discussion of e.g. policy issues is misplaced when dealing with scientific matters. The scientific “truth” is not guaranteed to be found at the center of the extreme viewpoints found in the public discourse. Claims, for example, that 20th century climate change can be attributed to natural variability, and that anthropogenic impacts on climate are neglible, are not viewed as credible by the vast majority of scientists studying the earth’s climate system. As “extraordinary claims require extraordinary evidence”, it is appropriate to put papers making such claims under particular scrutiny, as we have done. – Mike]
Below comment #20 of this thread (Peer Review: A Necessary But Not Sufficient Condition), a Response by “Mike” begins with: “Not all of the papers” …
In comment #11 above, I wrote:
The paper can be viewed at the Minnesotans for Sustainability Climate Change website, at: http://www.mnforsustain.org/climate_change.htm
… In my case, I don’t think I could have gone for peer reviewers, because the regional office did not take any action on my paper. I have no idea who peer reviewers would have been. Does anyone have any idea who peer reviewers might have been for my paper? What went wrong? If time, please review my paper… url above.
Comment by Pat N: Self-only 21 Jan 2005
“Mike”,
I would like to know if you or others at realclimate.org intend to review my paper, as I requested above, on 21 Jan.
Pat N
[Response: I’m afraid that is not within the scope of our mission here at RealClimate – Mike]
I wrote (in #11):
In my case, I don’t think I could have gone for peer reviewers, because the regional office did not take any action on my paper. I have no idea who peer reviewers would have been. Does anyone have any idea who peer reviewers might have been for my paper? What went wrong? If time, please review my paper… url above.
I wrote (in #21):
I would like to know if you or others at realclimate.org intend to review my paper, as I requested above, on 21 Jan.
[Response: I’m afraid that is not within the scope of our mission here at RealClimate – Mike]
Above, I wrote:
Does anyone have any idea who peer reviewers might have been for my paper? What went wrong?
In “Peer Review: A Necessary But Not Sufficient Condition”, Michael Mann and Gavin Schmidt wrote:
“Thus, while un-peer-reviewed claims should not be given much credence,”
That means to me (according to what Michael Mann and Gavin Schmidt wrote), that my work on “Earlier snowmelt runoff and increasing dewpoints in the Upper Midwest” should not be given much credence.
[Response: this is the unfortunate implication, though it can be qualified. Clearly, if a given piece of good science gets through peer review with minimal changes (as many papers do) then the paper was just as good before review as after. If you are capable of understanding all the details in a paper and verifying them, there is no need to rely on peer review and (as the post says) peer review does not mean “this paper is right”. However, to people outside the field who cannot themselves verify the results, it is reasonable to use peer review as a filter. In your case, if it didn’t even get to PR, then PR becomes moot – William]
I worked on the study, on my person time over a severeal year period. I could not get peer review because climate change was considered too controversial or political by the agency that I work for. Does that seem responsible? … given the severity of climate change consequences.
Is peer review more important than helping in public awareness efforts on climate change?
My paper is at:
http://www.mnforsustain.org/climate_change.htm
Re #11, 21, 22: I’m a little confused about what happened. It seems that you wrote a paper and put it into your agency’s internal review process. Even though you didn’t say so, I guess that someone in the heirarchy wouldn’t let you submit it for formal publication with the agency name on it. Is that the gist of the story? If it didn’t go out for formal publication, it never got to the peer review stage.
In 22, you make it sound like you did the work on your own time. If so, there’s no reason to seek agency approval. Submit it someplace (I don’t think International Journal of Climatology has page charges) on your own without affiliation.
Comments you would get off of the internet version wouldn’t be peer review in any sense of the concept. They might be an informal review, but it wouldn’t be peer review. That’s not to say that they might not be valuable.
I know that as an author, reviewer, and editor, I’ve seen the process work really well and I’ve also seen it fail (both accepting and rejecting papers that probably shouldn’t have had that fate), but it’s a reasonably good process. The presence of the imprimatur doesn’t prove that the work is any more correct than its absence, but it’s not a bad first guess and the standard is one that’s relatively well-understood.
Another issue highlighted by this article: peer review for publication is just the first step in peer review. Post-publication review, critique, favorable or unfavorable citation, and so on, are really the heart of the scientific peer-review process, once review for publication gets a report into this process. This is in sharp contrast to the peer-review/publication/press-release policies of those with special interests to serve. It’s not that press releases are bad in themselves; in fact, they are essential to getting science findings to the public. But neither are they part of the scientific vetting process; and they are definitely employed by special interests either that are anti-scientific in themselves (e.g., Creationists) or that have a concern to obfuscate or falsify scientific evidence, as in the cases you describe.
I worked on the paper on my person time, and on the job in 2003. The agency I work for paid my registration fee several weeks before the conference date in Reno.
The findings in my paper show trends for earlier snowmelt runoff and increasing dewpoints in the Upper Midwest. I considered it essential that my findings reach the public.
I submitted my snowmelt runoff paper to management six weeks before the workshop. Local management signed off on the paper immediately, then sent it to regional management for evaluation. Region did not take action to approve or disapprove of the paper.
In order for the paper to be credible, I felt it was necessary to provide information on where I’ve worked. I worked for the agency from 1976 to current.
In the days just before the workshop, agency management would not act to allow me to identify my official position and agency name in the paper. I traveled to Reno on my own time and expense, no travel papers, and gave my presentation, not knowing if I could or couldn’t identify myself as an employee of the agency.
As a result of no action taken by the agency on my request to evauate my paper, I decided to issue a press release … to give public notice of my findings. It was my understanding that my press release would be my own. I felt it essential to include statements that greenhouse gas emissions are known to be causing the rapid global warming that’s being observed.
I included my place of employment and my position title in the press release for the purpose of providing description of my scientific background, and not to show that the agency I work for was in support of my efforts. I was not aware that important climate legislation was being considered by Congress on the date of my press release, which my supervisor later informed me about.
I considered adding a disclaimer to the paper and press release that the agency was not necessarily in support of work. However, I recalled an earlier official memorandum from the agency that said to me that although you have asserted that this effort was in your capacity as a private citizen, that assertion is inconsistent with the fact that you included your professional work location. Thus I concluded that a disclaimer in the paper or press release would merely be viewed by the agency as “inconsistent” if I also showed my work place and position.
As long as we are talking about peer review, let us also discuss dispassionate skepticism, the other virtue (if scientists can be said to have any as a group) of scientists. It is the decoupling of dispassionate from skepticism that makes public discussions about climate science and environmental issues in general so uninformative. One might say that skepticism alone is a nasty, but insufficient condition for policy.
I second an earlier commenter’s hope that you will soon devote a post to the skeptic Richard Lindzen. An MIT professor and member of the National Academy of Sciences, his views seemingly count for more than, say, Fred Singer and Patrick Michaels. So what is the real story here?
As a philosophy professor teaching a course in environmental ethics, I want to do justice to the issue of global warming–at least, as best I can given my layperson status when it comes to climatology. Blogs like yours are great for that purpose. As a philospher, too, I like all my colleagues take skepticism seriously, and within bounds respect it. So I’m not one for instant dismissals of the skeptics. Again, for this [urpose your blog is great. But what is the deal with Richard Lindzen? How skeptical is he and how good are his arguments?
#28 asks: “But what is the deal with Richard Lindzen?”
In Oct. 2002 I attended a public forum held at the University of Minnesota with speakers that included Dennis Hartmann, Chief Atmospheric Scientist, University of Washington, Seattle; and Professor Richard Lindzen, MIT.
Hartmann seemed knowledgable on climate change, particlularly water vapor feedbacks. Lindzen focused on cartoons and jokes.
In the question/answer period … I asked the panel to give a best estimate of the percent of warming due to CO2 emissions versus that due to changes in land use.
I had just read an article by Roger Pielke Sr., professor at Colorado State and president of the American Association of State Climatologists (AASC). The Pielke article said that land use changes had more influence on climate change than CO2 emissions, both on regional and global climate change.
Dennis Hartmann offered to respond to my question, saying that in comparison to the amound of global warming due to CO2 emissions, the amount of warming due to land use changes was very small.
Dennis Hartmann also gave a best estimate for the amount of global warming for the 21st century. He showned the range given by the IPCC (1.5 C to 4.5 C), saying that the middle of the range (3.0 C) was his best estimate at that time.
Well, I have to chime in. As a result of my employment and specialization, I find myself somewhat “in the eye of the storm” on the issues raised in #29.
First of all, Dr. Roger Pielke Sr. and I have known each other for more than 20 years. We both have worked for a quarter century or more in the area of land-atmosphere interactions. It was my dissertation topic. And one of Roger’s early students, Pat Gannon, wrote an early paper on the influence of land use on the initiation of Florida thunderstorms in the mid 1970’s.
I believe Dr. Hartmann has little personal experience in this area, and I find Dr. Hartmann’s remark on the subject (assuming it was accurately quoted) to be very telling. He seems to have been assuming that a Land-use effect would cause warming (or perhaps that was the way the question to him was worded).
The complex effect of land use change cannot be adequately considered in simple “warming or cooling” terms. The effect of land use change on the surface temperature record is to warm it. Yet at the same time, the effect on climate is to cool it. This apparent paradox can be explained fairly simply. The land surface temperature record relies on temperature data collected at thermometers within a meter or two of the surface. If forests are cleared, the temperature at this level is warmed because of a reduction in both the “roughness length” and the “zero plane displacement” of the surface. (In simple terms, a very flat surface must warm up much more before its heat can escape to the free atmosphere, compared to a very rough surface). However, at the same time, the surface albedo is increased by converting forests to croplands or grass, so more sunlight is reflected back to space. Also at the same time, the much higher daytime skin surface temperature (more than offsetting the somewhat colder night-time skin surface temperature which is often ameliorated by condensation and shallow fog layers) causes more infrared radiation to be emitted to space. So anthropogenic land use changes (which are strongly biased toward deforestation and desertification) tend to raise the temperature observed at thermometer shelters around the world, while at the same time they tend to reduce the amount of energy available to warm the atmosphere above the surface. Unfortunately these concepts have not yet made their way into the IPCC deliberations. They have simply not been given enough aggressive promotion, and not enough peer reviewed papers have been produced. To date, IPCC has chosen to ignore all Land Use effects beyond the simplest one, the Albedo effect. I’m hopeful that this egregious failure will soon be corrected.
Now, on to the issue of Lindzen’s Iris effect. My wonderful employer, NASA, has a set of web pages which do an excellent job of framing the debate on this topic. Here are the links:
http://earthobservatory.nasa.gov/Study/Iris/
http://earthobservatory.nasa.gov/Study/Iris/iris2.html
http://earthobservatory.nasa.gov/Study/Iris/iris3.html
Please read them all. The issue of who is right is pretty much entirely unresolved at this point, simply because the interaction between surface processes which force Cumulonimbus clouds and the cloud droplet microphysical processes which drive the internal processes within the clouds themselves are exceedingly complex.
I do not know Richard Lindzen. But his two co-authors, Ming-Dah Chou and Arthur Hou are/were in my laboratory. Arthur Hou works three floors below me in the same wing (the “C” wing) of the large Earth Sciences building (affectionately known as “building 33”) at NASA Goddard Space Flight Center. Ming-Dah and I have had many conversations on atmospheric radiation processes. His wife, Sue (Shu-Hsien) shared an office with me for most of the 1990’s. Ming-Dah and Sue retired from U.S. govt. civil service and moved back to their native Taiwan last year. I have the highest respect for Ming-Dah, whose work in parameterizing radiation processes for Global Climate Models is widely used in climate models.
Nevertheless, after having read the original paper by Lindzen, Chou and Hou, I found myself rather skeptical. First of all (and this is just a matter of process, really), the paper was published in the Bulletin of the American Meteorological Society (AMS) — which does not have the same rigorous peer review standards as the more main-stream AMS science journals have. Why did they not submit the paper to the Journal of Climate, where the most rigorous peer review would have been exercised?
But more to the point of their science. Their argument is that tropical Cumulonimbus (thunderstorm) clouds procuce less high-level cirrus-cloud outflow when sea surface temperatures (SST’s) are warmer and atmospheric water vapor is higher. This argument hinges on the contention that more water vapor means greater density of water droplets in the active rising updraft of the storm clouds. In turn, this argument requires that the greater density of droplets produces a greater likelihood of precipitation-sized droplets forming by the collision/coalescence process.
What this argument fails to consider is that the greater SST also produces a more vigorous updraft, so that the rising moist air has less time in which the collision/coalescence process can work before the air reaches the upper cloud layers where spontaneous ice nucleation takes place (at somewhere around -40C, reached near the top of the troposphere). When the droplets turn to ice, they are much more likely to “blow off” into the cirrus anvil of the storm rather than fall as precipitiation.
So there can be a significant trade-off between competing processes. The Lin et al. results are much more consistent with an expected balance, or trade-off, between these processes. We await further, still more sophisticated studies to resolve the issue.
I’m betting that Lin et al. are right, simply because of my long memory. I arrived on the meteorological scene as a fresh graduate student at Colorado State U. in time to become actively involved in the National Hail Research Experiment in the early 1970’s. This was an experiment which aimed to test the hypothesis that cloud seeding with silver iodide could suppress hail by creating an excess of nucleating embryos that would compete for the available cloud water (and thus keep all the hydrometeors smaller) — more precipitation, in fewer big “globs” of hail. This hypothesis was quite quickly rejected when results began to conflict (there was more and bigger hail, or at least no detectable hail suppression as a result of the seeding) and it became understood that the seeding also produced stronger updrafts (due to the accelerated release of the latent heat of freezing by the silver iodide seeding), which, in turn, produced an environment which was conducive to the formation of even larger hailstones.
The greater lesson: Cloud microphysics and cloud updraft dynamics interact with surface processes in very complex and unexpected ways which defy simple hypotheses.
Unfortunately, when someone takes a highly visible public position (as Lindzen did in 2001 with the publication of the Iris paper), the entirely unscientific issue of “preserving face” can sometimes override the otherwise pure intent of the scientist to retain an open mind. This is a sad human trait, and it is unbecoming of the objective scientist. I do not accuse Linzen of deliberately skewing his science. But I do suggest that he may no longer be an unbiased source of information on this subject.
The land use changes referred to by Peter Wetzel also remove CO2 sinks, eg. trees and sod. A forest sequesters CO2 on balance. Removing the trees, especially by burning, returns a bunch of carbon to the atmosphere. A farm field is essentially CO2 neutral on an annual basis. The same is obviously true of desertification.
You can see this pioneer effect in the growth of CO2 mixing ratios from a bit earlier than 1800 to maybe 1900 when fossil fuel burning takes over. It was certainly the major driver before 1850.
Hydrology is also an issue.
As far as the IRIS paper goes, I would simply recall the PR blitz that accompanied its publication. This was about as innocent an occurance as Pons and Fleishmann’s little circus act.