We’ve often discussed the how’s and why’s of correcting incorrect information that is occasionally found in the peer-reviewed literature. There are multiple recent instances of heavily-promoted papers that contained fundamental flaws that were addressed both on blogs and in submitted comments or follow-up papers (e.g. McLean et al, Douglass et al., Schwartz). Each of those wasted a huge amount of everyone’s time, though there is usually some (small) payoff in terms of a clearer statement of the problems and lessons for subsequent work. However, in each of those cases, the papers were already “in press” by the time other people were aware of the problems.
What is the situation though when problems (of whatever seriousness) are pointed out at an earlier stage? For instance, when a paper has been accepted in principle but a final version has not been sent in and well before the proofs have been sent out? At that point it would seem to be incumbent on the authors to ensure that any errors are fixed before they have a chance to confuse or mislead a wider readership. Often in earlier times corrections and adjustments would have been made using the ‘Note added in proof’, but this is less used these days since it is so easy to fix electronic versions.
My attention was drawn in August to a draft version of a paper by Phil Klotzbach and colleagues that discussed the differences between global temperature products. This paper also attracted a lot of comment at the time, and some conclusions were (to be generous) rather unclearly communicated (I’m not going to discuss this in this post, but feel free to bring it up in the comments). One bit that interested me was that the authors hypothesised that the apparent lack of an amplification of the MSU-LT satellite-derived trends over the surface record trends over land might be a signal of some undiagnosed problem in the surface temperature record. That is not an unreasonable hypothesis (though it is not an obvious one), but when I saw why they anticipated that there should be an amplification, I was a little troubled. The key passage was as follows:
The global amplification ratio of 19 climate models listed in CCSP SAP 1.1 indicates a ratio of 1.25 for the models’ composite mean trends …. This was also demonstrated for land-only model output (R. McKitrick, personal communication) in which a 24-year record (1979-2002) of GISS-E results indicated an amplification factor of 1.25 averaged over the five runs. Thus, we choose a value of 1.2 as the amplification factor based on these model results.
which leads pretty directly to their final conclusion:
We conclude that the fact that trends in thermometer-estimated surface warming over land areas have been larger than trends in the lower troposphere estimated from satellites and radiosondes is most parsimoniously explained by the first possible explanation offered by Santer et al. [2005]. Specifically, the characteristics of the divergence across the datasets are strongly suggestive that it is an artifact resulting from the data quality of the surface, satellite and/or radiosonde observations.
(my emphasis).
For reference, the amplification is related to the sensitivity of the moist adiabat to increasing surface temperatures (air parcels saturated in water vapour move up because of convection where the water vapour condenses and releases heat in a predictable way). The data analysis in this paper mainly concerned the trends over land, thus a key assumption for this study appears to rest solely on a personal communication from an economics professor purporting to be the results from the GISS coupled climate model. (For people who don’t know, the GISS model is the one I help develop). This is doubly odd – first that this assumption is not properly cited (how is anyone supposed to be able to check?), and secondly, the personal communication is from someone completely unconnected with the model in question. Indeed, even McKitrick emailed me to say that he thought that the referencing was inappropriate and that the authors had apologized and agreed to correct it.
So where did this analysis come from? The data actually came from a specific set of model output that I had placed online as part of the supplemental data to Schmidt (2009) which was, in part, a critique on some earlier work by McKitrick and Michaels (2007). This dataset included trends in the model-derived synthetic MSU-LT diagnostics and surface temperatures over one specific time period and for a small subset of model grid-boxes that coincided with grid-boxes in the CRUTEM data product. However, this is decidedly not a ‘land-only’ analysis (since many met stations are on islands or areas that are in the middle of the ocean in the model), nor is it commensurate with the diagnostic used in the Klotzbach et al paper (which was based on the relationships over time of the land-only averages in both products, properly weighted for area etc.).
It was easy for me to do the correct calculations using the same source data that I used in putting together the Schmidt (2009) SI. First I calculated the land-only, ocean-only and global mean temperatures and MSU-LT values for 5 ensemble members, then I looked at the trends in each of these timeseries and calculated the ratios. Interestingly, there was a significant difference between the three ratios. In the global mean, the ratio was 1.25 as expected and completely in line with the results from other models. Over the oceans where most tropical moist convection occurs, the amplification in the model is greater – about a factor of 1.4. However, over land, where there is not very much moist convection, which is not dominated by the tropics and where one expects surface trends to be greater than for the oceans, there was no amplification at all!
The land-only ‘amplification’ factor was actually close to 0.95 (+/-0.07, 95% uncertainty in an individual simulation arising from fitting a linear trend), implying that you should be expecting that land surface temperatures to rise (slightly) faster than the satellite values. Obviously, this is very different to what Klotzbach et al initially assumed, and leaves one of the hypotheses of the Klotzbach paper somewhat devoid of empirical support. If it had been incorporated into their Figures 1 and 2 (where they use the 1.2 number to plot the ‘expected’ result) it would (at minimum) have left a somewhat different impression.
For reference, if you plot the equivalent quantities in the model that were in their figures, you’d get this:
(for 5 different simulations). Note that the ‘expected amplification’ line is not actually what you would expect in any one realisation, nor the real world. The differences on a year to year basis are quite large. Obviously, I don’t know what this would be like in different models, but absent that information, an expectation that land-only trend ratios should go like the global ratios can’t be supported.
Since I thought this was very likely an inadvertent mistake, I let Phil Klotzbach know about this immediately (in mid-August) and he and his co-authors quickly redid their analysis (within a week) and claimed that it was not a big deal (though their reply also made some statements that I thought unwarranted). Additionally, I provided them with the raw output from the model so that they could check my calculations. I therefore anticipated that the paper would be corrected at the proof stage since I didn’t expect the authors to want to put something incorrect into the literature. After a few clarifying emails, I heard nothing more.
So when the paper finally came out this week, I anticipated that some edits would have been made. At minimum I expected a replacement for the inappropriate McKitrick reference, a proper citation for the model output, acknowledgment that the amplification assumption might not be valid and an adjustment to the figures. I didn’t expect that the authors would have needed to change very much in terms of the discussion and so it shouldn’t have been too tricky. Note that the last paragraph in the paper does directly link the non-amplification over land to possible artifacts in the data products and so some rewriting would have been necessary.
To my great surprise, no changes had been made to the above-mentioned section, the figures or the conclusion at all. None. Not even the referencing correction they promised McKitrick.
This is very strange. Why put things in the literature that you know are wrong? The weird thing is that this is not a matter of interpretation or opinion about which reasonable people could disagree, but a straightforward analysis of a model that gives only one answer. If they thought McKitrick’s source data were appropriate, why wouldn’t they want the correct answer?
It’s almost always possible to make some edits in the proofs, and papers can always be delayed if there are more substantive changes required. Indeed, they were able to rewrite a section dealing with a misreading of the Lin et al paper that had been pointed out in September by Urs Neu (oddly, there is no acknowledgment of this contribution in the paper). Is it because they want to write a new paper? That’s fine, but why leave the paper with the old mistakes up without any comment about the problems (and two of the co-authors have already blogged about this paper without mentioning any of this or any of the other criticisms)? If these issues are trivial, then it would have been easy to fix and so why not do it? However, if they are substantive, the paper should have been delayed and not put in the literature un-edited.
I have to say I find this all very puzzling.
Note added in proof: I sent a draft of this blog post to Dr. Klotzbach and he assures me that the non-correction was just an oversight and that they will be submitting a corrigendum. He and his co-authors are of the opinion that the differences made by using the correct amplification factors are minor.
I think (FWIW) the “science or coal” debate is pretty sterile. It wasn’t just one thing that created the Industrial Revolution; it was the synergistic action of many things.
And engineering and science tend to blend into one another at the frontiers; no experimentalist is without engineering smarts, and engineers take their theory seriously–albeit from a different perspective. (John Tyndall–surveyor and physicist–would have made a fine engineer, had he chosen that profession. Witness his famous demonstrations and apparatus.)
I think it’s a case of “when it’s time to railroad, you railroad.”
Hank (146), fair answers…
realclimate.org is still only intermittently working for me; it works to come in ‘sideways’ via a Google search or other link. I left a query at “webfaction.com” reporting that some of us are seeing their “Site Not Configured” message off and on, trusting that they’re the right people to tell.
CM, we cross-posted I think, so no worries. And you’re right – science hasn’t changed, but the public is ceasing to believe it. I think there’s a few psychological factors at work – for example, many don’t want to believe it so will cling to any hope from anywhere, and under an extremely fierce contrarian campaign they have optimistically bought into “doubt”. Another big factor is that the last 2-3 years many of us have had milder weather, and on a purely intuitive basis, it takes the reality off people’s radar (which of course has little-to-no scientific relevance).
Which leads on to Ray’s comments. What DO we do? Would love a full blog post on this! I do think that the basic science case has got buried recently and now needs something both new and very grounded in facts. The whole scientific issue has become a) politicised (“they just want another excuse to raise taxes” and b) emotive (“Tony Blair still flies, they’re all hypocrites, I’m not giving up my 4×4”). People seem to be incapable of separating science from dumb conspiracy and general cynicism about the media and politicians, and I think that’s our starting point.
And hence my paper database idea, really. Hardly the final solution, but a useful contribution if done right. I remember when watching An Inconvenient Truth, the contrast between the Science journal macro-study and the study into media coverage was one of the most persuasive elements. Let’s try and get back to this – as concrete analysis as reasonably possible of where science really is. Crucially – make it as broad, watertight and as objective as science can. Sadly in this particular battle I think we have to forget the IPCC, Copenhagen etc – we need something powerful which has nothing to do with governments and pressure groups.
Gee, one or two more recent peer-reviewed metadata studies would be an excellent start…
Hank (& anyone else with that problem): I fixed this (in Firefox) by removing real climate from my cache and cookies. Now works fine after a couple of days of the same message you’re getting.
thanks ffrancis, that should’ve been obvious to try and seems to have worked.
#145: Chris, and Gavin (re: #130)
For a given quantity of water vapor in a tropospheric column, different distributions of the water vapor will result in different magnitudes of the ‘greenhouse’ effect from the radiative absorption of IR by the water vapor.
From IPCC AR4 Chapter 8, Box 8.1: Upper-Tropospheric Humidity and Water Vapour Feedback :
“Tropospheric water vapour concentration diminishes rapidly with height, since it is ultimately limited by saturation-specific humidity, which strongly decreases as temperature decreases. Nevertheless, these relatively low upper-tropospheric concentrations contribute disproportionately to the ‘natural’ greenhouse effect, both because temperature contrast with the surface increases with height, and because lower down the atmosphere is nearly opaque at wavelengths of strong water vapour absorption.”
[..]
“The radiative effect of absorption by water vapour is roughly proportional to the logarithm of its concentration, so it is the fractional change in water vapour concentration, not the absolute change, that governs its strength as a feedback mechanism.”
The 3 papers under discussion found that the models do well at reproducing the total column integrated water vapor profile but have a systematic tendency to overestimate the moisture levels in the upper troposphere, and to underestimate them in the lower troposphere.
Since the total amount is the nearly the same in models and observations, the presense of any extra water vapor in the upper troposhere (compared to observations) must correspond to a decrease of that quantity of water vapor in the lower troposphere.
Since water vapor is present at much greater concentrations in the lower troposphere, a decrease of water vapor in the lower troposphere will diminish the greenhouse radiative effect in the lower troposphere by a much smaller amount than the greenhouse radiative effect in the upper troposphere will be increased for an increase of the same quantity of water vapor.
This seems to show that models are overestimating the greenhouse warming from water vapor?
Response: You might think so, but it is more complicated. A better way of looking at it is to think about the effective emitting height of the IR from the water. Since all the models have the same temperature controlled decrease in humidity with height (more or less), the consequence of having too much upper trop water is that the mean emitting height will be slightly higher than in the real world. However, the feedback depends on the change of the emitting height in a warmer situation, and so the model’s sensitivity might not be that sensitive to the exact location (and so it proves). More generally, the sensitivity of any change in the models is only loosely connected to biases in the mean climatology. – gavin]
Chris, by the way, thank you for your article “Re-visiting climate forcing/feedback concepts”, which you directed me to – I found it well-written and very helpful.
Kevin McK. (151), you’re correct, of course, but talking away all the fun…
Guy, the problem with your proposal is that you are assuming that evidence can convince people that climate change is an issue. However, we already have evidence–mountains of it. It should be obvious to anyone over the age of 30 that they are not living in the same climate into which they were born.
This problem goes much deeper than evidence. It goes to getting people to believe something they really don’t want to believe. If they buy into science, then this is possible. If they reject science then we’re just waiting around until we become extinct.
Gavin said,
“[Response: We have never claimed to my knowledge that no papers purporting to be skeptical of AGW have ever been published.”
Comment by Ray Ladbury — 17 November 2009 @ 2:42 PM
“The problem is that the denialists have no science, no evidence. All they have is a nagging suspicion that all the experts must be wrong because their conclusions don’t allow unlimited consumption in perpetuity.”
This regular drumbeat by the Rays that skeptics have nothing is blatantly wrong and inconsistent with the
acknowledgments by fellow team members.
Obviously the skeptics have science, evidence and peer reviewed publications.
You may debate it, disparage it and reject it but claiming it doesn’t exist is not wise.
[Response: You appear to be having a bit of a logical thinking fail. “Paper in peer-reviewed literature” does not imply ‘science and evidence’. There are many papers that are wrong, that misrepresent evidence and distrort science. G&T, Khilyuk and Chilingar are prime examples. What evidence or science is that? – gavin]
#157:
Gavin: “[…] the consequence of having too much upper trop water is that the mean emitting height will be slightly higher than in the real world.”
Exactly – the models have a higher effective emitting height, which means a lower emitting temperature and less escaping energy through OLR ie. a more powerful ‘greenhouse’ effect than the the real world.
[Response: No. The emitting temperature is always the same, and the net GHE in the models can’t be very different from the real world since it is constrained by the solar input (which equals the IR output at TOA at steady state) and the surface temperature (which sets the surface LW up). The differences you get are a Watt/m2 or two (out of ~150 W/m2), which is a small error. – gavin]
Ray, we recently had an exchange involving Newton.
If we can set aside for the moment his role as (co-)inventor of the calculus and his seminal work in optics, I should like share with you my understanding of his contribution to mechanics.
3rd law: Action and reaction are equal and opposite. Hardly insightful. I walk on the floor: the floor supports me. Commonplace.
1st law: (Law of inertia). Everyone knows it and experiences it. Try turning quickly when you’re running: you can’t (without difficulty). Commonplace.
But the 2nd law: P = mf (giving away my age). Just an amazing synthesis of the apple and the planet. Maybe science’s most wonderful insight, modestly attributed by Newton (the shoulders of giants) mainly to Kepler. (Poor old neglected Hooke).
And the Principia? Just the embodiment of the Enlightenment. Altogether dumbfounding.
Ray, when Newton has been your lifelong hero, it is galling to be asked if you’ve ever heard of him.
Sorry if this is OT.
#162:
Gavin,
The emitting temperature isn’t constant because the absorbed solar input varies with changes in albedo and solar irradiance at the TOA.
I came across this concise description of the ‘greenhouse’ effect, which I hope I can use to demonstrate the point I’m trying to make concerning a model-overestimated GHE :
“In the terrestrial spectrum, there are large numbers of absorption lines (which are broadened by both Doppler and pressure effects), owing mostly to the triatomic molecules water vapor and carbon dioxide, but with some notable contributions from other trace species such as methane. Water vapor content decreases sharply with height, but there are still large water absorption effects at 11 km. Taken together, the relative opacity of the atmosphere to terrestrial radiation makes for a strong “greenhouse effect”. ”
From the MIT Program in Atmospheres, Oceans and Climate.
Increasing the WV in the upper trop wil increase the opacity there. Decreasing the WV in the lower trop will decrease the opacity, but not as much because of the differences in existing concentration. The lower trop’s high WV content can sometimes make it completely opaque to IR radiation in WV’s absorption ranges, so lowering the WV wouldn’t change the GHE in those situations.
It would seem to follow that the model atmospheres, with distributions which has more WV in the upper troposphere and less in the lower than the real word does, will be more opaque to terrestial IR radiation and so have a stronger greenhouse effect, following from the definition above.
So, Terran, where did you get the wrong idea Gavin just corrected?
You stated it as though you thought it were a fact. Your own work?
Someone else told you? Where’d you get it?
Net results: millions dead in WWI due to Newtonian mechanics making it possible to create accurate artillery tables.
Not significant, of course. Everyone dies, eventually, anyway …
Something lighthearted, with a serious message, that all should check out in the lead up to COP15.
http://www.youtube.com/watch?v=KBzR0-j0O0o
some levity… now carry on!
John H., I would suggest that you might look into how often the denialist diatribes have been cited since their publication.
http://www.eecg.utoronto.ca/~prall/climate/climate_authors_table.html
Most often, you will find zero or at must a citation in a refutation. The denialists simply do not present a coherent framework for understanding climate. The consensus model does.
So should we reject the only model that allows us to understand the climate simply because it tells us things we’d rather not here? That’s the question facing the human race–whether policy will be guided by science or anti-science.
Simon Abingdon, Sooooo…. let me get this straight. Newton is your idol, but you just never bothered to consider the implications of his work? Or you just don’t understand it. Because it’s really, really hard to imagine the industrial revolution occurring without Newton’s influence.
As I say, the coal was always there, and it had been known since ancient times. Kind of hard to argue that energy was the sole driving force behind the industrial revolution. Human understanding is a greater driver than fire.
“3rd law: Action and reaction are equal and opposite. Hardly insightful. I walk on the floor: the floor supports me. Commonplace.
“1st law: (Law of inertia). Everyone knows it and experiences it. Try turning quickly when you’re running: you can’t (without difficulty). Commonplace.”
Simon, FWIW, I think you’re underestimating the amount of insight required to look clearly at the mundane. (Perhaps you’ve read about those (20th-century) critics of Goddard who thought that rockets couldn’t work in hard vacuum because they “need something to push against?” They clearly had flunked 3rd law, despite their “commonplace” experience of its consequences.) And formalizing it certainly was consequential, as has already been pointed out.
Guy (~154, 19 November 2009 @ 3:30 AM):
Use “communicating science” in the RC site search to find several topic threads on this subject. This might be helpful for refining your ideas.
Steve
#169 Ray. Ray, of course you’re quite right. If the Chinese had had a Newton they would have got farther than gunpowder.
I got suckered in to arguing with you that the exploitation of fossil fuel resources was what drove the IR. But yes, there could have been many Abraham Darbys discovering how to make coke and iron, but only one Newton. He really was extraordinary, and I agree that most of the aspects of the western lifestyle we now enjoy are thanks to him.
Ray, I’m 66 (before retiring I logged over 10,000 hrs in Boeings and the Airbus BTW) but I still have a lot of resistance to the idea that AGW could be true. (My protected childhood perhaps). However, I do think the current Hadcrut thing is very tacky. Wouldn’t want to identify with that at all. RC is for me always the authentic blog and source of climate info.
Simon
Gavin,
From the paper “Mechanisms for climate variability during glacial and interglacial periods” (Loving & Vallis 2004) :
“Reducing greenhouse gases reduces the effective emitting height of the atmosphere and cools the climate, and in our model increasing the planetary emissivity (e) is a parameterization for this.”
Changing the emitting height by changing the opacity of the atmosphere to terrestrial radiation is more or less the definition of the greenhouse effect. With greater opacity, the emitting height of the downward LW radiation decreases, warming the surface, and the emitting height of the spaceward LW increases, meaning less energy lost to space, keeping the planet warmer.
This is my understanding of it, at any rate.
#165: Hank, it’s my argument, though probably not an novel one.
Simon, that was very gracious, and I appreciate it. FWIW, I apologize for the snark. It probably came across as more vitriolic than it was intended.
As to anthropogenic causation, I’ve been over the science in as much detail as I can, given that I am only a physicist and not an expert. I really don’t see any way to avoid the conclusion that we are seriously altering the climate. The consensus model has too much explanatory and predictive power to be seriously wrong–and the same forcings and feedbacks that drive anthropogenic warming are indispensible in that model. The theory and all the evidence say we’ve got a serious poblem.
We agree on the East Anglia hacking. The hackers, and those using the private communications to smear science and scientists are lower than snakesh*t. It is just another reminder of caliber of the anti-science camp.