We should have done this ages ago, but better late than never!
We have set up a permanent page to host all of the model projection-observation comparisons that we have monitored over the years. This includes comparisons to early predictions for global mean surface temperature from the 1980’s as well as more complete projections from the CMIP3 and CMIP5. The aim is to maintain this annually, or more often if new datasets or versions become relevant.
We are also happy to get advice on stylistic choices or variations that might make the graphs easier to comprehend or be more accurate – feel free to suggest them in the comments below (since the page itself will be updated over time, it doesn’t have comments associated with it).
If there are additional comparisons you are aware of that you think would be useful to include, please point to the model and observational data set(s) and we’ll try and include that too. We should have the Arctic sea ice trends up shortly for instance.
You might want to consider sea level also.
[Response: Good idea. Though the SLR projections aren’t taken directly from the climate models. Stefan has got a graph that assesses the IPCC projections though… Thanks. – gavin]
Thanks, it’s great to have the information–and long overdue.
Thank you. This goes straight into tomorrow’s lecture. I has prepared slides from Skeptical Science but these are, in my opinion, more suitable for my application. In particular I like the discussion and the care with the references.
Thank you again.
Chuck
The new page is terrific, and I realise the intention is to let the graphs speak for themselves.
But for journalists and others who are not climate scientists, some narrative would help, as inline text and more clarification as footnotes if needed including, cover for example:
– being very clear for a graph what was being forecast (people play silly games with Hansen, confusing which was BAU)
– Perhaps showing original graph first “This is what was predicted …” in [clearly a] sidebar THEN annotated/ overlayed graph with “And this is how they did …” sidebar
– placing the prediction in context of the evolving data and science (e.g. we’d reached 3xx ppm and trajectory was; or “used improved ocean model”; or whatever)
– perhaps a nod to the successive IPCC reports and links to their narrative, so the historical evolution is clear, and also perhaps, how the confidence level has evolved.
Just some thoughts on what might be covered; but basically some historiography of climate model performance.
Any chance you could add RATPAC (or other radiosonde sets) to the atmospheric comparisons?
Where is it linked from the navigation bar? Seems like a page that ought to be easy to find. Thanks.
Great idea! I was getting tired of searching for old Twitter posts or Guardian articles for figures showing these comparisons. Not that the “but the models are all wrong” fake skeptics will be convinced, but this is useful for anyone actually interested in the science (and the facts).
A great idea to collect everything together on a/some handy reference page(s).
I’d love to know what metric(s) you’re intending to use for “observed Arctic sea ice trends”.
Extent, area, volume?
http://bartonlevenson.com/ModelsReliable.html
Just for kicks, how bout adding in Manabe and Weatherald’s early (late 60s early 70s) model projections?
Sea ice?
Excellent idea!
I see two virtually identical graphs, both labelled “CMIP TMT vs. Satellite Observations”?? I think one should be removed?
I agree that more labeling is needed. In particular, the Global TMT trends bar graphs can only be understood by people who already know what the elements in them, like the bars, represent.
An important factor that should be addressed in each plot is volcanic eruptions.
For example, Hansen et al. (1988) included volcanic eruptions in its future forcings for scenarios B and C, but not A. That is one reason why A warms so much more. And since the real world did not have those large eruptions (1963 Agung and 1982 El Chichón repeated), the agreement of observations with the forecasts is not as good as it appears.
For the CMIP5 simulations, there were no volcanic eruptions in the future forcing, yet some small eruptions had a small effect on the actual observations.
Please add those comments for each panel.
AR 13,
You’re forgetting Pinatubo in 1991.
“We are also happy to get advice on stylistic choices or variations that might make the graphs easier to comprehend or be more accurate – feel free to suggest them in the comments below ”
Yes!
For CMIP5 (circa 2011), I recommend that you draw the uncertainty of the *forcing-adjusted* mean+spread in gray.
This the important metric. If possible, we’d prefer to look at how observations compare to the forcing-adjusted model projections, versus the not-adjusted-for-forcing version. Right?
The eye is naturally drawn to the dark gray uncertainty range and how the colorful lines for the actual observations fit within that range. And that gray uncertainty range is for the not-forcing-adjusted models, if I understand correctly. So the eye is drawn to the worse metric, which gives the impression that the models are doing worse than they are.
TL;DR: make the forcing-adjusted uncertainty range the dark section, as that’s what the eye is drawn to. Always always draw the eye to the best metrics, remembering that most people are just going to glance, not read.
Bookmarked! An outstanding one-click resource. RC’s worth for the reality-based community grows daily. For as much or as little as they are worth, you have my personal thanks.
Hansen 81 / 88 — making the “Observed” line the same color in both graphs would make it easier to compare the two.
Hansen 88 / CMIP3 — It’s a little jarring to see the GISTEMP line terminating just short of 1.0 anomaly in one graph and just short of 0.7 anomaly in an adjacent graph.
CMIP 5 needs a hindcast / forecast line.
You should consider adding a chart collating skeptic predictions.
Yeah, great idea!
@10 the Manabe group (Princeton GFDL) papers of interest are the ones where the ocean-atmosphere coupling is developed and CO2 changes are applied, c. 1975 & c. 1991
Gory development details:
Manabe et al. 1975 Ocean-atmospheric coupled model description, the atmospheric circulation
https://www.gfdl.noaa.gov/bibliography/related_files/sm7502.pdf
Bryan et al. 1975 Ocean-atmosphere coupled model description, the oceanic circulation
https://www.gfdl.noaa.gov/bibliography/related_files/kb_sm_rcp7501.pdf
Application to CO2 changes:
Manabe et al 1991, Transient responses of a coupled ocean-atmosphere model (1) Annual mean response
https://www.gfdl.noaa.gov/bibliography/related_files/sm9101.pdf
(*cited by 1004)
Manabe et al. 1991, Transient responses of a coupled ocean-atmosphere model (2) Seasonal response
https://www.gfdl.noaa.gov/bibliography/related_files/sm9201.pdf
However their predictions are about much more than just the average near-surface air temperature, they are mainly focused on how heat mixes into the ocean and how that affects the rise in surface temperature as CO2 is doubled over 100 years. But if you want to understand how climate models were developed, tested and used to make predictions, those are the sources to read. Only 100 pages or so of very dense material and figures! Should keep the denialist crowd busy for a few years at least.
As a retired technical writer I’m reminded of the job I used to do: translating engineer-speak into technician-speak. In this case I reckon the greatest impact would be achieved if the science-speak were translated into layman-speak. As it stands, the information presented is excellent for people with basic knowledge of the subject, but it would puzzle some science-challenged friends I know.
Unfortunately such a “translation” would imply a bit of work. So the question is: who would do that? I don’t think it would be fair to lumber overworked climate scientists with the job. A technical writer is required — but I’m not volunteering for the job!
On the other hand, modifying a graph to aid understanding might bring the accusation that one has fudged the data. Still, to take one example, I know people who would not grasp the idea behind “temperature anomaly”. To fix this, one would have to mention somehow that the temperature is being compared to the average for some stated period.
This is not a simple matter. I suppose one should ask: who are the intended readers? For people like those who comment here the information is fine as it is. But if one wishes to reach doubters in the street, it might not work as well without some “translation”.
Terrific, thanks a lot.
Maybe use a smoothed temperature trend – say 5 year moving average. There are major short-term impacts – el nino, volcanos – that greatly affect the temperature line but have little to do with what is actually going on (that is, the overall trend) and are not predicted in the models.
The models do not give moving average forecasts, but the curves are so regular that that would add little.
RP @ 19. Thanks for those. Here are a couple more odds and ends on Manabe:
An article in Forbes (of all places..)https://www.forbes.com/sites/startswithabang/2017/03/15/the-first-climate-model-turns-50-and-predicted-global-warming-almost-perfectly/#19f7fd446614
And an interview:
https://www.carbonbrief.org/the-carbon-brief-interview-syukuro-manabe
They’re not exactly “observations” in the traditional sense, but will you be adding any models that attempt to model palaeoclimate reconstructions?
Actually, never mind, I totally misread the purpose of the page and posted too quickly. How about ocean temperatures? Sea ice? And maybe stratospheric temperature reconstructions?
It appears CMIP5 is missing data from 2016, but has Feb 2017 listed as the last update. If this is the case, can you please correct it.
Tokodave @10 says: “Just for kicks, how bout adding in Manabe and Weatherald’s early (late 60s early 70s) model projections?”
And for more kicks how about adding:
McCormick and Ludwig (1967): 67
Barrett (1971): 14
Rasool and Schneider (1971): 144
Hamilton and Seliga (1972): 12
Chýlek and Coakley (1974): 38
Bryson and Dittberner (1976): 31
Twomey (1977): 19
Cheers
Titus
> consider adding a chart collating skeptic predictions.
Yes. The forecasts of future energy use, for example
Some scientists say “the models run hot” or “the models predict more warming than has been observed.”
What does this group think? Do the models, in general and overall, predict more warming than has been observed, or not?
12 Spencer Weart –“I see two virtually identical graphs, both labelled “CMIP TMT vs. Satellite Observations”?? I think one should be removed?”
One is for global mean, the other is for tropical mean.
Titus:
That’s actually a good idea, although our unskilled-and-unaware-of-it friend may not care for the treatment his links receive from Real Climate scientists.
It might take a little work because the axis is calibrated in CO2 rather than years, but Callendar 1938 has a graph predicting global average temperature change. It shows about 0.6C in warming (compared to 1938, about 0.8C compared to pre-industrial) when CO2 hits 400ppm. This would be especially interesting since it is the first long term projection of AGW (as far as I know) and we are now half way to 2100 since the prediction was made.
Would be nice to add in Broecker 1975 as well. Seems like these guys did about as well with pencil and paper as later work with the computer models.
Rocketeer @32,
Callendar (1938) is a facinating document but, given much of what GS Callendar presents was based on numbers he had very poor estimates for, it doesn’t represent a quantative reality very well. As an instance of the problem he faced with his estimates, his value for 1938 CO2 in Table 1 is 296ppm which we now know was the level of fifty years earlier. (Figure 2 showing temperature v CO2 suggests a 1938 CO2 level at 305ppm which is much closer.)
The figure you refer to (Figure 2 again, so not the most accurate of graphics) does show a +0.7ºC rise to 400ppm. This I assume is an equilibrium rise so not applicable to today’s ΔT/CO2 levels. You will also note the projections of atmospheric CO2 in Table 1 (335ppm by 2000, 396ppm by 2100, equal to +1.3Gt(C) of atmospheric CO2/year.) and in Table VI his calculated average temperatures for coming centuries.
@MA Rodger #33. Under no illusions about the limitations of Callendar, but it seems like a lot of those errors cancel out leaving a graph that wouldn’t be too far off from the observations. Lucky perhaps, but I had a chemistry professor who told me that luck comes to those who work hard – Emil Fischer in that case. I think he would agree this is not a testable hypothesis ;-).
I think its really important to emphasize what is a hindcast and what is a forecast, as RE 4 pointed out. Here are a few suggestions for each one:
Hansen et al 1981: draw a vertical line where the observations stop and forecast begins, like in the subsequent figs.
Hansen et al 1988: Good hindcast/forecast vertical line. One suggestion is to change the line/symbol type of GISTEMP in the Forecast period. As it is the GISTEMP time series looks equivalent to a model scenario. Perhaps use the same format as in Hansen et al 1981 as that would make for consistency between the figs, or perhaps use a different symbol type during the forecast period (open black symbols instead of closed symbols?)
CMIP3 (circa 2004): Good, I like this one the best. Ideas to make it better could be to follow RE 4’s suggestion to have side-by-side panels with the left one showing the original fig where data stops at the hindcast/forecast line and the right one showing the full data set. If you made it a single image it would be much easier to share on social media. Another idea is to change the symbol type of the data during the forecast period to open symbols (or something like that) to really emphasize that the forecasting models had no idea what the resulting temperatures would be.
CMIP5 (circa 2011): I agree with BW 15 that the grey shading should follow the adjusted forcing. There also should be a vertical line indicating when the forcing adjusted hindcast ends and where the forecast begins for this figure.
Satellite-derived atmospheric temperatures: This may be confusing to folks because all of the other figures are for global surface temps, but these are mid-troposphere temps in the tropics and globally. I’m not sure there is sufficient general interest in mid troposphere temps for these figs to be on this page. The post was really in response to the poor/misleading figures from John Christy, and maybe its useful to have a link for people who are interested in comparing the satellite temps to obs since that is such a misinformation talking point, but I think it will confuse more people than not (especially non-scientists). Perhaps just have a link section to other blog entries that discuss model-data comparisons at the bottom so interested folks can find it, but don’t highlight it with figs on the main page?
You could also have links to other efforts to maintain model-data comparisons, such as this one: https://www.climate-lab-book.ac.uk/comparing-cmip5-observations/
Titus, back in #27, thought he was making some kind of point by listing papers from back in the 60s and 70s that expected, or least examined the possibility of, global *cooling*.
The reference is here:
http://journals.ametsoc.org/doi/pdf/10.1175/2008BAMS2370.1
The inscrutable numbers following the author list is the number of cites the paper received. It’s excerpted from a table that shows that, indeed, there were a whole lot more ‘warming papers’, even back then during the height of the ‘aerosol pollution era’, and media cooling hype.
Ho hum, but I was curious.
Arctic sea ice
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.393.9526&rep=rep1&type=pdf
http://neven1.typepad.com/.a/6a0133f03a1e37970b017744cf5360970d-pi
http://www.tandfonline.com/doi/full/10.1080/16742834.2015.1126153
http://www.tandfonline.com/na101/home/literatum/publisher/tandf/journals/content/taos20/2016/taos20.v009.i01/16742834.2015.1126153/20160201/images/large/taos_a_1126153_f0001_oc.jpeg
https://haveland.com/share/arctic-death-spiral.png
Also for Arctic sea ice – https://ntrs.nasa.gov/archive/nasa/casi.ntrs.nasa.gov/20140016849.pdf