RealClimate

Climate science from climate scientists...

You are here: Home / Climate Science / HadCRUT4 data now available

HadCRUT4 data now available

by

Just a quick note to point out that the HadCRUT4 data are now fully available for download. Feel free to discuss (or point to) any analyses you’d like to see done in the comments, and perhaps we’ll update this post with the more interesting ones.

Reader Interactions

90 Responses to "HadCRUT4 data now available"

  1. Elsewhere, someone posted a link to this:

    http://icecap.us/index.php/go/political-climate/no_global_warming_for_15_years/

    It includes the comment, “No statistically significant trend can be discerned from the [HadCRUT4] data.”

    I’ve seen other comments that point out the variability of the data points, in terms of uncertainties, and also the recent study about removing temporary negative and positive influences on temperature, showing a definite continuing warming trend.

    Is there any chance of a comprehensive rebuttal of the above blog entry?

    [Response:It’s probably pointless to point out the obvious in a new post in response to this specific example of the same silly claims by people that ought to (and often do) know better. However, you could point people to various already existing rebuttals, such as this one: http://tamino.wordpress.com/2011/11/05/the-real-problem-with-the-global-warming-debate/ –eric]

  3. @Tom

    This is a favourite of the fake sceptics. It is what I call cherry picking your starting point. They’ve started in 1997 not because they are looking for a 15 year window but because 1998 was a very hot year due to a massive el nino effect.

  4. One question:

    What do the columns, except col 1 which is obviously year, represent in ‘hadcrut4_annual_ns_avg”? There are 11 of them so can’t be month, I need to know as I want to do a series of graphs showing how choosing your stating point aaffects the regression line slope (dramatically) if you only concentrate on the last two decades.

  6. The burning issue everyone will surely be wanting to hear is ‘how much has HadCRUT4 adjusted recent anomalies from HadCRUT3?’
    The answer is that up to 2010 (where HadCRUT4 presently ends) it gives an increase of some 0.04 deg C.

    This link should lead to a graph of HadCRUT3 & 4 10-year rolling averages.

  7. Additional to my previous comment, the HadCRUT4 anomaly in the 1970s has also increased (by a whopping 0.009 deg C) so 2010 is up by 0.031 deg C relative to the 1970s.

  8. The 15 year trend (1996-2010 inclusive) is close to being significant different from zero at the 2σ level using the Foster and Rahmstorf ARMA(1,1) error model:
    Trend: 0.136 ±0.145 °C/decade (2σ)

    You can check it for yourself here:
    http://www.skepticalscience.com/trend.php

    (For comparison with other datasets, be sure to set the end data to 2011 (or 2010.99) because HadCRUT4 only runs to 2010 at the moment)

    [Wow, the recaptchas are hard now]

  9. Tony,
    I would be suspect of anyone who chooses particular starting (or ending) points to show exceptionally low (or high) warming trends. While the statement is true, it should be viewed in light of the longer data trend. Temperatures have been rising since the 19th century, with periods of greater and lesser warming (even cooling). I have not analyzed the new CRU4 data yet, but have done considerable work with CRU3. While the claim by some that the trend is negative over the past decade is true, the claim by others of the high trend of 0.18C/decade for the previous 30 years is also true. But as Paula mentions above, these are cherry picking. Over the long haul (130+ years), the temperature has risen at an average of 0.6C/century, with the previous high and low periods, preceded by similar high and low periods.

  10. > I have not analyzed the new CRU4 data yet,
    > but have done considerable work with CRU3.

    Well, that’s a new claim. What work does “Dan H.” claim to have done in this field? “Dan H.” claims expertise in the science.

    This seems an extraordinary claim. Can it be supported?

    I’m not asking our hosts to break his anonymity, but to vouch for his claim that he’s done work in climatology. If we’re to trust people posting under pseudonyms, claiming expertise, someone’s got to vouch for them.

  11. Choosing subsets of the data will always be a tactic of folks trying to support their pre-conceptions. My favorite recent cherry-pick was the commenter who said:

    If the right points are picked both before and after the huge El Nino of 1998, the slope line can be perfectly flat either way. That is because La Ninas around 1998 balance things out.

    (From a recent Spencer thread)

    Oddly enough, they were unable to recognize the problems with their approach…

  15. Ah, true to form, the last period in Dan’s period is only 10 years–of course starting in a bit El Nino. Nice to know we can count of Dan H. whenever we want a cherry to be picked.

  17. Dan H:

    Anybody can use wood for trees to make linear plots which don’t show any statistical relevance, I imagine that Hank had hopes that “working with” the data meant something more, like, for instance, unlike everyone else you’ve managed to show that a 10-year cherry pick shows flat temps with statistical significance, just as a 30-year plot shows a rising trend with statistical significance, therefore vastly changing our knowledge of what statistical significance means in global temperature time series.

    Sadly, though, it appears that you’ve fallen short of my guess at Hank’s expectations.

    Not that this is anything new, you’ve played this game before.

  18. With regard to 15 year trend, it’s a test of whether or not global warming has stopped. The drawback is that it’s a completely unreliable test. And it’s very easy to show that.

    Pick a period over which there’s agreement that warming occurred. So, the globe warmed between 1950 and 1999, say, and we all agree on that. Take (in my case) the GISS series, apply the 15 year trend test successively to the years 1950 … 1985. How reliably does the test show that warming is occurring?

    Answer: It’s substantially less reliable than flipping a fair coin. For 14 of 36 cases, the trend is positive with P < .05. (With no adjustment for autocorrelation, because that's how the dummies usually run it.)

    First observation: The test is modestly less reliable than flipping a coin if it is reported fairly, year after year. But it's only wide reported when it shows that global warming "has stopped". Once you've accounted for the "publication bias", it's totally unreliable as observed. I would call it a "stopped clock", but that does disservice to the clock, as even a (US) stopped clock is right twice a day.

    Second observation: For as long as you live, in slightly more than half the years, you'll see somebody telling you that global warming has stopped, based on that test.

  21. Kevin,
    Maybe he was referring to the old saying, “the tire is only flat on one side, roll it over, and it should work fine.”

  22. Science and math can be such a pain. As long as you don’t need to be concerned about the science or math there are an almost infinite number of ways to prove that global warming/climate change doesn’t exist.

  23. 10 Hank,

    Strangely enough, a “Dan H.” made several comments in the Tamino thread mentioned above, and didn’t impress anyone with his analysis.

  24. Some people in the Netherlands also were complaining that there was no significant warming over here over the last 15 years. When investigating the homogenised The Bilt series (KNMI), this is correct: no significant warming over 1997-2011. But two years ago there was significant warming over 1995-2009. Looking at the last decades, of all possible 15-year periods around half had significant warming and the other half didn’t.
    This indicates that warming isn’t a monotone process; no surprise at all. Aside, the last time there was significant cooling over 15 years was 1911-1925.

    But why looking at just 15 years? The WMO defines ‘normals’ as ‘Period averages computed for over a uniform and relatively long period comprising at least three consecutive 10-year periods.’ So why not looking at 30-year periods when the data is available?
    The Bilt (hom.) had significant warming over the years 1982-2011. The last time there was no significant warming over 30 years was 23 years ago (1959-1988).

  27. If one could harness the energy spent by fake skeptics bolstering their prejudices, it would provide a great deal of “clean” energy, though the pollution of time wasted doing so might not be offset.

    At the risk of saying what has already had to be said far too often, this:
    http://www.skepticalscience.com/graphics.php?g=47
    The up the down staircase graph gives it to you in one, if you will just pry open your mind a teensy teensy bit.

    Meanwhile, back at the ranch, could you guys return to doing what you do best – science?

  28. I meant to add in the energy spent in trying to open the door of knowledge by those who have better things to do as well to what is turning into a perpetual motion machine – though perpetuity is ill served by this nonsense.

  29. Oh, look! Isn’t it cute. Dan H. is teaching Isotopious how to cherrypick!

  30. KR @ #11 (quoting a comment at Spencer’s site):

    If the right points are picked both before and after the huge El Nino of 1998, the slope line can be perfectly flat either way. That is because La Ninas around 1998 balance things out.

    (From a recent Spencer thread)

    Oddly enough, they were unable to recognize the problems with their approach…

    Doesn’t this fall into the “it’s not a bug, it’s a feature!” class?

    I remember reading a Master’s student’s thesis proposal which said that he would try a variety of statistical techniques until he found one that showed the results were significant. I was more worried that his supervisor had not seen anything wrong with the proposal…

  31. After following the topic from street-level for about two decades, the first funny part about the ‘stopping’ argument is that temps keeps ‘stopping’. Every decade there’s been a claim that “it” is over, and every decade new global peaks are reached. It’s like the broken turn-signal signal joke.

    The second funny part is the focus on cherry-picking a start point, when the real card-trick is picking the end-point. The volume level of the stoppers is directly proportional to the inverse of current ENSO conditions. Add a couple of rough, cold anomalies in Europe when the Jet Stream short-circuits and, voila – the perfect low end-point to hook up to 1995, 1998, and December 2001.

    On their playing field:

    24 Months of Baked Graph
    http://www.woodfortrees.org/plot/wti/from:2008.2/to:2010.2
    fyi – that’s 2.5dC PER DECADE.

  32. dhogaza,
    I see you, for one, picked up my assertion that anyone can cherry-pick the data to post what they want to show. Some people have even gone so far as to change the slope of the past 10 (or 15) years to match the previous 30. Looking at the entire dataset, shows the folly in these type of displays. I really do not concern myself with Hank’s expectations.

  35. 1 Tony asks, “Is there any chance of a comprehensive rebuttal of the above blog entry?”

    Sure. The point the post makes (but also tries to hide) is that a 15 year period without warming is not a refutation of theory, but it REQUIRES AN EXPLANATION. It then drops the ball by not looking for an explanation. Well, “not looking for” is probably not right. Far far FAR more likely is that they knew about Foster and Rahmstorf 2011 (which completely answers your question) and chose to try to fool you by not mentioning it. So, google F&R, or just look it up here on RC. After that, come back and tell us what you think of the integrity of your source.

    Yes, this is about integrity, not scientific disagreement. Otherwise they’d have said, “F&R2011 is wrong because _____”.

  36. #1, et seq, et seq, et seq–

    Lord, but I’m tired of this particular–claim? meme? folly?

    So easy to refute.

    So tedious after the first couple of thousand times.

    So–an attempted epigram:

    “Folly is the determined, repeated asking of the wrong question.”

    (Or ‘not even wrong’ question.)

  37. #28 Susan Anderson

    “I meant to add in the energy spent in trying to open the door of knowledge by those who have better things to do as well to what is turning into a perpetual motion machine – though perpetuity is ill served by this nonsense”.

    I think this sentiment is worth more than a passing reflection and certainly deserves to be made known to a wider audience. Well said!

  38. Somebody should tell McIntyre, who has been silent so far. Expect him to write a post saying that “the team” is still hiding something.

  39. Folks, Dan H. in #9 was (by my reading) pointing out the folly of cherry-picking, not endorsing it. And I would agree with him on that point.

    It’s always possible to choose a non-significant period providing you a trend line of your preference, it’s just that doing so is a misrepresentation of the data.

    If anyone here hasn’t seen the Escalator graph, I would suggest it as a reference.

  40. Tony Weddle @1
    Further to comment from Jim Larsen @35, the piece you highlight is simply republished ‘work’ by “Dr David Whitehouse, science editor of the GWPF.”
    Myself, I didn’t realise the GWPF had such an editor. There is no announcement of it on the GWPF website. And anyway, the GWPF claim not to hold any position on science (but then claims made by the GWPF are very seldom borne out by reality).
    I wonder if Whitehouse is using shorthand to describe his position at the GWPF. He has up until now been “…a former BBC Science Editor and a scientific adviser to the Global Warming Policy Foundation.”

    Given it is supposed to be an educational charity, my own quite extensive researches (most recently published here) demonstrate the GWPF to be a singularly untrustworthy organisation.
    The conclusions inferred by Whitehouse from his findings are pretty impossible to support unless you interpret”…great uncertainties in our understanding… to refer to ‘GWPF understanding‘. Note also how Whitehouse cannot wait for a 2011 figure for HadCRUT4 and so makes up his own figure. However well argued this act, it has to ring alarm bells – “Idiot on the lose!”

  41. KR wrote: “Dan H. in #9 was (by my reading) pointing out the folly of cherry-picking, not endorsing it.”

    Yes, Don H. frequently chides us about cherry picking — usually, just before he presents a link to a particularly egregious, deceptively cherry-picked graph.

  42. Simon Abingdon, I’m pretty sure I’m not flattered. My sentence was rather gummy and the point could easily be reversed, which I assume was why you admired it. If you’ve had a change of heart, I’d like to know. For some reason, you find my amateur efforts easy to target, and provide lots of honey in the trap.

    Let’s start with the original to provide some context:
    “If one could harness the energy spent by fake skeptics bolstering their prejudices, it would provide a great deal of “clean” energy, though the pollution of time wasted doing so might not be offset.”

    I then suggested that I had failed to mention real scientists were being forced to waste their energy repeating obvious truths to try to get through the muck rather than continuing their excellent and necessary work finding out more about the truth embodied in our planetary circumstances.
    —
    Brian Dodge @~33 – Speaking of amateurism I had fun changing the end year to later than 2005 and changing the database (from HADCrut3) which provided some even stronger illustrations. Thanks. Don’t even need to know much science to see the obvious there.

    Kevin McKinney @~36 – exaactly: so far, 4 cites of the elevator graph. Once more into the breach, dear friends, trying to get the truth out. There are none so blind as those who *will* not see.

  43. KR,
    Thank you for being so observant. Sometimes, I think the reactionary response is too quick, and the full meaning is missed.

  44. > pointing out the folly of cherry-picking

    If only. Dan H. muddies the discussion by confusing two different issues.

    One: sufficient data to have good probability of detecting a trend — not making claims using short time spans.

    Two: assessing changes in forcings associated with changes in climate — using data sufficient to do the statistics.

    If you “use all of the data” you can’t detect any change in trend from forcings known to make a difference (e.g. sulfate aerosols, which peaked in the 1940-1970 range from US sources and again later from Chinese).

    Using “all the data” precludes detecting change during the time span.
    Cherrypicking short term data precludes confidently detecting change.

    “… “Dan H.” stated that what mattered was the long-term trend, which was a steady increase at a rate between about 0.006 and 0.0075 deg.C/yr ….” at Tamino’s and particularly November 8, 2011 at 12:24 pm

    None of us want climate to change as fast as it’s happening; rather than grasping at straws, we inspect them — really closely — lest we be fooled by false hope and fake assurance from business-as-usual PR guys.

  46. KR:

    Folks, Dan H. in #9 was (by my reading) pointing out the folly of cherry-picking, not endorsing it. And I would agree with him on that point.

    No, he’s claiming that an interval that shows a statistically significant trend is “cherry picking” in the same sense that chosing an interval that does not show a statistically significant trend is.

    In other words, statistical significance (and by extension, statistical trend analysis) is worthless.

    He then moves on to say that the only trend that’s significant is the last 130 years, which he chooses because it conveniently leaves out most of the additional forcing due to mankind’s dumping of CO2 into the atmosphere, and various other reasons put forth by Hank above.

    Go read the Tamino thread devoted to Dan H’s obsfucation. I don’t see how anyone can read that and still believe that Dan H is just innocently warning against cherry-picking. Tamino’s a professional statistician specializing in time-series analysis, BTW …

  47. Dan H:

    Thank you for being so observant. Sometimes, I think the reactionary response is too quick, and the full meaning is missed.

    Never fear, we understand your obsfucation perfectly.

  48. Thank you Paul Clark for your site.

    The original post invited us readers to ask for analyses.

    Paul offers both HADCRUT and CRUTEM v3 and v4 for comparison and invites people to compare them on his home page.

    Question for the statistically competent — can we say the trends are most likely different?

  49. From woodfortrees.org:
    ___________excerpt follows__________

    HADCRUT4 too!
    18th April 2012:
    WFT now includes the new HADCRUT4
    dataset from Met Office Hadley.
    Here’s a quick comparison of HADCRUT4 with HADCRUT3.

    CRUTEM4

    Now including CRUTEM4 data
    12th April 2012:
    Rather late to the party, but WFT now includes the new CRUTEM4
    dataset from Met Office Hadley and East Anglia CRU.
    Here’s a quick comparison of CRUTEM4 and CRUTEM3. CRUTEM4 seems to have a markedly higher recent trend (2.8°C/century) than CRUTEM3 (2.2°C/century).
    Draw your own conclusions!
    ______________________end excerpt________

  50. Paul Clark,
    Thank ye! Thank ye! Thank ye!

    You provide an invaluable service.