This is a continuation of the last thread which is getting a little unwieldy. The emails cover a 13 year period in which many things happened, and very few people are up to speed on some of the long-buried issues. So to save some time, I’ve pulled a few bits out of the comment thread that shed some light on some of the context which is missing in some of the discussion of various emails.
- Trenberth: You need to read his recent paper on quantifying the current changes in the Earth’s energy budget to realise why he is concerned about our inability currently to track small year-to-year variations in the radiative fluxes.
- Wigley: The concern with sea surface temperatures in the 1940s stems from the paper by Thompson et al (2007) which identified a spurious discontinuity in ocean temperatures. The impact of this has not yet been fully corrected for in the HadSST data set, but people still want to assess what impact it might have on any work that used the original data.
- Climate Research and peer-review: You should read about the issues from the editors (Claire Goodess, Hans von Storch) who resigned because of a breakdown of the peer review process at that journal, that came to light with the particularly egregious (and well-publicised) paper by Soon and Baliunas (2003). The publisher’s assessment is here.
Update: Pulling out some of the common points being raised in the comments.
- HARRY_read_me.txt. This is a 4 year-long work log of Ian (Harry) Harris who was working to upgrade the documentation, metadata and databases associated with the legacy CRU TS 2.1 product, which is not the same as the HadCRUT data (see Mitchell and Jones, 2003 for details). The CSU TS 3.0 is available now (via ClimateExplorer for instance), and so presumably the database problems got fixed. Anyone who has ever worked on constructing a database from dozens of individual, sometimes contradictory and inconsistently formatted datasets will share his evident frustration with how tedious that can be.
- “Redefine the peer-reviewed literature!” . Nobody actually gets to do that, and both papers discussed in that comment – McKitrick and Michaels (2004) and Kalnay and Cai (2003) were both cited and discussed in Chapter
2 of3 the IPCC AR4 report. As an aside, neither has stood the test of time. - “Declines” in the MXD record. This decline was
hiddenwritten up in Nature in 1998 where the authors suggested not using the post 1960 data. Their actual programs (in IDL script), unsurprisingly warn against using post 1960 data. Added: Note that the ‘hide the decline’ comment was made in 1999 – 10 years ago, and has no connection whatsoever to more recent instrumental records. - CRU data accessibility. From the date of the first FOI request to CRU (in 2007), it has been made abundantly clear that the main impediment to releasing the whole CRU archive is the small % of it that was given to CRU on the understanding it wouldn’t be passed on to third parties. Those restrictions are in place because of the originating organisations (the various National Met. Services) around the world and are not CRU’s to break. As of Nov 13, the response to the umpteenth FOI request for the same data met with exactly the same response. This is an unfortunate situation, and pressure should be brought to bear on the National Met Services to release CRU from that obligation. It is not however the fault of CRU. The vast majority of the data in the HadCRU records is publicly available from GHCN (v2.mean.Z).
- Suggestions that FOI-related material be deleted … are ill-advised even if not carried out. What is and is not responsive and deliverable to an FOI request is however a subject that it is very appropriate to discuss.
- Fudge factors (update) IDL code in the some of the attached files calculates and applies an artificial ‘fudge factor’ to the MXD proxies to artificially eliminate the ‘divergence pattern’. This was done for a set of experiments reported in this submitted 2004 draft by Osborn and colleagues but which was never published. Section 4.3 explains the rationale very clearly which was to test the sensitivity of the calibration of the MXD proxies should the divergence end up being anthropogenic. It has nothing to do with any temperature record, has not been used in any published reconstruction and is not the source of any hockey stick blade anywhere.
Further update: This comment from Halldór Björnsson of the Icelandic Met. Service goes right to the heart of the accessibility issue:
Re: CRU data accessibility.
National Meteorological Services (NMSs) have different rules on data exchange. The World Meteorological Organization (WMO) organizes the exchange of “basic data”, i.e. data that are needed for weather forecasts. For details on these see WMO resolution number 40 (see http://bit.ly/8jOjX1).
This document acknowledges that WMO member states can place restrictions on the dissemination of data to third parties “for reasons such as national laws or costs of production”. These restrictions are only supposed to apply to commercial use, the research and education community is supposed to have free access to all the data.
Now, for researchers this sounds open and fine. In practice it hasn’t proved to be so.
Most NMSs also can distribute all sorts of data that are classified as “additional data and products”. Restrictions can be placed on these. These special data and products (which can range from regular weather data from a specific station to maps of rain intensity based on satellite and radar data). Many nations do place restrictions on such data (see link for additional data on above WMO-40 webpage for details).
The reasons for restricting access is often commercial, NMSs are often required by law to have substantial income from commercial sources, in other cases it can be for national security reasons, but in many cases (in my experience) the reasons simply seem to be “because we can”.
What has this got to do with CRU? The data that CRU needs for their data base comes from entities that restrict access to much of their data. And even better, since the UK has submitted an exception for additional data, some nations that otherwise would provide data without question will not provide data to the UK. I know this from experience, since my nation (Iceland) did send in such conditions and for years I had problem getting certain data from the US.
The ideal, that all data should be free and open is unfortunately not adhered to by a large portion of the meteorological community. Probably only a small portion of the CRU data is “locked” but the end effect is that all their data becomes closed. It is not their fault, and I am sure that they dislike them as much as any other researcher who has tried to get access to all data from stations in region X in country Y.
These restrictions end up by wasting resources and hurting everyone. The research community (CRU included) and the public are the victims. If you don’t like it, write to you NMSs and urge them to open all their data.
I can update (further) this if there is demand. Please let me know in the comments, which, as always, should be substantive, non-insulting and on topic.
Comments continue here.
EL said ” All bugs in software can be completely and totally eliminated by using mathematical rigor.”
I’m a computer programmer with formal university training, an engineering degree, mathematical background, and 20 years programming experience in C and C++, and it is completely untrue that “all bugs in software can be … eliminated by using mathematical rigor”.
The majority of bugs in software are outside of the code. They are things like a misunderstanding of requirements, poor mappings from requirements into design, inaccurate validations on input, inaccurate specifications etc. None of these problems can be solved with mathematical rigor.
The problems I’ve just described exist in the interface between humans and code. Proving the code to be correct is only a minor part of the battle.
Gavin:
Thanks for your reply on the RC crack attempt the last time. I have another question (if you have the time): was the FOI2009.zip file that the cracker tried to upload the same as the zip file that’s now floating around at megaupload and elsewhere?
Incidentally, I found that most of the files in the zip archive (which I downloaded from megaupload) seem to be created under a timezone of -0500 or -0400, and a uid and gid of 1,002. Hope this helps. :-)
— bi
[Response: Yes. – gavin]
Concerning your comment, “More interesting is what is not contained in the emails. There is no evidence of any worldwide conspiracy, no mention of George Soros nefariously funding climate research, … no admission that global warming is a hoax….”
So? There’s also no evidence in the e-mails that the Earth is a spheroid instead of flat, that organized crime actually exists, or that the Earth revolves around the Sun instead of the other way around. Does that mean those things aren’t true, and that meanwhile we should ignore the evidence that is there? Really, that’s the flimsiest defense I’ve ever read.
I have the suspicion that “Climategate” is a last ditch effort by the sceptics in the USA to influence Copenhagen.
Although it’s not very scientific I put “Climategate” into Google insights.
If you can’t be bothered… large interest in the US… and that’s it.
If you go to a deeper level, it primarily seems to be California. WUWT?
http://www.google.com/insights/search/#q=climategate%20%2Ccru%20hack%2C&date=today%201-m&cmpt=q
Bob, (770)
“To PHG (761)So you do not even think it is a possibility that the science is just plain wrong.”
I’d like to believe it’s wrong but I work hard at not letting my personal beliefs get in the way of
reality. It is a well established fact the CO2 is a greenhouse gas, therefore more CO2 means more
warming. Out of curiousity I read skeptic/denialist sites that attempt to discredit the science but I have yet
to see anything credible and the chances of some new discovery countering the accepted science is
extremely small.
“Do you realize how many hundreds of millions of lives would be dislocated for wrong science?”
Since the science has not been proven wrong, nor is it likely to, this statement is not logical.
The “dislocation of millions” is not very well defined and I see no technical reason why this
should occur with some of the potential solutions that are already available.
“Why don’t we plan now for a direct meteor hit? You can’t be serious.”
You’re comparing a meteor strike, which has a very low probability and virtually nothing we can do about
to greenhouse gas emissions which we can do something about and has a very high probability of causing significant environmental damage? You can’t be serious.
“Climatatologists could learn something from genetic scientists who made the Human Genome Project such a shining example of international scientifc cooperation.”
Humm: is that such a shining example: IIUC the publicly funded researchers made their data available, but the privately funded researchers didn’t.
The grid system used by CRU to obtain the land surface temperature does come across as being confused in the Harry Readme. But the grid is almost entirely redundant. If you take the derivative of temperature at all stations each year, average them, and integrate the result from one year to the next, you obtain almost exactly the same curve that CRU obtained, but with only four lines of code, and operating on the raw data in less than a minute. To see the results, look at the Home Analysis section here:
http://www.hashemifamily.com/Kevan/Climate/
The grid serves to remove some duplicate stations, but does not affect the result in any significant way. So, it may be true that the grid system is messed up, but the underlying trend in the surface data appears to be there.
OK, I’ll bite. What is your evidence that climate science is a worldwide conspiracy – and what is the goal of that worldwide conspiracy?
Where is your evidence that this worldwide conspiracy is funded by George Soros?
Where is your evidence that global warming is a hoax?
You’re claiming to have evidence comparable to evidence that the earth’s not flat, so … show us!
It would be interesting to have the complete Freedom of Information Act requests made by Mr. Eschenbach published on-line somewhere, anywhere.
I also notice there is no explicit statement that Gradivus is a moron, and he has just given us incontrovertible evidence of that proposition.
Bart in 739: Because the beam flux is high I worry about multi-particle ionization interactions. As the recent Sinha NO2 ozone hoo haa shows, showing that something is due to high flux interactions or single events is not a very simple thing.
[Response: Not every FOI request is valid and other laws and various exemptions also need to be taken into account. Plus after ~100 FOI requests, most of which are asking for the same data that was justifiably rejected back in 2007, I think it is understandable that Jones was frustrated and defensive. This isn’t necessarily the ‘proper’ reaction but I see no evidence that this had any material effect. – gavin]
It is reasonable to reject FOI requests on the basis that the the law provides exceptions, or that the requests are vexatious or frivolous. But the CRU did not have to comply with the request 100 times. It had to comply only once -just once!-, especially if the most of the requests were asking for the same data. It is completely irrelevant that so many of the FOI requests came as a result of a campaign.
The exception in the laws that was used -namely, the confidentiality agreements- to deny the FOI requests seems to be only a convenient cover no to comply. If indeed, there is such binding agreements with the NMSs, then Phil Jones himself should have taken steps to find ways to release the CRU from such obligations so that the data became available to all researchers. Instead, as the emails suggest, his intentions were to do completely the opposite.
It does not take a scientist or a lawyer to conclude that this is breach of trust, if not breach of law. It only takes a reasonable person to figure out that his and the CRU’s actions(or non-actions) was not right.
#792 – aceq, you probably missed some of the earlier posts, but here is a summary to clarify you comments on the plot.
http://www.imagenerd.com/uploads/climate4u-lt-temps-Ljbug.gif
The filters used were a 40 year Fourier convolution filter. This filter works over the whole data range better then MOV or recursive filters, as was shown in the comparison between the EMD and Fourier filtering methods, in a previous post, in this CRU thread. The plot your talking about, is at the end of the Hadcet data from the 1850’s, and the E. England data from 1659. The Ave14 data series is a average of those temperature stations that began recording temperatures prior to 1800. In addition to the E.England series, they include Uppsala, Geneve, Berlin, Paris, Stockholm, Praha, Edinburgh, etc. These were obtained from
http://www.rimfrost.no/
The total Ave14 plot, reflecting western Europe is
http://www.imagenerd.com/uploads/ave14-smoothed-rev_cheb-j0m9Y.gif
so you can see, that is a long term series of actual temperature measurements. Since you say you work financial data every day, I don’t have to explain the MOV, Fourier convolution or “filtfilt” recursive filters as they are standard features of any good financial analysis (i.e. Tradestation, etc). So I assume you know the legend of the plot.
A couple of points, you might note is the persistent 50-60 cycle wave that shows up over the centuries, and the “global warming “ trend line is much flatter then those starting in the 1850’s.
Sorry about the Ave14 misunderstanding, I should have made that more clear.
Here is a very good discussion of this event from the ORNL Director of the Center for Molecular BioPhysics, Jeremy Smith:
http://clubmod.blogspot.com/2009/11/hacked-climate-change-e-mails.html
http://clubmod.blogspot.com/2009/11/massive-climate-data-scam.html
First, he describes the normal state of affairs:
In another set of e-mails Phil Jones, the present CRU director, is revealed strongly criticising other scientists, and even a journal as being not a legitimate peer review affair etc. All this seems par for the course as far as I can see: normal scientist private chit-chat. I wouldn’t be surprised if, in my own e-mails sent over the last 15 years there were evidence of lopsided views, bias, and discussions with collaborators as to cunning strategies to get our own work pushed on the community at the expense of other, competing scientific philosophies!
Yes, science can be a political minefield. Often, the required experiment or study is very obvious, and it’s just a political struggle over who gets to do the work, or rather who gets to claim the credit from finishing it first, hence who gets their next round of grant applications financed with no questions asked. It’s not a bad system, as compared to say, Lysenko, but there are obviously some problems:
How does science overcome the scheming, biases and collusion? By working with fact-based consensus. Occasionally, even a leading scientist who has published respected work may then publish something he/she believes is rigorously demonstrable and proven but in fact is unfounded, wrong and has simply slipped through the peer-review process (the reviewers didn’t recognise the problem). Plenty of papers in top journals qualify as such. However, if the work seems important several other groups, maybe from all over the planet, will independently take it up, repeat it, do other experiments/calculations that test it, and fail to substantiate it. The work is then naturally consigned to oblivion or moved to the slow-burner. Only by fact-based reinforcement do ideas gradually receive a solid consensus.
The author concludes with:
Although I am not a climate expert, my knowledge of the system leads me to believe that it is impossible that climate change be some kind of massive hoax or collusion or that the consensus is fundamentally wrong.
That’s about the best discussion of the issue I’ve seen. Consider what it would take to “rig” the data, for example:
1) The direct observational data – radiosondes, surface temperatues, buoys, etc. – all that data was rigged? Recall, it was collected largely for the purposes of Weather forecasting, not Climate forcasting – it’s difficult to believe that someone was messing with all those instruments all this time.
2) The climate models? The development of climate models predates more recent concerns over the rate of global warming. Arrhenius predicted that burning fossil fuels would lead to global warming, but he expected it to take a few thousand years – he didn’t foresee the sheer scale of the 20th century explosion in fossil fuel-based industrial and agricultural methods.
Putting that on more solid ground required numerical modeling, beginning with radiative transfer methods. Here’s a fundamental paper on the subject:
Manabe & Moller, On the Radiative Equilibrium and Heat Balance of the Atmosphere, 1961
http://www.atmosp.physics.utoronto.ca/people/guido/PHY2502/articles/rad-convec/Manabe_Moller_1961.pdf
How could all these models be set to “predetermine” the conclusion that a 3C increase in surface temperature would result from a doubling of atmospheric CO2, assuming a 1%/year growth rate in the atmosphere?
3) The paleoclimate view? All those ice cores and sediment cores? Was that data ‘predetermined’ to show a link between CO2 and glacial cycles? How was that done?
Note also that for paleoclimate records, ice cores and sediment cores are the main lines of evidence for the CO2-glacial link – tree rings are pretty far down the list of reliable proxies, because tree ring thickness is also sensitive to water and nutrient supply, not just temperature.
All in all, the only thing the email hack reveals is the depths to which some lobbyists and media outlets have fallen in their efforts to head off binding renewable energy targets in Copenhagen. Such binding limits would put an end to coal-to-gasoline and tar sand projects, which appears to be where the most fervent opposition is coming from.
@sHx: why would it be Phil Jones’ obligation to get the data released by the NMSs? He’d have to contact each and every organisation separately, have significant back-and-forth discussions with each of them, have *nothing* to offer them…for what? 2% of the data, which carries very limited extra information. And if he just released the data to anyone, he’d breach the law(!!), since he had confidentiality agreements with many of the NMSs.
I hope you are also aware that Phil Jones is a scientist, not an administrator whose job it is to help others all the time? He’s also human, and would become ‘contrarian’ when he knows the data will not be used in a proper scientific way, which he would then have to correct again. If there is a breach of trust, it was caused by the actions of many ‘skeptics’ abusing data. Just look how the New Zealand Climate Science Coalition started an attack in New Zealand, based on data abuse. I don’t see any of the ‘skeptics’ harumphing about that abuse by the NZCSC. I haven’t seen one ‘skeptic’ being angry at Soon and Baliunas for their abuse of prior literature. I have seen very few ‘skeptics’ dismiss the McLean et al paper on ENSO (mainly after Lucia Liljegren noted it could be used to claim a larger underlying warming trend). “Hypocritical” does not even come close.
I think you yourself are mixing up the letter of the law and the spirit of the law. FOIA was not set up to allow abuse of the released information. We know from the actions of the people asking for the data that they *would* do so.
Simon Abingdon thinks the lack of coding provably correct applications in industry and academia is “All a bit worrying really.”
Do you fly? Does it bother you that flight software is not perfect and is continually upgraded? How about medical device software? Also not perfect. I’ve worked in both aviation and medical software, and I still fly — and buy carbon offsets when I do!
> “Why don’t we plan now for a direct meteor hit? You can’t be serious.”
How did you miss the planning already underway to anticipate exactly that?
> Since you say you work financial data every day
Do you consider doubling CO2 to be analogous to doubling (or halving) the velocity of money, the quantity of money in circulation, or the base interest rate? Or do you assume no change in the underlying condition?
Please don’t answer that here. You have a live topic at Stoat, and Gavin asked you to take this enthusiam elsewhere, not use unrelated topics here.
Fyi, I just bought Gavin’s and James Hoggan’s books as a little Solstice gift for myself.
By law, they have to assign each to a review officer and process and respond to each one, individually.
Yeah, complying with contractual agreements is just “convenient cover” … you guys really have no respect for IP, contract/agreements or the law, do you?
> J. Bob
Oh, not at Stoat, somewhere though. I’m sure you’ve gone on about that at length. Maybe at Tamino’s.
Gavin. You say “Plus after ~100 FOI requests, most of which are asking for the same data that was justifiably rejected back in 2007.” Is that really the case??
My understanding of what happened is that initial FOI requests were rejected on the basis that there were agreements with certain countries that prevented the release of data relating to those countries. The many FOI requests were actually a single FOI request in relation to each country, requesting that the agreements relating to each of those countries be released, thus proving the statements made. The FOI requests were therefore specifically NOT for the same data. Each requests was for different data.
The many FOI requests came about in response to the “unresponsive” stance taken by CRU in relation to the initial FOI requests.
[Response: It’s not ‘unresponsive’ to tell people why that particular data can’t be released. It may however be unsatisfying. – gavin]
The case against the scientists’ professional behavior does not necessarily disprove the theoretical case for anthropogenic global warming. But it does provide valid arguments against receiving the majority AGW theory as “settled science” on which to decide public policy, simply because the non-scientific way by which much of the consensus was promoted is unsettling.
Dr. Jerry Pournelle IMHO provides one of the clearest view of the relevant and material issues here:
http://www.jerrypournelle.com/view/2009/Q4/view598.html
[edit- text replaced with link]
[Response: Pournelle is wrong in every single factual statement and dramatically misreads the emails to support a preconceived narrative. This ends up neither being clear nor relevant. – gavin]
Spencer:
It’s completely discrediting, and unethical, and probably illegal that you’ve covered up the shaky grounds of radiation science so long, and that we had to discover it with FOI actions. If Ms. Curie didn’t respect the ethical bonds of a marriage contract, what evidence is there that she respected the openness requirement of publicly funded research? As usual, the French taxpayer is the last to know. Now I understand why so many said they would cheerfully live on a pile of allegedly radioactive waste or eat plutonium or what have you. And now I am left to wonder what’s really going on when that smiling nurse or dental assistant leaves the room and turns her bug zapper on.
To sHx – post #812.
The problem with your post is that you use words like “seems” and “suggest” as the main plank of your claim of a CRU conspiracy to withhold evidence. It only “seems” and “suggests” that to you, and your fellow conspiracy theorists.
Ridiculous.
Re: divergence.
(From AR4 chapter 6)
“This ‘divergence’ is apparently restricted to some northern, high-
latitude regions, but it is certainly not ubiquitous even there. In
their large-scale reconstructions based on tree ring density data,
Briffa et al. (2001) specifi cally excluded the post-1960 data in
their calibration against instrumental records, to avoid biasing
the estimation of the earlier reconstructions (hence they are not
shown in Figure 6.10), implicitly assuming that the ‘divergence’
was a uniquely recent phenomenon, as has also been argued by
Cook et al. (2004a). Others, however, argue for a breakdown
in the assumed linear tree growth response to continued
warming, invoking a possible threshold exceedance beyond
which moisture stress now limits further growth (D’Arrigo
et al., 2004). If true, this would imply a similar limit on the
potential to reconstruct possible warm periods in earlier times
at such sites. At this time there is no consensus on these issues
(for further references see NRC, 2006) and the possibility of
investigating them further is restricted by the lack of recent tree
ring data at most of the sites from which tree ring data discussed
in this chapter were acquired. ”
Is this supposed to assuage the concerns of people who think the divergence problem is indeed a problem? I don’t think it does a very good job.
This peer reviewed paper also throws some more light on the divergence problem:
http://www.springerlink.com/content/45u6287u37x5566n/
#78
Like most geologists, the commenter fails to take into account the huge differences in the time scales involved. (Note his reference to the “interglacial WEATHER pattern”.) Taking this approach, Mr. Reynolds presumably advocates that pilots be required to take continental drift into account when plotting a course.
Yep, (out of context)
I see the BBC’s Richard Black gets mentioned by Dr. Mann:
…extremely disappointing to see something like this appear on BBC. its particularly odd, since climate is usually Richard Black’s beat at BBC (and he does a great job).
…We may do something about this on RealClimate, but meanwhile it might be appropriate for the Met Office to have a say about this, I might ask Richard Black what’s up here?
What happened to global warming?:
http://news.bbc.co.uk/2/hi/science/nature/8299079.stm
[Response: Your point? Richard Black does indeed do a pretty good job. – gavin]
Hi,
My interests mirror those of Mark in comment 795. Perhaps you guys could do a posting that addresses this divergence problem more clearly sometime. The problems that seem to be inherent in dendrochronolgy are being communicated quite concisely and effectively to the layman. The questions being posed should be fairly standard fare for the experts in the field and I suspect (or hope) should be able to be cleared up very easily. Gavin, your point has been made a number of times that we don’t need trees to confirm that it has warmed over the past couple decades. Very true, we have thermometers these days. I think Mark summed up the real concerns when he says.
‘Now my question is given all of this – why should I believe the tree rings are a good proxy from the year 1000-1850 (I think 1850 is when the temperature record starts) if we do not understand why it does not work from 1960-2009? How can we be sure the same problem is not present at earlier times thus rendering the reconstructed temperature record invalid?”
Thanks
[Response: Of course. That’s why people are so interested in the issue. However, the curious thing is that this particular proxy is highly correlated with other proxy measures and instrumental records for the rest of the time. So while for this MXD proxy there are questions remaining before you would rely on it absolutely, there are plenty of other proxies that don’t have this problem that allow to say something about previous centuries. – gavin]
This is kinda late to be chiming in but I don’t find any earlier comments on the “rid themselves of this troublesome editor” phrase in the hacked messages. I know you’re all scientists but doesn’t anyone recognize the reference to Henry II and Thomas á Becket? Henry asked his court “Who will rid me of this troublesome priest?” and a couple of knights obligingly went over and killed Thomas. Google “Thomas Becket”.
I assume that Phil Jones is the victim of a classical British education and was using the Becket trope to add a little ironic backspin to what he was saying. I guess you have to have read T.S. Eliot’s Murder in the Cathedral to get the point.
Mike E — 24 November 2009 @ 6:13 AM:
“It would be handy, please, to have an acronyms index…”
Mike, this is a problem that has been very frustrating to me and, I think, a great disservice to newcomers to the site. This is especially true because this RC (Real Climate) site is intended for educating the general public. It is common for scientists to use acronyms in their publications, but here the convention is to define the acronym the first time it is used. In general the climate scientists that post here do this, but almost all of the other commenters do not. My posts of your complaint have fallen into the same black hole yours has. I am sure that the lack of an acronym index is due to the hassle and time it would take to assemble one.
So, here is my version of an index that has been slowly accumulated from reading carefully, and from using AcronymFinder (.com). AcronymFinder can be very trying, because of the large number of possibilities for each acronym, but also very amusing because of how the substitution of an alternate acronym alters the meaning of a post (e.g. “I am very concerned with how AGW [A Girl’s World, a website] will impact our society and very existence”).
Anybody, please correct and add to this as necessary. Steve
[edit]
[Response: Thanks I’ve moved this to a special Acronym Index page. Please comment if there are any additions required. – gavin]
#781 Dayahka
This is the old Michael Griffin argument (former head of NASA during the Bush administration).
Part I
http://www.uscentrist.org/videos/word-items/mission-control
Part II
http://www.uscentrist.org/videos/word-items/airogance
Gavin has raised relevant considerations, to paraphrase a campaign phrase, ‘It’s the infrastructure/economy stupid’ and of course resource capacity in relation to the multiple climate system shifts and oceanic chemistry, and land use capacity changes…
Consensus is not the equivalent of truth. You can easily achieve the wrong “consensus” when a point of view is adopted by a large enough number of people who have not properly questioned a prevailing orthodoxy. In this case, a major pillar of the argument in favor of AGW seems to be resting on some very questionable computer code. The fact that a large number of scientists may have glossed over these problems is not that hard to believe. This kind of thing happens not only in science, but in many other fields, most notably politics, and it happens quite often. Remember the “consensus” that Saddam Hussein had WMD’s and their use was imminent?
[Response: No argument for AGW relies on any of the CRU computer code. The GW bit is viewable in any similar dataset (GISTEMP for instance, all code available), glacier retreat, ocean temperatures etc. The ‘A’ bit comes from studies in attribution and that doesn’t depend on any single data source either. – gavin]
simon abingdon: Sorry Gavin, it won’t wash. You have to structure your programs properly from the outset. You can’t incorporate good practice retrospectively.
BPL: I was a professional computer programmer for 15 years, much of it spent translating old FORTRAN code to new versions–e.g. at the USX Edgar Thomson 44″ Slab Mill from 1988 to 1992. Doing it a bit at a time while the code was actually in use was EXACTLY how we did it. One OBVIOUS technique (if you understand programming!) is to rewrite one subroutine at a time, thoroughly test the new version, then link it in in place of the old version. I assume without looking that the GCMs are masses of subroutines, not a single solid block of code. That means you damn well can take it a piece at a time, retrospectively.
@ J. Bob — 27 November 2009 @ 11:24 AM
“The filters used were a 40 year Fourier convolution filter.”
If you apply the same Fourier Convolution filter to the Arctic summer sea Ice minima time series, what period is required to show a recovery? I would surmise that a 40 year period is inappropriate for the much shorter dataset http://arctic.atmos.uiuc.edu/cryosphere/IMAGES/seasonal.extent.1900-2008.jpg (starting at 1900 instead of 1650), but can you show a sea ice recovery wth a 20, 10, 5 or 2 year Fourier convolution filter?
Alasdair Green: Climate science is in its infancy, is it not?
BPL: Not even remotely. I’ll refrain from mentioning Aristotle’s division of world climate zones into torrid, temperate and frozen 2300 years ago, or Torricelli’s barometer work. I’ll just recommend that you Google the “Hadley” the Hadley CRU is named after. BTW, even the theory of anthropogenic global warming is 113 years old, which makes it older than relativity or quantum mechanics.
I’m not a climate expert, but I am an engineer and someone that tries to get beyond hype and look at things from a reasonable perspective. The emails regarding keeping data private and blocking FOIA requests made me wonder: Have reputable scientists that are in disagreement with CRU (such as Lindzen at MIT) been given access to any and all raw data used by CRU if they requested it?
– Ed
[Response: Why do you think that Lindzen is in disagreement with CRU? Every time I have seen him he has been very happy to accept the actual fact of global warming (he disputes it’s interpretation, but that’s a whole other story). – gavin]
Gavin,
Putting the CRU hacked emails in context is no easy task. You have done a good job thus far.
As a climate expert/insider, we understand that your task as moderator is limited to the formal issues at hand.
However, one of the big “Meta-issues” is the current state of climate science and public perception, where Climatologists are rapidly losing ground. I am one of many fence-sitters who do not believe for a minute that all of you are evil and nefarious.
That being said, there is a widely recognized global warming detractor (more level headed than most, I might add) who has extended a hand to all climate experts who may want to contribute editorally at his blog, with full author privileges and no restrictions on content.
IMO, you would be a great contributor. If you feel there would be a conflict of interest, perhaps you could talk/persuade a colleague who might be up to the task.
Sooner or later, the climate science community will need to address the Meta-issues.
What is your opinion?
Since the CRU story broke, I’ve been having to fend off an avalanche of hysterical deniers who have hijacked Dot Earth- temporarily, I hope. My qualifications cannot match any of the blog managers here, or many of the expert commenters, either.
A few have come to my aid, like Chuck Wilson, but more are needed. We can’t blame Andy this time- he’s actually acting pretty fairmindedly. Insight about the specifics of the CRU incident are needed over there, too.
There is a sympathetic (neutral?)report/interview on past climates as seen by Michael Mann on the BBC (UK) news website today.
This is a part snipped from that:
“Most climate models used by the Intergovernmental Panel on Climate Change (IPCC) show that the Earth will respond in an El Nino-like way to global warming.
But a few of the models do recreate this dynamic “La Nina effect”, and suggest that that when you heat the Earth’s surface, the climate system tries to offset and cool.”
“If the response of the Earth in the past is analogous to the temperature increase caused by greenhouse gases… it could lend credence to this counterintuitive notion of a La Nina response to global warming,” said Professor Mann.”
In the present over-heated politicisation of climate change, I am a wee bit uncertain what Mann is saying.
Is he saying that CO2 (or other forcings) could mean that (excuse the Irish logic here) that global warming _could_ cause global cooling?
[Response: No! But there is a lot of uncertainty in how the ENSO phenomena will react to global warming. The models in IPCC are all over the place, and there is plenty of reasons not to put too much credence in their projections of this (insufficient resolution, biases in the control simulations etc.). Thus people are looking for paleo-climate examples when there was a change in forcing and a detectable change in ENSO. The medieval period is one point where you have some evidence for increased solar irradiance combined with a La Niña-like response (derived from coral and other tropical proxies). Thus if this was a generic response to positive radiative forcings, then that might be an interesting projection for the future. Note though that this leads only to a relative and localised cooling in the tropical Pacific, not a global cooling. – gavin]
#786
[Response: …..was simply an internal test to see what it looked like…… – gavin]
Why on earth would they wanna like to test something that weird? Any ideas Gavin?
[Response: I think they were trying to test whether the regional calibrations were stable so that if it turns out that the divergence is anthropogenic, the proxy record going back would be different or similar. Either way, it was a dead end and not used in any paper. – gavin]
antonyclark wrote in 800:
Gotos can’t take you anywhere — not in C#. It has to stay in scope. See the piece I quoted above in 754. And a good programmer or team would be able to achieve the same with the appropriate discipline — regardless of what the language makes possible.
But as I pointed out in 754, bad use of GOTOs really isn’t what is responsible for an inability to “thoroughly” debug software. That goes much deeper, and it isn’t simply an issue for programming but for algorithms and human methods in general. And one aspect of it has to do with combinatorial complexity — particularly where the potential number of paths grows super-exponentially. Another (Godel’s Theorem) has to do with the language in which programs are written — but which grows out of the nature of mathematics. However, neither prevent us from writing good software.
If we wade through the chaff of this brouhaha we will find that the strutting and fretting has nothing to do with climatology as a science, but with three meta-principles, three protocols, that need to be affirmed or re-affirmed by the scientific community generally and climatology in particular; I’m afraid that it is violation of these meta-principles that has led to this mess.
The first meta-principle is that broad sweeping generalizations, ad hominems of any kind, and assertions of certitude, completion, or of unquestionability are fallacious, irrational, and unscientific. Climatology needs to repudiate anyone who resorts to characterizing swaths of the population in broad, sweeping, ad hominem terms, or who cast aspersions on the intelligence, maternity, paternity, progeny, and education of anyone. Climatology needs, for example, to repudiate Al Gore, a non-scientific journalist of dubious scholarly credentials as having the right to speak for or about climatology; the same for Senator Inhofe of Oklahoma, and for a host of other partisans in the grips of the most dangerous nonsense, binary logic, with its principal axiom, I am right and you are wrong.
The second meta-principle that needs to be affirmed or re-affirmed is that any scientific hypothesis or methodology can be questioned; science is, after all, the act of proposing hypotheses and then inviting anyone to chop off your head by finding that one beautiful little fact that slays your hypothesis. We have to accept intra-climatology disagreements. Someone may claim that some event is an outlier, while another claims it is paradigmatic of some trend. Enormous consequences follow from adopting one or the other interpretation. The very word climate can be subject to controversy. I would claim, for example, that there are only three types of climate, frigid, torrid, and inter-glacial, and that within these three you will find variability that tends towards the frigid, the torrid, or the inter-glacial normal. I would thus repudiate any claim that a climate system is anything less than a 10,000 year system; and so, anyone who says climate must be changing because last year I was in zone 6 and this year I am in zone 5, is just simply mistaken and has taken variability events as trend or type events.
The third meta-principle that needs to be affirmed or re-affirmed is that there may be genuine and legitimate differences over what is to be done because of some trend in the climate. In Response #122 and #781, for example, Gavin affirms the view that the reason why we need to oppose and try to stop warming is that it would be severely disruptive of our current civilization and exorbitantly, if not prohibitively, expensive to have to re-build all the cities, infrastructure, and so on of the current system.
But you see, it is precisely for some of these reasons why I think we should oppose all attempts to mitigate the climate system, as I do not think the current genocidal and ecocidal civilization is worth preserving. I would rather see us burn up all our oil, and then face all the turmoils of finding new locales for habitation instead of the cubicles for the slaves of industrial civilization that we call cities. If we were spending all our time and energy in desperate attempts to build suitable habitations, develop a different system of agriculture based on wastes, we would then have no time for war or toxic derivatives or all the oil-based flotsam that clogs our oceans, spoils our air, and poisons our land.
Duae Quartunciea says – “The CRU has been working to try and make the datasets more available; but they are constrained by their own legal obligations to other data providers”
Again, it would add tremendous credibility to this statement if even one decision maker at one of the data providers said “yes, we will not allow CRU to release our data, and if they do we will not be able to provide data to them in the future.”
Surely you see how a simple statement like this from a data provider would go a long way towards validating the CRU stance?
I mentioned earlier, and had a question on, an interview/report with Michael Mann, carried on the BBC website (so also on radio and TV?).
What was pleasing was the CRU hack was nowhere mentioned, and if Mann was asked about this, the reporter did not see fit to publish it, as someone is considering it a storm/teacup thing.
(Mind you,of course the BBC is a hotbed of lefties who want to take us back to the stone age.)
Judith Curry:
Judith Curry has some interesting things to say about the CRUgate scandal, and general skeptic vs alarmist interaction
http://dotearth.blogs.nytimes.com/2009/11/27/a-climate-scientist-on-climate-skeptics/#more-11377
http://camirror.wordpress.com/2009/11/22/curry-on-the-credibility-of-climate-research/
I think her tribalism analysis is pretty much on the button, as is her ‘circling the wagons metaphor’.
– she has a good insider/outsider perspective
– any thoughts, Gavin?
Cheers
799 – “This made me laugh! Clearly you are unfamiliar with the halting problem, and any number of other undecidable problems”
The issue with provability comes from Kurt Godel’s theorem of incompleteness which states that no self referencing system with formal rules can be complete and consistent at the same time. In a basic nutshell, the halting problem was one of the first undecidable problems proved in mathematics.
The halting problem, however, does not mean that you cannot prove your code. The halting problems means that you can by no general method prove all algorithms. You can prove a specific algorithm to be true, false, or even undecidable.
“If all software was “provable”, then all our lives would be a good deal simpler.”
All software is provable as stated above.
800 – “A BREAK statement is NOT essentially the same as a GOTO or RETURN.”
Yes it is. The difference between a break and a GOTO is that you do not get to specify the address to jump to in a break statement; instead, the compiler does that for you.
“A BREAK statement takes you to the end of the scope of a loop, it does not take you out of the scope of the loop.”
Yes it does.. Look… (this code may come out ugly on the blog, but hopefully, it will be ok)
for (i=0; i<20; i++)
00411395 mov dword ptr [i],0
0041139C jmp wmain+37h (4113A7h)
0041139E mov eax,dword ptr [i]
004113A1 add eax,1
004113A4 mov dword ptr [i],eax
004113A7 cmp dword ptr [i],14h
004113AB jge wmain+50h (4113C0h)
{
if (i == 105) break;
004113AD cmp dword ptr [i],69h
004113B1 jne wmain+45h (4113B5h)
004113B3 jmp wmain+50h (4113C0h)
i++;
004113B5 mov eax,dword ptr [i]
004113B8 add eax,1
004113BB mov dword ptr [i],eax
}
004113BE jmp wmain+2Eh (41139Eh)
return 0;
004113C0 xor eax,eax
See the jump statement at 004113B3? It goes to the return statement when break is hit. In other words, the break command cause the loop to be exited completely.
801. “The majority of bugs in software are outside of the code. They are things like a misunderstanding of requirements, poor mappings from requirements into design, inaccurate validations on input, inaccurate specifications etc. None of these problems can be solved with mathematical rigor. “
Protocols can be validated mathematically and so can validations on inputs. Now if someone was suppose to design a nuclear simulator and designed a fast food simulator instead, no amount of mathematics can cure that problem.
There is quite a lot of half-remembered computer science talk here. On mathematical provability – Djikstra, author of the goto considered harmful letter, was also very concerned with mathematical provability, but his efforts were never generalised to production-use languages. No-one develops proofs, not because they are incomprehensibly lazy, but because it cannot be done for real-world applications with real-world languages. As to goto considered harmful article, Knuth wrote a response titled “goto considered useful”. Of the two, Knuth had by far the larger real-world impact. Abingdon and some of the others are just displaying the usual reflexive appeal to science (only mathematics or computer science, never the other kinds) that you get when you ask a programmer to think about hard things outside of syntax. We know a hell of a lot more about software quality and the cost of writing software these days than we did when structured programming was being introduced in the 70s.
Mike Rivero of whatreallyhappened is a huge denial purveyor. He has a huge readership, usually is good on political observations, but I have emailed him, and he keeps repeating this line over and over as below. This is one email that I asked him to explain all of the information supporting scientists, and NOT politicians and Exxon Mobil.
—————————————————————————————————————————-
From: wrh@whatreallyhappened.com
To: XXXXXXX@hotmail.com
Subject: RE: Global Warming
Date: Mon, 23 Nov 2009 13:45:50 -1000
You’re raising on a busted flush.
Michael Rivero – What Really Happened.
whatreallyhappened.com
From: Old Hippy [mailto:XXXXX.com]
Sent: Monday, November 23, 2009 1:36 PM
To: wrh@whatreallyhappened.com
Subject: Global Warming
Please Explain these:
http://tinyurl.com/22go3n
http://tinyurl.com/yzgojux
http://tinyurl.com/2kzqd7
http://tinyurl.com/2ozbrc
http://tinyurl.com/yhealhs
http://tinyurl.com/mrmb9j
http://tinyurl.com/yfq5tam
http://tinyurl.com/yhuhceh
http://tinyurl.com/yf8pekp
#794 sHx
“That there is not a simple, international “open data” agreement in
place already is a finding I did not expect. Climatatologists could
learn something from genetic scientists who made the Human Genome
Project such a shining example of international scientifc cooperation.”
What utter poppycock. The sequencing of the human genome was vastly more
secretive and controversial than withholding a few bits of weather data.
It has been said in the past that such secrecy probably held back
medical advances by as much as 10 years or more, and it is still
impeding such in a major way.
“Statement by Dr. David King at HGA press briefing 14/6/00″
” ‘Any company that wants to be in the business of using genes,
proteins, or antibodies as drugs has a very high probability of running
afoul of our patents. From a commercial point of view, they are severely
constrained – and far more than they realize.’ Dr William A. Haseltine,
Chairman and CEO, Human Genome Sciences. ”
“It has been evident from the start that a major aim of the Human Genome
Project (HGP) was to support the pharmaceutical industry. However, it is
unlikely that the project’s founders anticipated the degree to which
basic genomics research would turn into a gold rush, and the unpleasant
conflicts that have arisen over patenting and sharing of data. Private
companies are now going to extraordinary lengths to monopolise
information in ways that conflict with the basic ethics of science. The
result is a situation in which a a relatively small number of companies
are in a position to exert significant control over the whole future of
medicine.” – from ‘The Human Genetics Alert’, a group dedicated to
rescinding patents etc covering the human genome (site seems to be
considered a potential spam site by RC server)
From the very start it was a race between commercial and non-commercial
interests. With the result that large tracts of the human genome are now
patented and cannot be freely used for, or in the development of, any
potential commercial use, and that includes any medical treatment.
Of course there is a difference between genetics and climate data: The
former is easy for anyone to get now at any time, simply by sequencing
their own genome. (hard to do that with historical temperature records).
In the early days it wasn’t so easy to get and there were any number of
companies (and Universities in some cases) keeping their data secret. It
was only as DNA sequencing became easier that it was realized that
keeping the data secret was no longer an option, and everyone turned to
patents.
Andrew
Uh, everyone credible understands that the divergence problem is a problem (note the use of the word “problem” in the name).
The issue is the misrepresentation and exaggeration of the problem by certain people with a certain political agenda…
So what does this tell you? It tells me that even if Briffa etc are wrong about the divergence problem being unique to the last few decades (most likely due to some anthropogenic effect such as pollution) then the divergence problem will crop up in the past during very warm periods.
Really, AFAIK, the only candidate for a period warm enough to trigger such a threshold effect would be the MWP.
So maybe the MWP was warmer, at least regionally, than some reconstructions show.
This disproves AGW how, exactly? This tells us we can’t worry about its effect on today’s human population and infrastructure for what reason, exactly?