Much of the discussion in recent days has been motivated by the idea that climate science is somehow unfairly restricting access to raw data upon which scientific conclusions are based. This is a powerful meme and one that has clear resonance far beyond the people who are actually interested in analysing data themselves. However, many of the people raising this issue are not aware of what and how much data is actually available.
Therefore, we have set up a page of data links to sources of temperature and other climate data, codes to process it, model outputs, model codes, reconstructions, paleo-records, the codes involved in reconstructions etc. We have made a start on this on a new Data Sources page, but if anyone has other links that we’ve missed, note them in the comments and we’ll update accordingly.
The climate science community fully understands how important it is that data sources are made as open and transparent as possible, for research purposes as well as for other interested parties, and is actively working to increase accessibility and usability of the data. We encourage people to investigate the various graphical portals to get a feel for the data and what can be done with it. The providers of these online resources are very interested in getting feedback on any of these sites and so don’t hesitate to contact them if you want to see improvements.
Update: Big thank you to all for all the additional links given below. Keep them coming!
>FWIW, the tack on talk radio (they move like a herd of hooligans) is now to refer to AGW as “crank science.”
This is a larger and more dangerous issue than just the one at hand. I spend as much if not more energy with my *left* leaning friends counteracting anti-vax crap, suspicion of modern medicine, and the post-modernist brain rot they picked up in school. In fact I believe it’s “deconstruction” as promoted by supposedly progressive academics that leaked out of the universities and enabled things to get this bad. If you’ve ever argued with one, the techniques aren’t quite as frat-boy ugly, but they are similar.
The fact is that the scientific method, and the body of knowledge built up, constrains people’s scope of plausible beliefs. This rankles them, as any constraint might.
Re the open source code, what might be really interesting is to take the source code and data and make a simulation “game” for the layperson. Input a set of assumptions, and show outcomes with best-guess probabilities, including side-paths to see the mechanisms involved. The same “game” could have a toolkit to show the impact of lifestyle choices and technology improvements.
Re the Briffa data- I searched for “Yamal” in the Tree Ring page you link to, and found two items. With a total of ~150 rows of undocumented data between them. :
Here’s the top page:
http://www.ncdc.noaa.gov/paleo/indextree.html
and the two reports:
http://www.ncdc.noaa.gov/paleo/metadata/noaa-tree-4770.html
http://www.ncdc.noaa.gov/paleo/metadata/noaa-tree-4769.html
These don’t seem to have any of the source data at all?
@Gavin,
I have nearly finished reading the Copenhagen Diagnosis. Could you re-open that thread so I can see what the regulars and you think of it? I so often learn more from those discussions than the more formal papers. Thanks.
Tom Franklin, you will of course be able to provide us exact reference to what data has been repressed, since the mails show this to be the case.
At the same time, you will of course also be able to provide evidence of what laws of physics the scientists have changed, to fit with AGW. This would be necessary, as AGW follows straightforward from the known physics – those dastardly scientists!
50. “Then I just kept hearing things about maybe there was some cooling going on. So, I thought, well that sounds like worth doing some serious checking on. Then I started to hear the old “it’s all settled” line. Gosh, I have never known anything to be settled in my long life.”
The world is not cooling:
http://www.huffingtonpost.com/wires/2009/10/26/ap-impact-statisticians-r_ws_333941.html
The skeptics were mathematically wrong with the entire global cooling nonsense that they were trying to convince people on. The math was wrong, and it was intentionally wrong.
30 “Perhaps you could clarify in the main post that all this data has been available for a long, long time? From some of the comments, it would seem that many still can’t get their head around the idea that there has always been oceans of data out there waiting for the deniers to analyse.
I can’t imagine what’s holding them back.”
People see whatever they want to see.
Wow! That’s a lot more data than I had found (or even know what to do with). Next time someone says, “They need to release the data” I will link them here. But first I will get them to specify which data they think hasn’t been released so they can’t move the goalpost as easily by claiming this isn’t the data they’re looking for.
O/T: I just read Idiot America. It has a chapter on global warming. A fun read unless you get most of your “science” from AM radio talk shows.
Hi Gavin,
It might be useful to some if you add a list of commonly used data analysis tools to the master list. Here’s a few (mostly for netCDF or OPeNDAP formatted data, which is one of the most commonly used formats for gridded climate data) that I find useful and might be good to start with:
Climate Data Operators
netCDF operators (NCO)
Ferret
GrADS
Pydap
Climate Data Analysis Tools
– Joe S
I think we’re seeing the effect of loosening the moderation…Gavin or the others, if you were planning to answer Silke later, just remove this post.
@Silke:
1. Climate change is climate change, regardless of the source. We’re part of nature, too, so if you want you can always call the current climate change “natural”…
2. People who are called “climate change deniers” are those that neglect an enormous body of research, misinterpret and overinterpret some other work, and subsequently claim that “it’s all natural” (but see point 1)
3. It’s funny you mention the peer-review process, since the e-mails discuss several papers that are fundamentally flawed, and of which one has very likely entered the literature because of a biased peer-review process. It was a paper by ‘skeptics’ (Soon and Baliunas), and bad processing by a ‘skeptic’ editor.
4. The IPCC does not fund research, and the fact that you do not know this is quite telling. The IPCC frequently does call for certain factors to investigate more, which notably include points brought in by skeptics (land use changes, solar influences, etc.). Majority opinions in science change rapidly when the minority opinion provides a better explanation. So far, the minority opinion is found extremely wanting, yet it is still discussed in the IPCC reports.
5. Debates are popularity contests. On top of that, all the ‘skeptic’ needs to do is create doubt. With a largely scientifically illiterate audience on the topic at hand, this is really easy, especially if there is something at stake. It is the preferred tactic of the creationists in debates with evolutionists.
6. “Common sense” isn’t always right. In the case of clouds, if we take your ‘common sense’, it is simply wrong. Real “common sense” actually would say that clouds should have both a cooling and warming effect: they reflect sunlight, but also trap heat.
7. Regarding model testing and validation, you can get some basic understanding here:
http://www.ccsm.ucar.edu/working_groups/Software/dev_guide/dev_guide/node14.html
The terminology is the same for ANY model, climate or not.
8. I guess Gavin can best enlighten you here. I can give you a pointer:
http://en.wikipedia.org/wiki/Global_climate_model (and references therein)
9. The IPCC was set up with the explicit goal to find out how much influence anthropogenic influence had on global climate. The claim that it was set up to blame humans and that scientists were bought is outright false, and indicates NWO conspiracy thinking.
10. Life on earth depends on many things. Without nitrous oxide, much of life will die out. Too much nitrous oxide, and it will die out, too. The idea that more CO2 is good for life needs a LOT of qualifiers. For starters, there are a lot of plants that are evolutionary optimised for low amounts of CO2 (this includes quite a few foodstuffs on which most people are dependent). With increasing CO2, other forms of plants take over, also exerting evolutionary pressure on the animal kingdom. We know that humans have only encountered low CO2-life, and thus can only hope (and pray, if you are religious) that the high CO2-life isn’t too deadly for us. The claim that all earth scientists believe that anything humans do is bad, is simply untrue. Earth scientists, however, are (hopefully) aware that there are many interlaced processes. Altering one part *will* have effects elsewhere. Sometimes calling something “good” or “bad” really depends on the desired outcome. If your desired outcome includes creating a new eco-system, making a dam for hydropower is fine. If you like to maintain an eco-system, building a dam is not a good idea. I can come with hundreds of such examples.
Oh, I see. Thank you. Still, it would be a lot of work to verify that the archives match the actual e-mails, this work has not been done, the source is criminal, yet here we are with people using these e-mails as evidence for very serious charges.
Data? What data?
http://www.timesonline.co.uk/tol/news/environment/article6936289.ece
If I could also make a suggestion – it’s not much good putting all that code up without a description of how to interpret it.
I decided to wrap my head around how a GCM worked two years ago. I downloaded code (modelE as it happens) and could not understand it. I thought documentation was the problem, so I got the NCAR documentation and model and could not understand it. The various IPCC reports might be fine literature reviews but they are lousy texts.
Then I ordered Washington and Parkinson’s textbook – with that (and EdGCM – no longer free sadly) finally I was getting somewhere.
For people who are actually interested in finding out how this stuff works in detail a short reading list of 3-5 textbooks would be a *lot* more useful than a mass of code. Perhaps a link to your “books” page and a brief reading list would be a good idea?
What you write above is exactly the logic that proves those claiming everything is hidden, so everything is suspect are basically lying.
First, you point out most of those casting stones have no clue *how* to sue the data. This is clearly shown by the request, as you make above, for info on *how* to use this material. And do they think they can become experts in climate modeling in the next week or two? Or even a year or two?
Second, the above point is augmented by the request for info showing EXACTLY how model results were achieved, how the data was created, how it was analyzed, etc. Yet, if you simply replicate exactly what others have already done, what have you gained? No new info. It is more useful and effective to do it on your own. This adds robustness to the data gathering.
If you suspect the work already done is in error, then examine that as is and show where the errors are. This is far more efficient than simply saying 100 other people must replicate exactly what others have already done.
This is a simple logical argument based on the fact that many, many lines of inquiry are used to elucidate Anthropogenic Climate Change. This focus on climate models is just an extension of the lie that climate models are all wrong, all suspect, and can’t do what they claim to do. This is a lie. The lie is proven by the FACT that the models do replicate past climate and do give us a sense of what is happening and will happen. All the more so because they are generally **underestimating** the change… which is another logical bullet hole through the denialists’ harangues. How can they claim the models don’t show warming, and can’t, when the models are showing less than observed? If anything, our models are even better than we think because they are obviously finding the correct trend even though we obviously need to improve the models to capture the extreme rate of change.
This is not to say replicating exactly what has been done has no value, it does. But with so many lines of inquiry all showing the same thing, this is not vital. It is more important at this point to increase our understanding, not repeat the last 100 years of science because some people who have an agenda want to slow things down even more than they already have.
Third, as alluded already, observation trumps all else and is the basis and bedrock of all climate science. Scientists don’t think the globe is warming because they made a model and it told them so, they think it is because they are looking at and measuring changes in the environment and THEN making models to help them understand why, how fast, and what the effects might be.
But this point is studiously ignored by the anti-AGW conspirators. (Yes, conspiracy is a historical fact, unlike the fantasy of a pro-AGW conspiracy which has no basis in fact, no evidence, no proof.)
It is good the scientists are going even further than they already have to prove their honesty, but it is a shame they have to. Simple logic proves AGW. All you have to do is open your eyes.
These people will not be able to disprove the science by having the (already mostly long available) code. They WILL use it via the GIGO method: they will claim THEIR data is better – though provided by garage scientists – and THEIR results prove AGW is bunk. Then the true scientists will have to go through and prove their work is badly done and wrong. Then the denialists will simply claim it’s all a conspiracy and that their bad science is being suppressed.
That is, Gavin, et al., are wasting their time. They should be left to do the science that has given us a chance to avert catastrophic, or even highly destructive and/or disruptive change. Instead, they must refute lies and distortions long ago manufactured by think tanks that knew they were lying and distorting the science even then, back in the 1990’s.
Let Anthony Watts, et al. have the data, but know hey will do nothing useful or positive with it. If I’m not mistaken, data has been supplied before to such people then never used. After all, delay is the point, isn’t it? (<– Rhetorical question.)
Re: Tom
“I’m guessing that if someone had hacked into Bernie Madoff’s email account and discovered he was fudging the numbers, people wouldn’t be screaming, “Leave Bernie alone! You’re a thief, what you did is illegal, and your GOING TO JAIL!!!””
I would expect both of them to take responsibility for their actions. If what they did was illegal, they should accept the consequences. The ends do not justify the means
personally, I see nothing illegal in the emails or code. I based my opinion on the scientific evidence. Everything the scientists have said in the way of explanations is plausible and the science backs up what they say. Personality issues have no bearing on the science.
The Raven (109) — It is not yet clear the extent to which the e-mail messages were doctored or just made up…
Silke #89, parroting the usual lies:
Indeed the quotation marks are appropriate, as what they try to block, as proper part of their job as “peers”, is junk papers. You can find this out; the CRU stolen mails refer to several such. If you produce such junk, you deserve to be relegated to “journals” where peer review is a standing joke. Or to blog science. There are dozens of reputable journals to choose from. Claiming that they are all centrally controlled is classic conspiracy theory.
That day has already come, see the CRU stolen emails. Editors resigned because of it.
You mean: appeal to expertise. Denying the reality of expertise is shamefully anti-intellectual. You may not want to admit it, but science is hard. It takes effort to learn. You have to be good at it to get it right, most of the time. And when you don’t, you have your peers to tell you so. Learn to listen to them.
Debating is for lawyers, politicians and moncktons, not for those honestly interested in the truth. The proper process for that takes place on the pages of the peer reviewed literature, like it or not. Dozens of journals to choose from; their reputation hangs on their record of publishing science rather than junk.
What you are missing is that you need to look at the change in cloud cover that comes with a change in temperature. Not the absolute quantities. So much for common sense…
As for the rest of your falsehoods, I’m running out of time and stamina. Just to note that you’re rather overestimating the weight of general circulation models in the package of multiple independent lines of evidence on which our current understanding of the reality and seriousness of man-made climate change is based. And, from the all-too-much-transparency department, did you have to mention Al Gore?!?
David Benson (113) – Come on now, not even the participants in these exchanges are claiming that the emails are doctored or made up.
Forrest M. Mims III says:
28 November 2009 at 11:55 AM
Real Climate,
Many of the Hadley CRU e-mails and the infamous HARRY-READ_ME.txt include deeply disturbing content. Almost as troubling is the under-reaction of Real Climate as this scandal expands. This scandal deserves an immediate and fully transparent response from the climate science community, not the “circling of the wagons” approach cited by some…
**screed filled with blatantly false claims deleted**
Forrest
Forrest M. Mims III
Dear Forrest,
There is nothing in the e-mails that support your false characterizations, which I suspect is why you’ve received no answer. Why should they reply to propaganda?
Given there is nothing in the e-mails that undermines the climate science, let us put our time to better use. There is zero evidence, let alone proof, of an agenda, let alone a conspiracy, to promote AGW. There is, however, proof conclusive of the conspiracy to hide the science.
Please respond to the following with the outrage it deserves:
The American Denial of Global Warming
http://www.youtube.com/watch?v=2T4UF_Rmlio
ExxonMobil’s Tobacco-like Disinformation Campaign on Global Warming Science
http://www.ucsusa.org/global_warming/science_and_impacts/global_warming_contrarians/exxonmobil-report-smoke.html
Industry Ignored Its Scientists on Climate
http://www.nytimes.com/2009/04/24/science/earth/24deny.html?_r=2
The Denial Machine
http://video.google.com/videoplay?docid=522784499045867811&q=denial+machine#
In those links you will find evidence/proof of payment for distorted science, a memo that proves the anti-AGW lobby knew the science was solid – from their own scientists – back in 1995, an explanation showing how the cigarette denialists became the climate denialists (and if they are proven liars already, are supposed to trust them now?), and of how the government itself intentionally distorted the truth.
What you won’t find there, or anywhere else, is anything even remotely approaching a conspiracy to **support** AGW.
Please respond to the above evidence/proof of conspiracy by the anti-AGW crowd. That is far more settled, far better evidenced, and is actually proven.
Deal with that, if you’ve the ethical and moral fortitude.
Times Online has posted an article claiming that RC pulled their own postings:
“It was a powerful and controversial mix — far too powerful for some. Real Climate is a website designed for scientists who share Jones’s belief in man-made climate change. Within hours the file had been stripped from the site.
Several hours later, however, it reappeared — this time on an obscure Russian server. Soon it had been copied to a host of other servers, first in Saudi Arabia and Turkey and then Europe and America.”
http://www.timesonline.co.uk/tol/news/environment/article6936289.ece
[Response: The zip file was temporarily available (about 30 min) from a link to this site until we shut it down and then removed it. It was never posted as a blog entry. – gavin]
“Mark A. York,
Since you know so much, perhaps you could tell me why the oceans have not been warming after 2003 — or if they have why it’s a very slight warming contrary to the projections of the IPCC’s models.”
Sounds like an either/or fallacy. To my knowledge oceans are warming. That would be first time I’ve seen such a claim. Sounds like a clone of the warming stopped in 1998 offhand.
Can somebody recommend a good paper that gives a basic overview of the physics in the line-by-line radiation transfer codes? Turns out I’m writing a model (for an entirely field) that has some of the same features (absorption, scattering and emission at multiple wavelengths), and I want to make sure I’m doing it right. I assume Beer’s Law is at the heart of the absorption description?
Mark A. York,
No, it’s not an either-or fallacy. NPR had an article last year about global warming’s missing heat.
http://www.npr.org/templates/story/story.php?storyId=88520025
The Mystery of Global Warming’s Missing Heat
http://www.npr.org/templates/story/story.php?storyId=88520025
I think I forgot to give the link in my last comment.
You might also put up a link to the NEMO Ocean model. Both the source, documentation, version control and bug database is available with registration:
http://www.nemo-ocean.eu/
GFDL also has put up plenty of data (http://data1.gfdl.noaa.gov/) and with registration, bits of their model (http://www.gfdl.noaa.gov/fms).
For Mark York:
I just pasted his question into Google, and this is near the top of the first page of results:
http://global-warming.accuweather.com/2008/03/oceans_are_not_warming_1.html
Likely that (or icecap or one of the other you know what sites) is the source of that idea, which comes probably from Domingues (see below)
Accuweather points to the Argo homepage,
where there is a pointer to a nearly empty Argo blog that includes one very old topic on temperatures (Domingues paper, see below)
That also points to a main ARGO page that CSIRO has been hosting, which provides weekly updated T/S profiles for the global Argo array:
http://www.marine.csiro.au/~ttchen/argo/gmap.htm
From there you can look up each individual float and see its data set and a recent summary.
I’d imagine there’s more there to find; I gave it about four minutes’ worth of poking around just for the exercise.
See also:
https://www.realclimate.org/index.php/archives/2008/06/ocean-heat-content-revisions/
https://www.realclimate.org/index.php/archives/2009/07/more-bubkes/
As always (for any new reader here who hasn’t heard this):
I’m just another reader here, I know nothing about this except what an amateur can do to look things up. Nothing I post is more than a “here’s what I found” example. The real scientists can correct or improve on what I find. So can you, whoever you are, just take a few more minutes — read the papers, read the footnotes, look up the more recent papers.
The latter is key. The “oceans not warming” thing was probably from a paper more than a year ago, and — peer review is only the beginning. Look for discussion of it to see if the idea has held up.
Can you comment on the attached graph here http://i44.tinypic.com/29dwsj7.gif ?
This appears to have the US temperature anomaly as reported by GISS in 2000 and 2009, and a large difference between them. What is the reason for these differences and is the source code as used in 2000 available?
[Response: The differences are well documented in the Hansen et al (2001) paper and are due to needed corrections for various problems (such as changes in the timing of observations) provided by NOAA. – gavin]
A PS for Gavin, if there is (or will be) a FAQ on the tools in use and how data is collected, this is marvelous:
http://smart-machines.blogspot.com/2008/02/slocum-glider-green-energy-underwater.html
The “Slocum glider” design was first published as science fiction, here:
http://www.tos.org/oceanography/issues/issue_archive/issue_pdfs/2_1/2.1_stommel.pdf
(Someone in a thread here mentioned the author in the last few days — THANK YOU! I looked him up. Oh, wow. Science. It really does work wonders.)
Here’s one way (there are other ways) that they’re in use:
http://www2.sese.uwa.edu.au/~hollings/anfog/
Detail:
http://www2.sese.uwa.edu.au/~hollings/anfog/index.php?page=deployments
An example from an ARGO device page I picked at random from the huge map:
http://www.marine.csiro.au/remotesensing/oceancurrents/profiles/5901161/latest.html
In addition to what Hank posted, see this article from the NASA site:
“In mid-2008, however, a team of scientists led by Catia Domingues and John Church from Australia’s CSIRO, and Peter Gleckler, from Lawrence Livermore National Laboratory in California, revised long-term estimates of ocean warming based on the corrected XBT data. Since the revision, says Willis, the bumps in the graph have largely disappeared, which means the observations and the models are in much better agreement. “That makes everyone happier,” Willis says.”
(Note that this postdates the NPR story.)
From:
http://earthobservatory.nasa.gov/Features/OceanCooling/
Gavin,
your code (as found here http://data.giss.nasa.gov/gistemp/sources/GISTEMP_sources.tar.gz) is rather flaky and lacks basic documentation to make it usable.
I spent time reading through the code since executing it in any meaningful manner seems arduous. The reading of the code points to it being nothing special … most of it is data conversion and rudimentary data filtering, but outside of that there is nothing/(very little) that would make this code interesting.
Can you provide better documentation on what this code is supposed to do and how can one verify that the output (after running all steps) is valid?
For the sake of simplicity, I imagine you have this code working with some dataset … is it possible for you to publish the input parameters that you are using and also the resulting output?
Thank you.
[Response: I never said it was perfect. It is the working code that they use (and it isn’t my project just fyi). It isn’t that complicated because most of it is database stuff plus some corrections for rural/urban differences plus some interpolation. What were you expecting? If you want to know more about it, read the references. The input is the public files (like v2.mean from GHCN) and the output is on the GISTEMP website. There are intermediate results available in the ‘Stations’ pages. – gavin]
For the processed data sets, could you put links to the code used to process those data sets? It looks like you have the code in a different section, it might be nice to have those side-by-side.
Thanks again for the data!
Gavin,
Re 124, I have just had a scan through the Hansen reference and one line sticks out:
“Some prefatory comments about adjustments to the temperature records are in order. The aim of
adjustments is to make the temperature record more “homogeneous,” i.e., a record in which the temperature change
is due only to local weather and climate. However, caution is required in making adjustments, as it is possible to
make the long-term change less realistic even in the process of eliminating an unrealistic short-term discontinuity.
Indeed, if the objective is to obtain the best estimate of long-term change, it might be argued that in the absence of
metadata defining all changes, it is better not to adjust discontinuities.”
So it would seem the pre 2001 version is a better reference for long term trends?
[Response: Only if you didn’t have any relevant metadata. If you know that people used to take mesurements a 5am, and then they change to noon, that needs to be corrected for. And that’s what done. – gavin]
Gavin, I wish you hadn’t told me not to look up the (other) definition of wip-wap. On the other hand, this gives me an idea: maybe you should tell the denialati not to read a good textbook on climate physics, and see if it has the same effect.
Gavin,
it is true that you didn’t say “it [the code] was perfect”. It also true that from computer science this code is very, very far from perfect. Rudimentary error handling is not in place. Someone has to copy the “temp results” folder from one step to the next. There is no way to ensure/validate if this “temp results” folder contains meaningful data.
I’m sure that someone is using this software for something … but apart from the simple utility that can be provided by this code, I can’t see how this code comes anywhere near “computer modeling” software. It is, at beast, translation utility.
What I was expecting is to see (or at least have a peek at) the modeling software, and the heuristic prowess contained therein.
BTW, I did read the references, within the source code package, and there isn’t much to talk about … I don’t know how using the software I can arrive at conclusion that, for example, “Earth is overheating based on human activities that produce C02” — as far as I’m concerned this rise in temperature might very well be true, precisely because of CO2. However the software should be at least able to model, unambiguously, the fact that Earth is warming up (regardless of cause), but it doesn’t seem that this software can be used to show even that. So again, this software seems to be just a conversion utility and nothing more. (?!)
BTW,
I realize that this project is not YOURS. I also understand that even if it was yours you would rather not admit it. ;-) (I wouldn’t admit it either.)
[Response: Who said this was computer modelling? Go to computer models for that! -gavin]
For those who have a particular interest in the GISTEMP code base, our Clear Climate Code project is in the process of re-implementing GISTEMP, with a particular emphasis on clarity. We’re using Python, chosen specifically as a language in which it is easy to be clear. There are various stages to the reimplementation, starting with running the original GISTEMP code, then ripping out the ksh dependency, then step-by-step reimplementing in Python (this part is mostly done now). Once we have an all-Python implementation producing the same results, we’re going to look at further improvements and clarifications (for instance, replacing the rather cumbersome file-to-file staging with faster and clearer iterator pipelining). It’s taking a long time because the people involved all have day jobs, families, etc.
The process has revealed some bugs in GISTEMP, all of which have been reported to GISS and none of which make any material difference to the results (we think we’ve recently found some similarly inconsequential problems in STEP5, which we will be communicating to GISS as soon as we’ve convinced ourselves).
The code in in Google Code, at
The discussion mailing list is at
We are reworking the website, so there’s nothing to see there right now.
Argh, link breakage, sorry. Google for ccc-gistemp or ccc-gistemp-discuss to find our code/resources/mailing list.
Benjamin (115) — It seems that some have rather suspicious time zones attached and some are strangely truncated; too many chare the same uid and gid. I reiterate that nobody has authenticated all of the e-mail messages wihtout asserting a claim that all or even any were fabricated, just that some doubts about complete authenticity have arisen.
Slightly related: I just sent an email to CNN about their coverage of Copenhagen tonight… instead of getting moderate people to come on and answer the questions Malveax was asking (what do you think Obama can accomplish, etc), they had Carville and of all people, Ben Stein, who naturally sat there editorializing with disinformation, including the insistence that data is being withheld due to malfeasance. I don’t know if you can watch the clip online or not, but anyone who has seen it, or the presumable re-run of it later this evening, please email CNN and let them know how appalling their coverage of CC has been and continues to be. They didn’t need to have a complete ideologue on this issue, especially when they are supposedly moving away from that. If they wanted both sides, they could have picked Lindsey Graham or someone else with a moderate approach. According to their own poll, shown in that segment, 51% of Republicans believe it’s happening, so don’t tell me they couldn’t have found anyone. CNN has consistently failed on this issue, and I couldn’t be much angrier right now! Their entire staff is completely uneducated in this area, while it’s a huge part of our reality and of news. Here is the link for the comment form for the Situation Room, so if you’ve seen it and are disgusted, please let them know! http://www.cnn.com/feedback/forms/form5.html?65
“Silke” (#89) appears to have lifted text from Roy Spencer — see http://www.drroyspencer.com/, “My Top 10 Annoyances…”. Marco #108 and Martin Vermeer #114 don’t seem to read widely on these issues and appear to be responding to a troll. :-)
Alan of Oz #50
Sure it’s nice to have the code but the code itself is irrelevant to the science.
I can’t believe you posted that. The source code for developing models is irrelevent? Why even have code then? I’m puzzled by where you were going with that.
Software configuration practices and OSS are not particularly usefull for reproducing results in the scientific sense since the same code will always produce the same results and you will be stuck with the impossible task of demonstrating the code is bug free.
There’s nothing stopping people from having different projects that examine the same data. In fact that’s what has been done in linux development — the scheduler for example was done in two different ways.
Reproducing the same results with the same data and different code is a much more powerfull test because it provides confidence that the results are independent of a particular implementation of the methods.
That’s is 100% correct and is why climate models / code needs to be open and freely available. Without open access to other’s codes how can one find out why the results of two programs differ? How can others expect to become better without looking at each other’s work? And finally how can one find out what’s going on in a code based without a history (in checkins and comments) of the code?
I find it more than a little ironic that climate scientists have to educate programmers on the benifits of multiple independent source trees. It’s not a hard concept to grasp, double entry bookeeping works on the same principle and even an accountant can do that ;)
Why does open source mean one repository only? The open source world is full of forks to projects that have either gone dead or not the way people wanted them to go.
Re:76 Holly Stick, 33 Garrett. I saw that as well. During summer even now the channels are effectively open, not even sure you would need an icebreaker, just vessels with a reinforced bow should be ok.
I think Big Oil can see the bginning of the end…the global accord and spirit of cooperation growing in leuw of Copenhagen is very impressive exept for the US who again is being led rather than taking a leading position.
Does anyone know what happening to the polar bears in the arctic region..I haven’t heard about them for quite a while.
DonQ, it’s research code, rather than a published widely-used application. It’s the software equivalent of a lab setup. To understand it, you have to read the docs, probably read the code, go to the published papers, and perhaps spend time as a student.
In another field, I’ve done some of the work of making research code into production code. It’s hugely time- consuming. Writing documentation takes at least as long as writing research code and isn’t nearly as much fun. Developing user interfaces also takes much time, and the skills required are usually different than those of developing research code. Perhaps worthwhile work but so far GISS hasn’t taken it one.
The Raven says “…we have no way of judging the honesty of the criminals who broke in…”
OK, I want to make sure everybody sees this. I damn near fell out of my chair. By all means, we want to make sure we are dealing with honest thieves…
Hey thanks Hank. Why is he asking me is my question? I mean I did a herring cruise off the coast of Maine once but that isn’t my specialty if I have one at all. When on couches a question like that with “if you know so much” then…it’s hard to take it seriously. It sounded like a contrarian talking point to me. Seems like it is. They don’t do nuance well.
under the listings for RAW data is the following:
Climate data (raw)
GHCN v.2 (Global Historical Climate Network: weather station records from around the world, temperature and precipitation)
When you look at the readme for that file it is:
The purpose of this file is to provide a description of and the format for the
following USHCN data files:
hcn_doe_max_data.Z Areal Edited, Time of Observation, and Filnet Adjusted
Maximum Monthly Temperature
hcn_calc_mean_data.Z Time of Observation and Filnet Adjusted Mean Monthly
Temperature (Calculated from hcn_doe_max_data.Z and hcn_doe_min_data.Z)
hcn_doe_mean_data.Z Areal Edited, Time of Observation, and Filnet Adjusted
Mean Monthly Temperature
hcn_doe_min_data.Z Areal Edited, Time of Observation, and Filnet Adjusted
Minimum Monthly Temperature
hcn_doe_pcp_data.Z Areal Edited, Time of Observation, and Filnet Adjusted
Monthly Precipitation
station.history.Z Station History
station.inventory.Z Station Inventory
station.landuse.Z Station Area Land Use and Land Cover
urban_max_fahr.Z Urban Heat Adjusted Maximum Monthly Temperature
urban_calc_mean_fahr.Z Urban Heat Adjusted Mean Monthly Temperature
(Calculated from urban.max.Z and urban.min.Z)
urban_mean_fahr.Z Urban Heat Adjusted Mean Monthly Temperature
urban_min_fahr.Z Urban Heat Adjusted Minimum Monthly Temperature
Not exactly raw data!
Gee, 2 weeks of denialists posting and so far not one has actually taken issue with or even mentioned the evidence. Hmm. Why would that be?
Post 116, in response to post 76:
>Dear Forrest…Deal with that, if you’ve the ethical and moral fortitude.
Ok, sorry but I call foul. By his linked website, Forrest is somewhat right leaning, and perhaps “on the other side” – but he’s *also* an admirable science geek. When in doubt, be the one toning it down. Straight rational argument works best when dealing with non-dittoheads (of course if they don’t respond with same, then disengage or mock…)
As I posted above, there is a larger fight in society against anti-science within which the smarter skeptics on AGW are our *allies*. Many are clearly strong on basic enlightenment values and the scientific method. Many are science geeks. They are in fact even in the best position to tone down the crazy of the real denialists through basic education about the mechanisms involved.
If there is a lesson in this mess, it is neither in the interests of the cause of reducing CO2 emmissions, nor the greater cause of upholding science itself, to assume all skeptics are Denialists. If someone starts throwing accusations around, even if it’s 99% lies, don’t let your ego stop you from improving your own behavior as a response to the criticism. As per the original the subject of this thread.
Stage 1: If you don’t make all the data and model code available, I refuse to believe you.
Stage 2: If you won’t teach me how to analyze the data and run the models, I refuse to believe you.
Stage 3?: This stuff is too difficult — it can’t be true. (“All I really need to know I learned in kindergarten.” — Robert Fulghum)
>your code (as found here http://data.giss.nasa.gov/gistemp/sources/GISTEMP_sources.tar.gz) is rather flaky and lacks basic documentation to make it usable.
It seems to me that this comment goes right to the heart of the issue. Documentation of raw data takes time (and staff) away from other projects. Even writing email in a context free style (for public consumption) takes a lot of effort. If only there was a way for scientific staff to devote resources to getting certain material in publishable shape, while keeping other materials under wraps!
” Barton Paul Levenson says:
Read my lips, TOA. Hacking into the CRU computers without CRU’s permissions was ILLEGAL. AGAINST THE LAW. A CRIME. If they can find out who did it, those people are going to jail. That’s what “illegal” MEANS.”
It is not a CRIME if it is shown that the people who did the deed can show that the public interest in disclosure of the information outweighs the public interest in non-disclosure. A public interest defense is a defense which allows a defendant who disclosed classified or protected information to avoid criminality, if (s)he can establish that the public interest in disclosure of the information outweighs the public interest in non-disclosure. This defense is part of UK law and is specifically aimed at protecting whistleblowers of governmental misconduct.
http://freedom.libdems.org.uk/the-freedom-bill/13-public-interest-defence-for-whistleblowers/
Now you do not know how 12 members of a British jury will act, when presented with such a defense, any more than you know about the amount of snow there will be at the North Pole in 20 years time; but the lack of knowledge has not stopped you talking out of your arse on both subjects.
I’m waiting for “open sourced” emails from American Enterprise Institute, the Heritage Foundation, the Cato Institute, Competitive Enterprise Institutes, the Reason Foundation, et al
After all, what’s good for the goose. . .
“This is a powerful meme and one that has clear resonance far beyond the people who are actually interested in analysing data themselves.”
That statement is arrogant and condescending and is contrary to the spirit of free inquiry. At pity indeed that realclimate.org will reject this post. Shame on you!
“I’m no scientist and I have no understanding about the numbers. But I can tell when someone is BSing me.”
So if you can’t do the numbers, what do you use instead – Tarot cards? flipping a coin? Rolling dice? Trust Glenn Beck or Rush Limbaugh tell you who’s BSing? Maybe you find an aristocratic British accent like Lord Monckton’s convincing? Or perhaps the erudite, sophisticated, content free vocabulary of George Will tells you what you want to know?
If you can’t tell why someone wants to run a 40 year Fourier convolution filter on one set of temperature data, but draws straight lines through two year peaks of Arctic ice data in order to claim warming isn’t happening, you might as well use some of the “proxies” suggested above. I would recommend flipping a coin, as this would give you the best (50%) odds of actually being right.