Traducido por Angela Carosio
Nature Geoscience tiene dos comentarios este mes sobre ciencia y blogging, uno mío y otro de Myles Allen (véase también estos correos de blogs sobre el tema aqui y aqui).
Mi comentario trata de establecer una proposición que es que la gran mayoría de lo que los científicos conocen es “tácito” (por ejemplo, no explícito o escrito en la literatura técnica) y es ese conocimiento que les permite distinguir rápidamente, con precisión razonable, que artículos nuevos valen la pena leer en detalle y cuáles no. Este contexto es lo que provee a RC, y a otros sitios científicos, la seguridad para comentar sobre nuevos artículos científicos y sobre la cobertura que éstos reciben en los medios.
El comentario de Myles destaca que la crítica de artículos y la reseña de pares debe permanecer en la literatura de reseña de pares y que la crítica informal, como la que se encuentra en un blog, podría desmerecer el artículo.
Estamos de acuerdo con que hay una gran diferencia entre rápidamente indicar problemas obvios en un artículo publicado (como el artículo de Douglass et al de diciembre pasado) y hacer un trabajo sustancial y un análisis extra que merece una respuesta reseñada por pares. Los enfoques no son necesariamente opuestos (por ejemplo, nuestra respuesta al artículo de Schwartz el año pasado, que dio lugar a un comentario referido). Pero dada la limitación de tiempo de todos, y la limitación de espacio del periodismo, se presentan y se publican muchas menos refutaciones comparando con la cantidad de quejas. Es más, es extremadamente raro que se escriba un comentario formal sobre un artículo particularmente excepcional, dando como resultado que las quejas son mucho más comunes que los aplausos en la reseña de pares. De hecho, hay mucho que aplaudirle a la ciencia moderna, y nos gustaría pensar que RC juega un rol positivo en resaltar algunos de los resultados más importantes y apasionantes que aparecen.
El comentario de Myles, si bien termina con un interesante punto de discusión, es ilustrado, en mi opinión, con un ejemplo fuera de lugar que involucra a RC, un correo y el posterior seguimiento de un artículo de Stainforth et al (2005) y la gran cobertura que éste tuvo en los medios. El correo original aborda el tema sobre como los modelos ejecutados en el sitio climateprediction.net han afectado nuestra expectativa actual sobre que es la sensibilidad climática y si se justifica una revisión de cualquier proyección a futuro. El segundo correo vino luego de un artículo de periodismo, un tanto inferior, en Radio 4 de la BBC que implicaba, injustificadamente, que el público estaba siendo engañado deliberadamente por el equipo CPDN (Climate Prediction Dot Net, por sus siglas en ingles) sobre la importancia de su trabajo. Hemos discutido entonces, así como le hemos hecho en otras ocasiones, si parte de la responsabilidad de artículos inexactos o exaltados publicados pertenece a la misma prensa o si nosotros como comunidad, podríamos hacer un esfuerzo por proveer un contexto en dichos casos. La razón por la cual esto no es pertinente a lo que Myles opinó es que no hemos criticado en absoluto el artículo en sí. En ese entonces pensábamos, y seguimos pensándolo ahora, que el esfuerzo del equipo CPDN valió la pena y que cualquier lección aprendida será informada a los modelos de simulación en un futuro. Nuestra crítica, tal como era, fue asociada en vez con la percepción del artículo en parte de los medios y en la comunidad en general. Estos medios no son para nada adecuados para comentar una reseña de pares.
Este no es el lugar adecuado para argüir el tema de la sensibilidad climática (prometo un nuevo correo respecto a eso pronto), de modo que se considerará fuera de tema. Sin embargo, estamos muy interesados en comentarios sobre el tema fundamental planteado: como se interceptan, o como debieran interceptarse los blogs sobre ciencia y las tradicionales reseñas de pares, y si la percepción de Myles que estas están en conflicto es ampliamente compartida.
You are raising an interesting problem, which seems to me similar to the general question if scientists should speak up about an issue in any other way but through peer-reviewed literature. I often here the argument that scientists should be careful not to lose credibility; and the only way not to lose it would be by publishing research in peer-reviewed journals.
However, I strongly believe that scientists have to reach out a lot more to the general public, so that non-scientists learn to better understand the scientific process, learn to better weigh the inherent uncertainty in all areas of science, and learn to know the issues at stake. By reading peer-reviewed articles it is not possible to follow the process of thinking and critique, whereas a blog like RC is an ideal way not only for scientists to discuss important issues, but also for the non-scientists to follow the thinking process and arguments of scientists.
I don’t think there is any conflict between peer-review and science blogs, but they wonderfully complement each other – one to present new results to the scientific community, the other to discuss the results with anybody interested. Thank you for doing that so well on your blog!
So many journals these days publish dubious or flawed or merely unclear analyses that quickly get picked up and respun lamely by the traditional media like a bad game of telephone.
So I think we can’t wait for a journal to publish some short, edited letter weeks or months later. Exhibit A, by the way, is Pielke et al.’s very misleading Commentary in Nature — is that stuff even peer-reviewed? Who knows? Anyway, it needed fast debunking.
http://climateprogress.org/2008/04/02/nature-pielke-pointless-misleading-embarrassing-ipcc-technology/
As did Nature’s unbelievable quoting of a colleague of Pielke’s (Marty Hoffert) on behalf of the paper in an accompanying new article without actually identifying them as colleagues!
http://climateprogress.org/2008/04/02/shame-on-nature-for-quoting-hoffert-on-behalf-of-pielke-without-noting-theyre-colleagues/
Long live the blogosphere!
With all due respect to the good Dr. Allen, his argument is pants.
The discussion of his paper will go on in a wide variety of forums, whether he likes it or not. The question is whether that discussion should or should not be informed by the contributions of actual scientists who understand it.
The answer seems obvious.
Blogs don’t serve very well for communication among scientists. Peer review does more than just protect us from being inundated with substandard work; it protects authors from their own mistakes and improves the quality of what we write. Peer review itself is an immensely valuable avenue of communication; who among us hasn’t at some time included a phrase like “We thank an anonymous referee for comments and suggestions which dramatically improved the final manuscript”?
But as bad as blogs are for actual research, peer-reviewed journals are far worse for communicating with and educating the lay reader. Yet when it comes to climate science the lay public is hungry for knowledge, and many of them are eager, and well-prepared, for a level of sophistication and detail that can’t be found in lay journalism or even popular literature; An Inconvenient Truth isn’t enough. So blogs serve an incredibly useful purpose, enabling the interested and well-educated reader to share insights with researchers who are at the cutting edge of new knowledge.
There’s yet another aspect which features prominently when it comes to climate science. We’re in a “propaganda war” in which one of the strategies used by the forces of ignorance and greed to sabotage action, is to spread fear, uncertainty, and doubt. We’re not just scientists (climate or otherwise), we’re also human beings with a moral obligation to leave the next generation a world worth inheriting. The blogosphere has been a primary target of the disinformation campaign; it’s the “trenches” of the propoganda war. We have to fight the enemy on this crucial battleground.
One of their strategies is to take legitimate peer-reviewed work out of context, blow it all out of proportion, or misrepresent it most falsely, to give the impression that it overthrows the reality of global warming when in fact such was never the intent or conclusion of the authors. The Schwartz paper is a fine example of this; it doesn’t overthrow any aspect of climate science — nor does it pretend to — but it was heralded by so many as doing exactly that. In a sense, Schwartz himself was the most direct and most damaged victim of their efforts. Refuting such nonsense can’t be done in the peer-reviewed literature; no respectable journal publishes that kind of stuff and no substantial fraction of the general public would ever see it.
So, while blogs aren’t part of the machinery for legitimate scientific research, they’re an indispensible tool for communication and combating misinformation. RealClimate is the best of the best; keep up the good work.
Dear Gavin,
You may feel you didn’t criticize the Stainforth et al paper, but you did misunderstand it in one crucial respect, in that you said explicitly that our “the most important result … is that by far most of the models had climate sensitivities between 2ºC and 4ºC, giving additional support to the widely accepted range.” I didn’t think it mattered at the time, but it turned out it definitely did matter, because this was the first thing those BBC journalists threw at me, and I continue to meet journalists and even scientists who are convinced that your analysis was in fact our “true” result, and we only drew attention to the fat tail extending out to much higher sensitivities because we wanted to alarm people. As you know (and noted in your later post, thanks), the cluster around 3 degrees wasn’t a result at all, but a simple artifact of the experimental design, so the fat tail was indeed the main noteworthy result of the experiment.
It would have been really easy to have picked up on this had you e-mailed me a draft of your post before posting it on the internet (and published our response along with your post if necessary). It’s all water under the bridge now, but this whole exchange does demonstrate the serious collateral damage that can be caused by relatively minor mistakes. Now that RealClimate has proved such a success and is so heavily used by journalists for source material, perhaps it is time to tighten up your procedures a little. I personally would never comment critically in public on a peer-reviewed paper even to point out “obvious problems” (who is the judge of what is obvious here?) without at least exchanging e-mails with the authors to make sure I had understood it correctly (I’m more than happy to criticize non-peer-reviewed material on Channel 4).
While I’m posting (I can see how you guys get into this) I’m also very uncomfortable with your notion of “tacit knowledge:” it certainly seems to be tacit knowledge in the blogosphere that the chances of the climate sensitivity (equilibrium warming on indefinite stabilization at 560ppm CO2, for the non-enthusiasts) being greater than or equal to 6 degrees are too small to be worth worrying about (meaning down at the level of an asteroid strike). If we accept this tacit knowledge, then your original post and the fuss over the press coverage of Stainforth et al make a lot more sense: we would have had no business drawing attention to these high-sensitivity models if we were as confident as you all seem to be that the real world sensitivity just cannot be that high. But (although of course I sincerely hope you’re all right) I just don’t see the evidence for this level of certainty in the peer-reviewed literature. In an environment without peer-review, it seems to me to be much easier for such “everyone knows X” myths to develop.
I appreciate that publish-first-and-ask-questions-later is “traditional” practice in blogging, but perhaps, as scientists, we should be challenging that practice. After all, if the New York Times can pass articles through a simple fact-check procedure before publishing them, why can’t RealClimate?
[Response: Dear Myles, thanks for your thoughts. I was only able to join this discussion late, so you find my response down at comment #116. -stefan]
I remember a boss telling us all that he wanted to hear what we thought of his style of management but we rapidly discovered he didn’t want to hear complaints. I suspect this plea is in the same vein. Your treatment of Douglass et al. was far from fair or even correct as Christie would have told you if he had been contacted first or been given right of reply. He did list your errors elsewhere but you didn’t acknowledge that either. And that is where you go wrong. Pointing out that obs and theory don’t match is what real science is about. It should actually be done as part of the normal validation procedure of models. At least though you brought out the truth about the real spread of uncertainties in the models, which had been cunningly disguised by the IPCC reports. I guess it was the only argument you had, but it was quite funny to hear that model results are acceptable if their huge error bars manage to clip the observation error bars. How scientific is that?
As far blaming journalists for parroting the press release given to them by a University. What else would you expect? Surely the scientists see and approve the press release so the fault lies squarely with them if it is grossly misleading.
Yes Tamino, it is about propaganda but the fear uncertainty and doubt is being spread by both sides. Some of you just don’t see it because you feel innately so much more virtuous than the other side. Bending the truth is then recast as countering the arguments of the evil doers. But it’s still bending the truth and it diminishes you.
[Response: Sorry, but you are just wrong. The error in the Douglass et al paper is clear and obvious and does not require a much thought to discern. It is simply that the statistical test they apply would reject ~80% of samples drawn from an exactly similar distribution. To make it clearer, take a fair die – the mean number of points is 3.5 and with around 100 throws, the mean will be known to within 0.1 or so. Then take the same die and imagine you get a 2, then the Douglass et al test would claim that this throw doesn’t match since it is below 3.3 (3.5 – 2 standard deviations). This is absurd. How checking with John Christy would have changed the situation is unclear. Matching data and theory is indeed the main activity of science, but it needs to be done correctly. – gavin]
RE “most of what scientists know is ‘tacit'” — I was just in a social science research meeting and someone brought up the problem of “if it isn’t quantifiable, it doesn’t exist”; she used ethics and other issues as examples. I brought up some issues in climate science, like the possible disintegration of the Greenland ice sheet, and how it wasn’t included in a calculation of sea rise in the IPCC assessment, since it wasn’t quantifiable in terms of timing, since it is a non-linear type thing. And there are many other aspects of climate science I gather that are not easily quantifiable, or for which there is no neat formula.
I think climate scientists know about these, but can’t really make definitive claims, so they don’t get into peer-reviewed articles much…or else other scientists might attack them with ferocity (even substracting denialists from the equation here).
It is that tacit knowledge that is so important, and needs to be used on blogs such as this, to let us know what makes sense at an expert level. Laypersons can proffer all sorts of weird notions that come to their minds, and scientists with their valuable (tacit and explicit) knowledge can put limits on it (what’s possible, impossible, not likely, likely, etc). And we need their assessment of the peer-reviewed articles and IPCC assessment, as well.
I’ve thought a lot about this since launching my blog a couple years ago. Blogs have created a forum for many people to informally discuss science. They are also a great forum for scientists to provide context to their own work and to the work of others in their field.
And some of the time, that may take the form of criticism. That’s ok. But we do have to be careful in blurring the line too much between peer-reviewed publications and blogs. The first problem is the obvious one. Blog posts are unfiltered, un-reviewed, and often written off the cuff, while journal articles are screened, reviewed, and (should be) meticulously researched. It is far easier to write a criticism of a paper on one’s blog than to write a response and submit it to the journal. The issue isn’t just that no reviewers check the work… the blog author is unlikely to do even close to the amount of research and analysis nor give the wording nearly the same level of consideration as is expected in a paper submitted to a high quality peer reviewed journal.
The other problem is that blog posts are readily accessible to anyone at anytime, both in language, and in the unlicensed nature of the internet. To a huge swath of the public, blog posts are the face of science. Like it or not, bloggers with scientific credentials are like self-appointed ambassadors for science. If we are going to write about our science, we should do it with thought, and we should do it well. That is a standard that myself and many other science bloggers often struggle to meet.
Tacit knowledge is an inescapable reality in all endeavours of skill (eg masonry), as far as I can tell, so I see no particular problem with Gavin being explicit about it. I can read any peer-reviewed article I like on modern climate models, but until I go through much of the process of building, running, validation, discussing with colleagues how they solved particular wrinkles etc of some models, I am unlikely to fully comprehend climate modelling as a skilled craft. That is where the “tacit knowledge” resides, in the main; knowing where the bodies are buried, to mal-appropriate an expression.
As for science blogs such as this one, I think on the whole they are invaluable. Having an interactive forum as opposed to a passive one such as a book or TV documentary is great. As far as I can see, so long as criticisms of scientific articles are explained on a factual basis, the readers can decide if the criticism seems warranted or not.
Perhaps the inevitable mistakes and spoke-too-soon cases could be dealt with by adding a “Corrections/Elaborations” section to the site.
Don.
Major areas where tacit knowledge plays an important role, is that people in a field know a) what didn’t work, although at first glance it should have or might have, b) what was published and is worth ignoring and c)the small tricks that make a lot of things work. Don’s mention of masonry is exactly correct. You don’t find that stuff in books, you do find it in discussions around the coffee pot and that is the chief value of graduate school. As the bunny put it long ago
Interesting thread. This sounds like the kind of discussion related to the Lockwood and Frohlich (2007) (L&F) paper on solar variation. The L&F paper was peer-reviewed, but two criticisms I found were published on websites (their analyses were not vetted).
The L&F paper is here:
http://publishing.royalsociety.org/media/proceedings_a/rspa20071880.pdf
The critiques are here:
http://members.shaw.ca/sch25/FOS/Lockwood/Gregory-CritiqueLockwood.pdf (Beware. “Friends” of Science did this one.)
http://www.spacecenter.dk/publications/scientific-report-series/Scient_No._3.pdf (By Svensmark and Friis-Christensen, whose research is solely on the cosmic ray theory.)
One of major benefits of a science blog like RealClimate, which is not replicated in formal comments submitted through the peer reviewed literature, is that it allows hundreds of people around the world to participate in an online discussion about issues of common interest. This provides a great vehicle for active learning and disseminating knowledge in both the scientific and general community. If there is an error in the original post, such as Myles (#5) suggests, or one of the subsequent comments, this can be ironed out through the discussion.
Thanks Gavin,
Myles’ paper lies behind a wall, although a free one in this case, which pretty much sums up the issue for a passionately interested observer such as myself. Much scientific literature is not accessible to non-experts (including other scientists), either for this reason or because they lack the necessary ‘tacit’ knowledge to read it critically.
Without blogs such as RealClimate I would not know how to recognise and counter denialist baloney. I would also not fully appreciate many of the implications of particular climate research, including the socio-political implications. And I wouldn’t have much insight into how the science is done and how solid the conclusions are. My views are very much in agreement with Tamino in post 4. Blogs are a vital and necessary part of opening the ‘universe’ of climate science to a broader non-expert audience.
Ultimately science blogs may make a fundamental difference in our response to climate change – as they help create the critical democratic mass to counter the ‘corporatocratic’ tendency to business as usual present in most developed countries.
PS – I struggled to find a better word than Corporatocracy. For a definition see:
http://en.wikipedia.org/wiki/Corporatocracy
One of Myles complaints in his piece is that his response got deleted from a blog, so the criticism was allowed to stand unchallenged and perhaps by inference, supreme. Yes, blogs are often more censored and less open than they appear.
[Response: You are misreading his comment. Myles did not ever publicly post his reply, while the comment was posted by it’s author. No deletion of anything occurred. – gavin]
But should serious criticism have to wade through peer review, in order to be taken seriously? Many serious criticisms are simplistic, such as important results that were not cited and discussed in the paper. If scientist X publishes a new generation model, and I want to point out that several issues published as widespread in the previous generation of models were not discussed in the paper, so we are left ignorant of whether those issues are resolved or addressed in the new model, will that be deemed as worthy of being published as a response in esteemed journals?
More likely, I would first inquire of the authors directly. Even if I found that the issues were not addressed and thus were still likely present in the new model, the absense of the references might be assumed to be evidence that the problems were not addressed. But is such an “assumption” publishable? Is a new model entitled to be a complete unknown except for the information published in the article, and thus not subject to all the limitations of previously published model results?
Yet, the peer review process is often assumed to be an attempt to guarantee that peers familiar with the literature would not have let the paper through, if it didn’t address known issues. Thus, the model is allowed to stand uncriticised in the peer review literature, as the apparent new pinnicle of model achievement. After all, the published criticisms don’t apply, since the model paper was publshed after the diagnostic studies of the previous generation.
Perhaps blogs and peer review should be combined, and perhaps the best way is with moderated blogs appended to the electronic version of the paper. That way potentially serious issues, that don’t rise to the level peer review publication themselves, can be raised and hopefully addressed.
Peer reviewed papers are the top of the pyramid of information and knowledge about climate change. But there are very few lay-folk who are able to truly validate let alone understand and usefully interpret the methodology, the equipment, the models, the inputs or the outputs, let alone the see the full scope of the social, political and economic implications of these heady documents.
So all we can really do is watch as the real scientists gently debate the worth of each new revelation and eventually accept that which they in the end conclude is good, pragmatic and generally safe to hang our hats on.
It is within these blogs that we discover how you the true scientists feel about the import of the most recent revelations, and it is upon that consensus that we tend to rely.
The ‘contributions’ to these bolgs by we non-scientists are the only way we have of exploring and eliminating our own uncertainties, and we are most grateful to RC for that opportunity. We hope we do not abuse it!
Please keep on doing what youre doing. Thanks
If criticism of papers and projects should be kept to the peer reviewed literature, perhaps promotion of said papers/projects should be also? Myles Allen might dislike BBC Radio 4’s implication that CPDN was deliberately misleading, but he was a regular sight on British television favourably discussing the project and encouraging people to get involved (for obvious reasons).
This is special pleading. If a project is going to be discussed in the public arena, then criticism (knowledgeable or otherwise) must be part of that discussion. It it were to be removed, then the public understanding of both the science and the issues would likely be even more distorted, having ceded most of the battlefield to unscientific projects and dubious criticism.
Myles Allen’s comments smack to much of “leave science to the scientists” for me. Science will be discussed by the public, and many will want to know other scientists’ views on particular topics; to say they must be at a level to understand the peer-reviewed literature to do so is elitist. Yes, talking to the public forum as RC, Pharyngula, Neurophilosophy and other science blogs do can be prone to errors or misrepresentations. But the answer is not to shut down the blogs, it is to point out the errors and misrepresentations.
Don hits the nail on the head. He describes climate modeling as a ‘skilled craft’. That is what it is. It is a craft not science which must arrived at by generating and testing hypotheses in empirical studies. A good recent description of computer models is that they are the mathematical representation of the modelers opinion. That is not science people.
Tacit knowledge is not necessarily accurate knowledge. The two may or may not be the same. Because you have tacit knowledge, it does not mean you are right or even knowledgeable. I has been shown in numerous studies that when making prediction, expert opinion is no more accurate than non-expert opinion. By Gavin saying that the tacit knowledge he has gives him the ability to sort the important papers from the frass, he is really saying that he is sorting his preferred papers from his non-preferred papers. i.e. the papers biased to his leaning. This is not the same as saying that he can sort worthy from unworthy papers.
Tacit knowledge should never ever, ever be regarded as truth until it has withstood empirical investigation.
[Response: Your idea about what science is, and is not, is deeply flawed. Modeling (of all sorts) is fundamental to science because it is only through modelling that the quantitative implications of basic theory can be found. All predictions (whether in a laboratory or natural setting) are based on such models and it is only from comparing predictions to observations that one progresses. – gavin]
Interesting. Also in the light of an extensive debate on the Nature website about peer review.
http://www.nature.com/nature/peerreview/debate/index.html
It seems that one should not overstate the value of peer review. It has its value and it has its weak spots.
The one thing I am convinced of though is that science progresses by debate. Anyone who suggests otherwise simply does not understand the relevance of the scientific debate [think about Al Gore and claims that the debate is over and that anyone who disagrees too much with him apparently is a fraud or payed by oil companies or whatever …].
You can differ in opinion about what the best format is for this debate to take place, but science should be about facts, observations, experiements, arguments, interpretations and discussing them. This can be sometimes a hard, long, difficult process, but science is not definitely not decided by committees or majority votes. As some politicians want us to believe.
Scientists should embrace the open scientific debate, and anyone who challenges that should be made very, very clear that without open debate, there simply is no science, no matter how much one is in favor of or opposes to particular people, statements and actions.
Jenne
I produced the Radio 4 programme you described as “rather scurrilous”. Unsuprisingly I think this is unfair and I am very proud of the programme. I invite people to make their own minds up. The programme is what climate change journalism should be it is challenging and rigourous. It does not feature any sceptics to falsely balance the debate and allows people we criticise to defend themselves.
The page below has the programme available in full in the top right-hand corner.
http://news.bbc.co.uk/1/hi/magazine/4923504.stm
Many scientists have contacted me privately to commend the programme but I will mention a couple of people close to CPDN who have written about the programme and the press release.
Tim Palmer, the head of the Probability and Seasonal Forecasting Division at the European Centre for Medium-Range Weather Forecasts wrote the following in Physics World:
” A recent well-researched BBC radio programme exposed a number of exaggerated press releases by climate institutes.”
Bryan Lawrence of NERC who fund CPDN said on his blog of the infamous press release:
“I was staggered to read the actual press release that caused all the fuss (predictions of 11C climate sensitivity etc). The bottom line is that had I read that press release without any prior knowledge I too might have believed that an 11 degree increase in global mean temperature was what they had predicted (which is not what they said in the paper). I can’t help putting some of the blame back on the ClimatePrediction.net team – the press release didn’t reflect the message of their results at all properly, and they shouldn’t have let that happen. I’m still naive enough to believe it’s incumbent on us as scientists to at least make sure the release is accurate, even if we can’t affect the resulting reporting.”
I can imagine that this is exactly the kind of blog comment that needs stopping.
In the original programme we also touch on frogs and the huge amounts of coverage giving to the frogs killed by climate change story. The brilliant Andrew Revkin wrote about this recently:
http://dotearth.blogs.nytimes.com/2008/03/24/vanishing-frogs-climate-and-the-front-page/
It looks like we were right in our approach to the story. It was a story I couldn’t have hoped to have covered without blogs. They helped me realise there was a story but only when I had spoken to many peer reviewed scientists did we broadcast.
[Response: I have edited out the term ‘scurrilous’, since that was perhaps a little strong. However the implication that the CPDN scientists were trying to deliberately mislead the public is unfounded. That implication, nor the accusation that they were being alarmist, did not arise from our original blog posting. – gavin]
[Response: Please also note Myles’ response below. – gavin]
I too am struggling with Gavin’s tacit comment
Not writing all the details would imply that there is universal agreement, but that’s where I have a problem. Universally we can agree on a full range of things. Mathematical symbols, simple formulas, etc. that are foundational, but as we move higher in the structure of the logic, constructs become supportive, and their use less accepted. Somewhere past that point the uses are far less agreed upon or understood in this new context. A “tacit” use of a construct becomes problematical.
Where this line of acceptance is depends on the readers’ understanding and agreement with the writer’s arguments.
So my concern is how are reviewers, unless in full agreement, supposed to accurately review a scientific paper. I will not go beyond this point as we get deeply into a mine field of intent, bias, egos and even writing/language skills.
[Response: I wasn’t trying to suggest that tacit knowledge was some kind of opinion that all scientists must agree with, but rather it is the shared background that, say, everyone using climate models has – i.e. why we use initial condition ensembles, how we decide that a change in the code is significant, what data comparisons are appropriate etc. This knowledge builds up with experience, but the reasons for it are rarely spelt out in the technical literature. – gavin]
Once upon a time on a mountain I met a prominent scientist who liked the idea of me acquiring data not done anywhere else in the world. He stated “not enough observations, too many thinkers” , I agreed. In the final analysis,
its not a press report which does good science, it is hard work, and as it turns out, not letting all this hard work get distorted by any media source is the job of anyone interested in good science.
Peer reviewed science and Blogs complement each other.
Without Realclimate’s explanations I’d probably still be a climate change sceptic (or at least an agnostic).
Without other blogs like Rabbett Run / Stoat / and Tamino’s Blog (Aghh! can’t recall the name), there’d still be many issues I’d be confused about, even with reading the peer reviewed science.
It’s taken me years of hard study to get to the stage where I can read the majority of papers I find (without paying for them) and understand or see problems on a single read. That’s for an upper second graduate in electronics with years of wider science reading and hobby level application. Yet even now I am amazed at the kickself simple objections real scientists make to papers I’ve read and thought OK. The guidance of working scientists in the field is crucial for non-professionals like me.
I also find I have reached the stage where I don’t swallow what I see in the media uncritically, virtually every story could bring some nit-picking objection from me. However apart from the obvious buffoonery of elements of the press (I’m thinking in particular of the entrenched ill-considered denial in the right-wing British press). I do think one has to be lenient with the press. In explaining issues such as climate science they are caught between the public’s demand for easy to understand sound bites and the complexity of the reality. I find that more and more I have to bite my tongue to stop myself constantly correcting the understanding of friends and colleagues. They have the basic message, the details will just confuse them.
This problem of the existing limits to public grasp is probably best shown by my experience during my degree. We had an excellent lecturer who, as the course proceeeded, would more and more say things like: “You remember how last term I explained…. Well, I was lying…”
In order to bring his pre-graduate students up to a graduate level of understanding my lecturer had to allow us to grasp a simple (and “wrong”) level of understanding so that we could then proceed to grasping the deeper reality and gain a “less wrong” level of understanding. Likewise it’s impossible to get most of the public to grasp the message, and when even amongst the vast majority of professionals who accept the broad scientific consensus the details of their respective understandings mean a plethora of different messages for the public.
From my position as an amateur following this issue I need blogs as much as I need access to the peer-reviewed science. But they are apples and oranges, blogs are in essence like part of the press, the journals are the true arena for resolving issues in the science.
However I have sympathy with Myles Allen feeling aggrieved that he was not informed of the RealClimate post he felt unfair (although I had taken on board the point Myles makes about models producing a wide range of apparent sensitivities). I think that any blog run by active scientists should contact the authors of papers they criticise as a matter of course.
If anything I’d like to see more scientists willing to write here to explain their own papers. I’ve often thought when reading posts here that having 2 scientists discuss with each other in a public forum could be very enlightening.
Myles Allen has posted here, RC planning another climate sensitivity post, perhaps Myles could spare the time…
I know you’re all busy, I know it’s cheeky, but if ya don’t ask… ;)
Isn’t it two dasy late for the suggestion that science is like chicken sexing?
The big problem with blogs is that there is no way for an outsider to know which are reasonably careful creations of informed scientists, which are opinions of the scientifically illiterate, and which are astroturf creations designed to confuse critics of science that is in conflict with an industry.
Given that terrain, I would rather have something like RealClimate than not: it helps to balance things out. Errors tend to be corrected quickly here as a consequence of a large informed readership (even if it is sometimes annoying that you get drive-by ignoramuses who don’t benefit from getting their misconceptions answered).
As far as critiquing peer reviewed work goes, if the work is really no good, does it deserve to have an inflated citation count by attracting a flood of peer reviewed rebuttals? A site like this is of value in providing a forum for such rebuttals — and developing a consensus on whether a formally published rebuttal is worth the effort.
What I see as missing is something that has the interactivity of a blog but leads to a final agreed version of an article. A possible model is the RFC (Request for Comments) approach used for agreeing standards in the Internet world. You publish a draft, and keep working at it until everyone likes it, then it becomes a standard.
Interesting question. I’ve discussed related topics over the years among scientists and science journalists. I have actually had science writers who are nonscientists say that they felt their status as laymen made it easier to relate to their audience, while I have had science journalists who are scientists claim they don’t see how one could be effective without experience as a scientist. Realclimate fulfills an important niche in that it provides a forum for scientists to speak ex cathedra as it were, while still providing refuge from the anarchy that pervades the blogosphere. I believe that the contributors are careful to present the consensus science and to note when they are expressing their own individual opinions. This important distinction makes Realclimate a science blog, rather than a blog about science.
As a blog, it is important that Realclimate not be bogged down in a lengthy formal peer review process, alghough I’m sure they share their entries with each other before posting and benefit from the criticism. As such, I think that Donals Oats’s suggestion of a Corrections/Elaborations section, cross referenced to the relevant posts would be a reasonable addition. It would provide a mechanism for redressing errors/grievances… that sometimes (albeit rarely) occur.
I believe the science community needs to be more involved in blogging. How many times have I heard a presentation or read a paper and thought wow that is great or that is ridiculous, or that gives me a great idea. Many. However, I have submitted very few such comments to a peer reviewed process. It is such a slow unwiedly process that it seldom advances the science much. Science builds on itself. With blogs we can express questions, concerns, confirming analogs etc. in a manner that speeds science faster toward better answers. Collectively we are aware of much more than we are individually, but until recently we labored mostly in are small groups with input from colleagues who managed to attend the meeting we were at and then comments are not usually collectively shared.
I have the exciting and often infuriating privalege of working in wildlife science where decisions are guided as much by “gut” feelings as by scientifically derived knowledge. I can’t count the number of times I’ve heard that decision-making in our field is an “art” as much as it is a “science”. It has allowed some land management actions, for example, to go on long after they should have been put to bed (e.g. the “science” behind the Savory grazing system or clearcut logging to replicate natural forest disturbance).
My recommendation is to be careful. I’ve noticed journalists sometimes give as much weight to information from other news articles and blog entries as a scientist would to a peer-reviewed article. Look at Andy Revkin’s blog for numerous examples including yesterday’s (April 3) entry. That is no different than a preacher citing scripture as proof of this or that.
Blogging is a great communication tool and I very much appreciate your work with this one, but it could also be a slippery slope that could lead to the degradation of science, simply because most of the end users don’t understand the scientific method. Maybe a blog entry about repeatable results and the ability to make predictions, etc. would be helpful.
I think Myles makes a good suggestion, that blog entries that point out possible ‘errors’ be offered to the original authors for comment prior to posting. This is equivalent to the comment and response used in peer reviewed journals, and every major newspaper offers the subject of an ‘expose’ the opportunity to comment before the papers hit the street. I find the give and take of these exchanges to be very useful in ubderstanding the significance of the original article. Journal articles, especially in ‘leading’ journals such as Nature and Science, are getting ridiculously short, and many details of analysis are necessarily omitted, and much can be buried in a simple figure. The comment and response format in postings on an RC-type forum would add greatly to the general understanding of new papers.
Somewhere, perhaps in RC, someone posted the suggestion that if you think you have found a fundemental error in a major scientific work, you probably haven’t. If you think you have, contact the author(s) for comment. Any responsible scientist is willing to openly and honestly discuss his or her work with scientific colleagues.
I would like to follow up on # 16. Many years ago, 1946 to be specific, I had the privelege of working in the Cavendish Laboratories as a Research Student. Prof. J.J.Thompson left 1000 GB pounds, (a lot of money in those days), in his will, to provide free tea in the afternoon. As a aside, J.J.Thompson’s office was called “The Tea Room”, because he spent a lot of time there, talking to people over a cup of tea. Naturally, every afternoon, at 3 pm, the scientists in Cavendish Labs collected to get their free tea, and it was here that an enormous amount of interchange of ideas took place. I can remember one memborable afternoon when I was able to converse with Prof. Dirac. I hope this ritual continues to this day. I believe that the informal exchange if ideas between scientists is something that should be encouraged as much as possible.
Gavin’s use of the phrase “tacit knowledge” seems to have caused confusion among some readers, who see it as some sort of attempt to push a specific scientific view. When a chess master discounts a side variation or mentions that advancing a pawn certainly loses, it is not immediately obvious to the lay person why this is so. He is using tacit knowledge that is gained from experience. This is no different than what a scientist uses.
I would also add that people who have not been involved with academic research or research conferences do not understand the harsh tone taken by some contributors. Scientists have to develop a thick skin due to the competitive environment within their research community. These criticisms are often interpreted by outsiders as vehement personal attacks, but that is usually not their intent.
Richard #17: This is pure, unadulterated horse puckey (and yes, I have wanted to say that about some of the papers I’ve reviewed, as well–another value to a blog). You clearly know nothing about climate models. They have nothing to do with any single modeller’s opinion. Rather, they put in the most important forcers (as suggested by data and studies) with the strengths constrained by data and then see how they reproduce the observations. These are experiments that validate or invalidate the models. The fact that they occur in a computer does not invalidate them.
Richard, if you want to peddle this stuff, you had best find an audience that is as ignorant of science as you seem to be.
Let’s not confuse realclimate discussions with the general question of how modern scientific discussions should be conducted in general. In the first place, as others have noted, climate change is unique as an intensely politicized public policy controversy (in fact, I think it’s unique in the entire history of science). In the second place, climate change involves experts in dozens of radically different fields, from aerosol chemistry to tree ring chronology, so communication and validation of expertise is far more difficult than in any other major scientific topic (in fact, I think again unique in the entire history of science). Discussion of the possible role of online forums in typical scientific work, or even ordinarily controversial work like stem cell research, is all interesting, but it’s a very separate matter.
RE #10 & “Al Gore and claims that the debate is over.”
I took that statement to mean that the general debate re whether or not AGW is happening is over. Now, of course, it’s a free country, and scientists can spend the next 30 years trying to disprove AGW if they wish (and there should be some doing just that).
It’s like saying the debate about evolution is over — it is re general scientific knowledge. The evolutionists will go on arguing vociferously about the details, but the main debate is over. And there is this outside chance that God will appear (or we will find out in heaven or hell) that he planted all those fossils as an April Fool’s joke, and that he actually did creation with his magic wand.
Science in the final analysis is provisional and contingent. It keeps changing…..but broad scientific debates (e.g. re the earth goes around the sun) come to a close, even if only a provisional and contingent close, and most scientists turn to putting their energies into other areas where there still is some debate (usually over what I call “the details”).
So, I agree with Al Gore (and most, if not all, climate scientists) that the general debate about AGW is over (tho some keep arguing on and on to the contrary like zombies), even though the scientists are still doing climate science and ironing out “the details.” I mean, how much science does the average person have to know to screw in a compact fluorescent bulb?
Okay, here’s my lightbulb joke. How many people does it take to screw in a compact fluorescent bulb?
1. One layperson; or
2. 3 scientists – one to collect the data, one to do the analysis and write the report, and one to screw it in; or
3. 6 rocket scientists – one to collect the data, one to do a dynamics study, one to do an ergonomic study, one to do a human impact study, one to report to NASA, and one to screw it in.
4. Denialists? They are unable to screw in compact fluorescent bulbs, but they are writing a 500 page thesis on why it’s impossible to do so.
Of course, peer-review adds a whole other set of scientists needed for this lightbulb project.
Oh, and one more thing. Gavin is coming from an unusual place. After reading in a wide variety of the climate science literature, I realized that computer modeling is far beyond any other field to the extent it depends upon tacit knowledge… by which I mean important knowledge not documented in peer-reviewed publications, nor even in textbooks and gray literature like conference proceedings, but widely shared. Even the very earliest papers, back to Arrhenius, basically just gave equations and results plus a few hints about how the one led to the other. As for the crucial model results in the latest IPCC report, they are essentially published in the report itself. Maybe I’m wrong, but as far as I can figure out, if I wanted to understand what lies behind the results I would need an expert guide just to figure out how to use the relevant databases etc.
I couldn’t get in to read Myles’s piece, but it dawned on me he’s the one behind the climate prediction project, which used people’s computers from around the world (unfortunately I had a bad computer & dial-up connection at the time).
Yes, this work is important, even though what’s in the high-end fat tail is less likely than what’s in the center glob (around 3C). For scientists from Planet Zork studying earth it’s just an academic issue, but for people living on planet earth, we really need to be aware of all possibilities in such a dire situation that we are slowly (in human terms, but fast in geological terms) creeping into. The media not only need to give accurate summaries of scientific findings in understandable, lay language, they also need to reflect people’s concerns about real extreme dangers, even if those extreme dangers are not as likely as lesser dangers.
I personally think the media on the whole, esp over here in the U.S., sink on AGW. Their silent treatment for nearly 15 years, interspersed with some pro-con formats (e.g., NIGHTLINE’S “IS SCIENCE FOR SALE” in 1995), making it seem there was an even debate, and not at all raising the issue of the dangers if the pro side were correct. And the sponsor for that program, Texaco — which led me to believe that whether or not science was for sale, the media sure are.
Gavin’s point about tacit knowledge is important. When it comes to peer reviewed papers, one has to presume the reader will have a minimum level of familiarity with the subject matter. One also presumes that the reader will have a day job, and so the question becomes whether the information in the paper is of sufficient interest to the average scientist in the community to say, “Hey, take a look at this. It looks mostly correct to me and has some interesting information/insights/methods…” This is not in any way the gold standard in the sciences. The gold standard comes when the community as a whole says, “Hey, cool, I can use this.” The paper is cited. The techniques are used. Science advances. Eventually, what was in the paper becomes part of the tacit knowledge assumed by reviewers.
The tacit knowledge one can presume for a blog like Realclimate is much lower. One presumes there is an interest in the subject–why else would the reader be perusing the blog. One presumes at least a passing acquaintance with the scientific method and maybe some familiarity with basic results like conservation of energy, etc. One could perhaps assume that the average reader has taken the time to acquaint him- or herself with material to which one is vectored via the “Start Here” button–although this is far from Universal.
For the average newspaper reader of a science story, the tacit knowledge is nearly nonexistent–or worse, wrong. And then we have the blogosphere, where information density is at best, rarified and often toxic.
It may be too much to ask that people become discriminating consumers of information. God knows well meaning friends unintentionally bombard me with emails that never would have been sent had the sender paid a quick visit to Snopes. However, in an information economy, it seems that all too many readers and journalists are content to remain paupers.
Think back to your first introduction to geometry, algebra or calculus. Your teacher/professor undoubtedly gave you all the theoretical knowledge to solve every single problem given to you. But how easy was it to apply that theory the first time out? How much worse would it have been if they didn’t tell you explicitly which piece of theory you needed to use to solve the first few questions?
You can learn all the theory in the world, in most cases you must first practice it so you know what theory/technique to apply in a given situation. Knowing stuff by itself is never enough, you must also have the experience to use what you know and this comes from years of practice and isn’t going to be part of the formal literature. This is the difference between a professional and an armature and is the difference between debate in peer reviewed journals and blogs.
That’s not to say blogs don’t serve an important purpose. Good blogs like this one can play a very big role in helping understand and digesting the debate playing out in the peer reviewed literature, but you shouldn’t fool yourself into thinking it’s where the real debate is going on. What you should be looking for from a blog is insight into the debate playing out in peer reviewed literature.
When blogs strike out on their own and try to do “original” work there is going to be a real problem because blogs are fundamentally appealing to and often written by armatures who don’t have the experience to apply what knowledge they have.
Were it not for blogs such as this one, I, a professional engineer, but strictly an amateur scientist, would have far fewer opportunities to learn about the topic of climate change, and the controversies surrounding it. I do not regard what I read here as peer-reviewed scientific papers, unless you say so, but rather, more like what I read about such papers in publications like Science News. I would be much more poorly informed without RealClimate. Occasionally it happens that serious scientists with the best of motives do disagree on conclusions and details. It is also the case that there are “scientists” who have dubious motives and more dubious “papers” used more for political activism than for actually advancing scientific knowledge. Reading RealClimate is very helpful to us who are not professional scientists to discern the difference. Please, keep up the excellent work you are doing.
Re. 19 Dear Richard Vadon,
I took the trouble to email you about your radio 4 program. I pointed out several inconsistencies between your program and the scientific literature. You need to read the Journals Science, Nature, Climate, Geophysical Letters.
These people that you list as approving your program do not publish about climate change in peer review journals. They have no system of checks and balances as such as can say anything they want without having evidnce to back it up.
I am afraid that your comments are very misleading and show a complete lack of knowlege about how science works.
Re Jenne @ 18: “Scientists should embrace the open scientific debate”
And they do, within the system of conferences, symposia and peer-reviewed journals set up to discuss and debate scientific issues. A problem arises when the debate changes venue into public fora where any and all comers are free to participate, no matter what their level of scientific knowledge or lack there of, no matter what their economic, ideological or political agenda is, and to present misunderstood scientific concepts, misinformation, deliberate disinformation, and outright fabricated untruths as equally valid points of argument. Don’t mistake such a free-for-all for scientific debate.
Peer reviews save you from publishing nonsense. Blogs don’t.
Re: Ray #36, in the field of climate science, I don’t think you can argue for assuming that different a level of tacit understanding by the reader for the peer review literature than for blogs like this one. Climate science is particularly multidisciplinary, with contributions from several disciplines of physics, chemistry, oceanography, geology, meterology, biology and historical records. The peer review writing that crosses the disciplines, and is not just within one specialty should assume much less tacit knowledge, and be more explicit about everything they are doing. Unfortunately, especially unifying parts of the science, such as coupled hightop models will probably have only a subset of the disciplines represented among the reviewers. The modelers especially should aspire to high quality writing accessible and complete enough in explanation to be reviewable by the scientifically literate of any discipline.
It seems obvious to me that science must reach beyond the peer-reviewed literature. Science education is one important activity that necessarily reaches beyond the literature; it deals in simplified versions of scientific models and procedures in order to introduce people to those ideas. (Jack Cohen and Ian Stewart have called these simplifications, “stories for children”, which is perhaps too condescending). In addition to formal education, which only reaches the next generation, science also has to provide materials for the public at large, such as popular science books.
ClimatePrediction.net was important not only for its results, but because it did engage the public, via the distributed computing infrastructure, via the BBC Horizon programmes, and via material for use in schools. It helped to teach people about climate prediction and more generally about computer modelling.
Computer modelling is an increasingly important part of many scientific endeavours. It is one aspect of the use of computing infrastructure in science – or “e-science” as it is sometimes called. This “in-silico” modelling has been called a third pillar of science, in addition to the long established pillars of experiment and theory.
Little of this is yet taught in schools or even in some undergraduate courses. We need a large effort to increase people’s awareness and understanding of these approaches and we can’t do that within the peer-reviewed literature.
The question of how blogs fit into this space is interesting. Clearly they are rather different from traditional educational material, which is subject to its own review processes.
It’s perhaps worth noting that members of the open science movement are not just discussing science on blogs; they’re actually recording their science on blogs. This seems to be happening more in the life sciences – take a look at openwetware.org as an example.
#42 martin
This is a laudable goal and I’ve long been a huge fan of interdisciplinary work, and smart authors try to check things out with an appropriate mix before they submit a paper.
BUT:
1) Have you run peer-review for papers? How easy is to collect enough good reviewers for a journal issue? [Note that with revisions, somebody may have to look at a paper several times.]
How much credit does an academic get for reviewing papers?
2) Page counts, especially in prestigious journals, are limited. A certain level of expertise has to be assumed. One can argue about what that should be, but the devil is in the details of how much time it takes, schedule limitations, page counts, etc.
At best, peer review gives a coarse screen, i.e., it’s necessary, but not sufficient. Lots of junk gets through, but there’s a cost/performance tradeoff like elsewhere: perfection is very, very expensive. Junk tends to get refuted, or just lie like there like a dead fish and not get cited.
Regarding blogs: I think science blogs are pretty useful, if one wants to use them properly. After all, blog-like things are hardly new – USENET discussion groups started in the mid-1980s, and some have actually been quite productive. [I just wish modern blogs were as sophisticated about killfiles. :-)
I don’t see the problem here. If there is a blogposting by an expert that criticizes a published article, then that may prompt other scientists to take a more critical look at the article to see if the criticism is justified or not. If the criticism is seen to be justified by most other experts then that puts pressure on the journal to improve their peer review standards.
Complaining about the peer review process when your article is rejected is also a good thing. E.g. read this rejected article by Michael Duff. It is a comment on an article by Paul Davies in Nature. Davies’s article was obviously flawed due to a very elementary units issue. Unfortunately, Duff’s Comment was rejected. Duff included the Referee reports in his preprint and everyone can now see how the Referees and Paul Davies himself make a fool of themselves (see page 3 and further of the preprint). Global warming also comes up in the exchange, see last remark before the start of Appendix B on page 9 :)
Two more examples:
Doron Zeilberger exploded in anger when his article was rejected :)
Another case of scientists complaining about flawed peer review process :)
Martin Lewitt: Actually, interdisciplinary disciplines are not more the norm than the exception. My own specialty (radiation effects in semiconductors) combines nuclear physics, semiconductor physics, electromagnetism, spacecraft design, radiation transport and details of semiconductor fabrication–and maybe a wee bit o’ psychology as well. If one does climate science, one has to be up on all the contributing disciplines at least to the extent that one knows the basics, the real experts, and could at least review a paper for general interest in the field. Similar considerations apply in materials science, planetary physics, particle astrophysics and on and on. This ain’t your father’s science.
As a concerned citizen but non-scientist, I ask that we always keep in mind that humans are facing a compressed timeline of looming change. It is encouraging to see scientists, policy makers and journalists efficiently exchange climate information, research progress and discuss solutions.
I really do not want to see any delay in meeting climate challenges. To me, ‘leisurely discourse’ is the equivalent of ‘business as usual’
Gavin’s reply to my post #17 cannot go unchallenged. Computer modeling is NOT fundamental to science. Hypothesis generation and confirmation by empirical study is. Computer modeling was not around when Darwin postulated evolution or Einstein postulate relativity. It was not fundamental to the development of the laws of gravity and thermodynamics or any other major theory. It is merely a tool for observing possible changes that might occur to a process given changes in the assumptions underlying that process.
How many of the climate models which are used as predictors of future temperature (i.e. climate forecasting) have actually undergone a forecast audit (yes there is such a thing)? I would suspect none of them. Probably because they would not pass such an audit. The paper by Green and Armstrong (2007) is essential reading for anyone who wants to understand the pitfalls of forecasts using climate models.
As for my lack of knowledge of science, I guess that my science degrees (note the plural) are just not as good as yours.
[Response: I have no comment on the worth of your degrees, but modelling (in the most general sense – i.e. not just GCMs) is fundamental to all science. Working out the consequence of hypotheses you generate in any particular system requires a model. Sometimes it’s simple and you can do it analytically (ie. a two body gravity problem), sometimes it’s not. In climate, it’s mostly not. The G+A paper is a great demonstration of what happens when people think they understand what they read when they clearly don’t. – gavin]
gavin (inline to 20) wrote:
Reminds me a bit of the math professor who skips from step 1 to step 10 in a proof because the steps in between are obvious to him. Of course those steps are probably obvious to other math professors or graduate students – but wouldn’t be to freshmen in college — which is why a professor will have to make a special effort not to skip steps when teaching freshmen. Of course, math is formal, and much of what you are speaking of is a familiarity with past literature, with problems which have been dealt with in the past but and which everyone now knows how they were solved, etc.. Things which one no longer has to articulate — because your audience of peers will recognize the short-hand and so forth.
Dear Richard (post 19),
I can understand that you made your programme with the right intentions, given that you felt the climateprediction.net team was a bunch of dishonest scaremongers from the outset (why else would you have taken such pains to disguise from us what the programme was actually about when you originally approached us for interview?).
And you could have been forgiven for getting that impression from what was available on the internet. Of course, Gavin and Stefan didn’t suggest we were dishonest, but if they were right that our most important result was the 2-4 degree cluster, then it would certainly have been dishonest not to have made sure that this cluster was mentioned in any press releases. But, to reiterate, that wasn’t a result at all: the study itself didn’t tell us anything about the likelihood or otherwise of the traditional range.
Since I am not aware that you actually interviewed any of the journalists who originally covered the story or who were present at the original press conference (please correct me if I’m wrong here), I can see how your views must have evolved. It is a very nice illustration of the dangers of getting too much information from cyberspace: internet discussions have their own momentum (“tacit knowledge”) that may not reflect what has actually gone on in the real world.
As I said in the discussion of your programme on RealClimate, we asked Fiona Fox of the Royal Institution to follow up with those who had actually covered the story. She kindly wrote to a all the journalists she had on record who were at the press conference asking them for their reaction to your accusations, stating:
========
…
My own clear memory of this briefing is that the scientists were very clear that the results showed a range of warming between 2 degrees and 11 degrees and that each time they were asked about the impact of 11 degrees they reminded journalists that this was the worst case scenario and it could just as easily be at the lower end. Obviously we all knew (the press officers that is) that you would report 11 degrees and the fact that this was twice the level suggested by previous studies was clearly a significant news story. However I believe that the scientists themselves were very measured and did not emphasise the 11 degrees.
…
Fiona Fox , Director
Science Media Centre
The Royal Institution
========
The responses Fiona received were as follows:
========
Hi Fiona,
My memory tallies with yours. They presented the range, they described the concept of the ensemble, they emphasised (in response to a very perceptive question from some star BBC journalist) the role of clouds in the uncertainty, they mentioned 6 main reasons for uncertainty.
If anyone went for the exaggeration it was the journalists – we all mentioned 11 degrees I’m sure but as far as I recall, PA and Metro presented it virtually as a fait accompli.
Richard Black, BBC
========
Thanks Fiona, my memory is as yours. Let me know what feedback you get and I’ll write you something properly tomorrow.
Ruth Francis, Nature
========
Hi Fiona,
As I recall, the researchers, and Myles Allen in particular, emphasised the fact that the bottom end of the range (ie the 2 in 2-11 degrees C) corresponded to previous predictions of 2-5 degrees C. I seem to remember that they said this gave strength to the prediction that there would be a warming of *at least* 2 degrees C, but that there was a greater degree of uncertainty at the top-end. This last point was definitely underlined. To back that up, refer to Myles’ quote in my article:
http://www.scidev.net/News/index.cfm?fuseaction=readNews&itemid=1878&language=1.
Hope this helps.
Catherine.
Catherine Brahic
Senior correspondent
Science and Development Network (SciDev.Net)
========
I’d agree with Catherine’s interpretation – as far as I recall, they were all quite careful to stress the greater temperature change the greater the degree of uncertainty. I’ll try and dig up the bulletins report.
Sarah Mukherjee, BBC
========
Hi Fiona – my memory is that the scientists took pains to point out that it was a range and quite a broad range at that. I also remember Myles in a rather vivid phrase saying that we had to remember that we could still take actions to avert the worst warming and that we shouldn’t assume “that our children will stand by and watch as the seas boil around them”, showing that the worst case wasn’t necessarily the most likely outcome.
Thanks,
Fiona Harvey
Environment Correspondent
Financial Times
========
I am not aware of anyone who covered the paper who did not either attend the press conference, speak to a project team member, or use an agency report from someone who had done (again, if your research revealed otherwise, please correct me). The Natural Environment Research Council Press Office assured me that all recipients of their press release (including all those quoted above) would have received it attached to the paper and would have known that it was intended simply to draw attention to some interesting results in the paper, not to provide a comprehensive summary. Judging from the responses above, it appears they were absolutely right.
[Other readers may like to know that all this information was available to Richard before the airing of his programme: since Richard is still encouraging you to go and listen to it, you might like to ask yourselves how balanced it really is in the light of the above responses from the people who were actually there.]
Tim Palmer and Bryan Lawrence would not have known about this context (nor, indeed, would any of the scientists you interviewed for your programme), since despite the fact that your programme was about coverage of a scientific story, you apparently didn’t want to talk to anyone who had actually covered it.
Gavin is probably right that scurrilous was a bit strong, since I accept your intentions were in the right place. Misguided would have been a better word.
Regards,
Myles Allen
Gavin: is there any way this response could be pushed up next to Richard’s? I’ve made these points before, but as far as I can tell no one noticed because they were too far down the thread (another example of the fallacy of the “you can always correct mistakes by responding on the blog” argument).