Guest commentary by William R. L. Anderegg, Jim Prall, Jacob Harold, Stephen H. Schneider
Note: Before Stephen Schneider’s untimely passing, he and his co-authors were working on a response to the conversation sparked by their recent paper in the Proceedings of the National Academy of Sciences on climate change expertise. One of Dr. Schneider’s final interviews also addresses and discusses many of the issues covered here.
We accept and rely upon the judgment and opinions of experts in many areas of our lives. We seek out lawyers with specific expertise relevant to the situation; we trust the pronouncement of well-trained airplane mechanics that the plane is fit to fly. Indeed, the more technical the subject area, the more we rely on experts. Very few of us have the technical ability or time to read all of the primary literature on each cancer treatment’s biology, outcome probabilities, side-effects, interactions with other treatments, and thus we follow the advice of oncologists. We trust the aggregate knowledge of experts – what do 97% of oncologists think about this cancer treatment – more than that of any single expert. And we recognize the importance of relevant expertise – the opinion of vocal cardiologists matters much less in picking a cancer treatment than does that of oncologists.
Our paper Expert Credibility in Climate Change is predicated on this idea. It presents a broad picture of the landscape of expertise in climate science as a way to synthesize expert opinion for the broader discourse. It is, of course, only a first contribution and, as such, we hope motivates discussion and future research. We encourage follow-up peer-reviewed research, as this is the mark of scientific progress. Nonetheless, some researchers have offered thoughtful critiques about our study and others have grossly mischaracterized the work. Thus, here we provide responses to salient comments raised.
Definition of groups: The first of four broad comments about our study examines the relevancy of our two studied groups – those Convinced of the Evidence that much of the warming of the last half century is due in large part to human emissions of greenhouse gases, as assessed by the IPCC, which we term “CE,” and those who are Unconvinced of the Evidence (“UE”). Some have claimed that such groups do not adequately capture the complexity of expert opinion and therefore lose meaning. To be sure, anthropogenic climate change (ACC) is an immensely multi-faceted and complex area and expert opinion mirrors this complexity. Nonetheless, society uses simplifications of complex opinion landscapes all the time (e.g. Democrat versus Republican for political views) that don’t “lose their meaning” by ignoring the complexity of nuanced differences on specific topics within these broad groups.
The central questions at hand are: are these groups (1) clearly defined, (2) different in views of ACC, (3) reasonably discrete, and (4) in the main mutually exclusive? Our definition of groups, based entirely in the case of the UE group on their self-selected, voluntarily signed statements and petitions expressing various versions of skepticism about ACC, is clearly defined in the published paper. The strongest evidence indicating that our CE and UE groups satisfy the second and third criteria is that only three of 1,372 researchers fell into both groups—and in two of those cases, the researcher unwittingly added themselves to a statement they did not in fact support. Thus, if only one researcher of 1,372, or 0.07%, legitimately falls into both of our groups, this suggests that the two groups both differ starkly and are discrete. Any statistical analysis would be only trivially altered by having three redundant members of the cohort. Furthermore, the CE and UE groups are coherent, as around 35% of signers in each group also signed another statement in that set.
Another researcher suggests that his views have been “misclassified” by our inclusion of older public statements, as he signed a 1992 statement. Using a sweeping set of public statements that cover a broad time period to define the UE group allows us to compile an extensive (e.g. make an effort to be as comprehensive as possible) dataset and to categorize a researcher’s opinion objectively. However, were we to reclassify this researcher, it would only strengthen our results as then none of the top fifty researchers (rather than one researcher, or 2%) would fall in the UE camp.
Others have contended that the only experts we should have analyzed were those researchers involved specifically in detection and attribution of human-caused climate change. Importantly, much of the most convincing evidence for ACC comes from our understanding of the underlying physics of the greenhouse effect, illuminated long before the first detection/attribution studies, and these studies provide only one statistical line of evidence. The study could have been done in this manner but let us follow that logic to its conclusion. Applying this stricter criterion to the CE list does cause it to dwindle substantially…but applying it to the UE list causes it to approach close to zero researchers. To our knowledge, there are virtually no UE researchers by this logic who publish research on detection and attribution. Following this logic one would have to conclude that the UE group has functionally no credibility as experts on ACC. We would, however, argue that even this premise is suspect, as ecologists in IPCC have done detection and attribution studies using plants and animals (e.g. Root et al. 2005). Finally, applying a criterion such as this would require subjective judgments of a researcher’s focus area. Our study quite purposefully avoids making such subjective determinations and uses only objective lists of researchers who are self-defined. They were not chosen by our assessment as to which groups they may or may not belong in.
Some have taken issue with our inclusion of IPCC AR4 WGI authors in with the CE group, in that the IPCC Reports are explicitly policy-neutral while the four other CE policy statements/petitions are policy prescriptive. However, we believe our definition of the CE group is scientifically sound. Do IPCC AR4 WGI authors subscribe to the basic tenets of ACC? We acknowledge that this is an assumption, but we believe it is very reasonable one, given the strength of the ultimate findings of the IPCC AR4 WGI report. We classify the AR4 WGI authors as CE because they authored a report in which they show that the evidence is convincing. Naturally, authors may not agree with everything in the report, but those who disagreed with the most fundamental conclusions of the report would likely have stepped down and not signed their names. The presence of only one of 619 WGI contributors on a UE statement or petition, compared to 117 that signed a CE statement, provides further evidence to support this assumption. Furthermore, repeating our analysis relying only on those who signed at least one of the four CE letters/petitions and not on IPCC authorship yields similar results to those published.
No grouping of scientists is perfect. We contend that ours is clear, meaningful, defensible, and scientifically sound. More importantly, it is based on the public behavior of the scientists involved, and not our subjective assignments based on our reading of individuals’ works. We believe it is far more objective for us to use choices by scientists (over which we have no influence) for our data instead of our subjective assessment of their opinions.
Scientists not counted: What about those scientists who have not been involved with the IPCC or signed a public statement? What is their opinion? Would this influence our finding that 97% of the leading researchers we studied endorse the broad consensus regarding ACC expressed in IPCC’s AR4? We openly acknowledge in the paper that this is a “credibility” study and only captures those researchers who have expressed their opinions explicitly by signing letters/petitions or by signing their names as authors of the IPCC AR4 WGI report. Some employers explicitly preclude their employees from signing public statements of this sort, and some individuals may self-limit in the same way on principle apart from employer rules. However, the undeclared are not necessarily undecided. Two recent studies tackle the same question with direct survey methods and arrive at the same conclusion as reached in our study. First, Doran and Kendall-Zimmerman (2009) surveyed 3,146 AGU members and found that 97% of actively publishing climate researchers believe that “human activity is a significant factor in changing mean global temperatures.” A recently published study, Rosenberg et al (2010), finds similar levels of support when surveying authors who have published during 1995-2004 in peer-reviewed journals highlighting climate research. Yes, our study cannot answer for – and does not claim to – those who have not publically expressed their opinions or worked with the IPCC, but other studies have and their results indicate that our findings that an overwhelming percentage of publishing scientists agree with the consensus are robust. Perfection is not possible in such analyses, but we believe that the level of agreement across studies indicates a high degree of robustness.
Publication bias: A frequent response to our paper’s analysis consists of attributing the patterns we found to a systematic, potentially conspiratorial suppression of peer-reviewed research from the UE group. As of yet, this is a totally unsupported assertion backed by no data, and appears untenable given the vast range of journals which publish climate-related studies. Notably, our publication and citation figures were taken from Google Scholar, which is one of the broadest academic databases and includes in its indexing journals openly receptive to papers taking a different view from the mainstream on climate. Furthermore, recently published analysis (Anderegg 2010) examines the PhD and research focus of a subsample of the UE group, compared to data collected by Rosenberg et al. 2010 for a portion of the climate science community publishing in peer-reviewed journals. If the two groups had similar background credentials and expertise (PhD topic and research focus – both non-publishing metrics), it might indicate a suppression of the UE group’s research. They don’t. The background credentials of the UE group differ starkly from that of the “mainstream” community. Thirty percent of the UE group sample either do not have a documented PhD or do not have a PhD in the natural sciences, as compared to an estimated 5% of the sample from Rosenberg et al; and nearly half of the remaining sample have a research focus in geology (see the interview by Schneider as well).
“Blacklist”: The idea that our grouping of researchers comprises some sort of “blacklist” is the most absurd and tragic misframing of our study. Our response is two-fold:
- Our study did not create any list. We simply compiled lists that were publicly available and created by people who voluntarily self-identified with the pronouncements on the statements/letters. We did not single out researchers, add researchers, drop researchers; we have only compiled individuals from a number of prominent and public lists and petitions that they themselves signed, and then used standard social science procedure to objectively test their relative credibility in the field of climate science.
- No names were used in our study nor listed in any attachments. We were very aware of the pressure that would be on us to provide the raw data used in our study. In fact, many journalists we spoke with beforehand asked for the list of names and for specific names, which we did not provide. We decided to compromise by posting only the links to the source documents – the ‘raw data’ in effect (the broader website is not the paper data), where interested parties can examine the publically available statements and petitions themselves. It is ironic that many of those now complaining about the list of names are generally the same people that have claimed that scientists do not release their data. Implying that our list is comparable to that created by Mark Morano when he worked for Senator Inhofe is decidedly unconvincing and irresponsible, given that he selected individuals based on his subjective reading and misreading of their work. See here for a full discussion of this problematic claim or read Schneider’s interview above.
In sum, the various comments and mischaracterizations discussed above do not in any way undermine the robust findings of our study. Furthermore, the vast majority of comments pertain to how the study could have been done differently. To the authors of such comments, we offer two words – do so! That’s the hallmark of science. We look forward to your scientific contributions – if and when they are peer-reviewed and published – and will be open to any such studies. In our study we were subjected to two rounds of reviews by three social scientists and in addition comments from the PNAS Board, causing us to prepare three drafts in response to those valuable peer comments that greatly improved the paper. We hope that this response further advances the conversation.
References
Anderegg, W.R.L. (2010) Moving Beyond Scientific Agreement. Climatic Change, 101 (3) 331-337.
Doran PT, Zimmerman MK (2009) Examining the Scientific Consensus on Climate Change. Eos Trans. AGU 90.
Root, T.L. et al. (2005) Human-modified temperatures induce species changes: Joint attribution. PNAS May 24, 2005 vol. 102 no. 21 7465-7469
Rosenberg, S. et al (2010) Climate Change: A Profile of U.S. Climate Scientists’ Perspectives. Climatic Change, 101 (3) 311-329.
from Definition and More from the Free Merriam-Webster … http://mw4.m-w.com/dictionary/
authority – … from Latin auctoritat-, auctoritas opinion, decision, power; synonyms – see influence, power
expert – having, involving, or displaying special skill or knowledge derived from training or experience; synonyms – see proficient
Brian Dodge @143 uses the phrase:
“(legally required, regularly tested)”
You kind of support my point. It’s the “process” that people “trust”. Experts are “respected” in their fields and have “opinions” and make “judgments” which feed the “process”. If the process is questionable people will not trust the opinion and judgment of experts. It may seem a nit but IMO a very big one.
Thanks for the substantive response.
Edit to previous post. To be clear:
It’s the “process” that people “trust” and not the opinion and judgment of the experts.
That’s me done for today…
2. Do you believe it has been proven that global climate models are “likely” or “very likely” to be accurate in their predictions that CO2 will generate accelerated warming that may be catastrophic (i.e. large positive CO2 forcings)?
Depends on your definition of “proof” of course, but there are some hints. See here:
Empirical: Modeling v. Observations
Just two graphs, sure, but they’re only a small part of a large litany. Looks to me that if there are problems with accuracy they are mostly confined to underestimation. Temperature observations vs. model outputs show an opposite bias but that’s quite consistent with model underestimations of ocean expansion as seen by observation.
“Proven?” No. Evidence strongly suggests? Yes. “Catastrophic?” Depends on your perspective.
# 122 Didactylos
#136 Maple Leaf
I thought I explained it pretty well, but I must be slipping.
Let me spell it out for this case. Take a random sample of size 472 from the CE group. Take ALL 472 as the sample from the UE group. Select the top 50 from each of these two “subsamples” and do the same comparison as was done in the paper. You still end up comparing 50 subjects to 50 subjects. This procedure can be repeated multiple times to overcome the effects of the random selection. Where does the “too small a sample“ come into play?
No, I don’t think that this paper is particularly important nor the ‘findings” inconvenient, but I do find them troublesome from the viewpoint of ethics and professionally. [edit – no sideswipes, stick to the point]
From a scientific viewpoint, it is not very solid. The groups were chosen in a poorly defined manner, the methods for gathering data were haphazard, the measures of “credibility is not particularly robust and the statistical analysis has already been discussed here. I still have not seen a credible answer to the question:
Was this paper peer reviewed?
It is my understanding that papers can be submitted PNAS by members without such a review so I would think that given the author’s desire for discussion to take place in peer-reviewed literature, this is a legitimate question.
It is interesting to apply the paper’s measures of “credibility” to the authors themselves:
The first three authors do not have a particularly large numbers of publications or citations (not surprising for Mr. Anderegg since he is just beginning his career). Although Prof. Schneider was prolific in the climate science area, I do not believe that he has published much in the area of evaluation of professional expertise ( Recall that the paper considered ONLY publications related to climate regardless of the other expertise of the individual). So using their criteria, the “research quality, expertise, and prominence of the authors” with regard to this a publication on this particular subject must be low.
I generally put much stock in the metrics in the quote from the paper, but it appears that some of the posters here do.
#140 Brian Dodge
Yes, airline flying has become very safe, but I would say that the history of commercial aviation is a tale not of “expert judgment and the precautionary principle to proactively prevent problems” but of belated response to a succession of unanticipated disasters. Today’s safety is the product of an unavoidable catalogue of tragedy.
What actually happens in real life is the great teacher. While climatology tries to alert the world to the possibility of planetary tragedy, we find that its development as a science is necessarily strait-jacketed because the essential experience of learning from mistakes cannot apply. We may be on a steep learning curve but we lack the real-life feedbacks which are necessary if increasing knowledge is to have utility.
149 (Andy),
Thank you for annoying me.
I’ve seen this quote so often of late, used as a way to disparage the idea that any scientists are to be trusted, that I was finally motivated to do what any skeptic must do, to go to the source, and read the quote for myself, in its proper context, rather than in isolation, and also dropped into another context where perhaps it does not belong.
I was delighted to find an imaginative and insightful lecture behind the quote, and one aimed at two subjects that are important to me, science, and teaching. I was more than a little perturbed at the way that Feynman described women, and while one could say “it was 1966,” it’s never-the-less appalling and reminds me how unsettling and unfair our society had been (which relates directly to his own statements about science, but applied to an ever maturing society).
But the important thing was the context. Context. Feynman was talking to teachers, about the teaching of science. He was emphasizing to the teachers the importance of teaching the method, but not as a rote recipe to be applied without thought or understanding. He was emphasizing the core approach that a well trained scientist must undertake. And he was emphasizing the importance of being a true skeptic, of keeping an open mind, and learning from the ongoing science and the observations and the experiment itself, rather than from past proclamations and the rote recipe for science.
He also said:
Considerable skill. Not blind dismissal of scientists. With skill. Considerable skill.
And he said:
Be patient and listen.
And he said:
Balance. Respect and disrespect.
This all seems very, very applicable to the question at hand, to the roles of varying scientists and experiments and opinions in a new and important question. To me, it adds even more emphasis to the post here, that years of past and recent research and ideas and experiments are what matter, and that the body of evidence which falls in line with the larger body of active scientists, properly trained and executing their discipline, are what matter… and that the working scientists are in fact always the true “skeptics” in this whole mess.
But most importantly, the original quote, that “science is the belief in the ignorance of experts,” does not in any way say what it implies when taken out of context, that one should never trust science, or the experts, the scientists. That is, in fact, not in contradiction but rather orthogonal to Feynman’s intent.
Trust and distrust. Pick and choose. Find a way to find the truth. That was his message. Train good scientists, by teaching the proper method, which begins with doubt, and ends with a better (if imperfect) understanding.
It’s also the point of the original post here, that there is a body of varied and cumulative evidence, uncovered and presented by a body of scientists, which together greatly outweighs an opposing and stagnant idea that the human race cannot impact climate. How you choose to use the information, with considerable skill or blind expectation, with respect or disrespect, is up to you, but following the rote recipe of distrusting the experts is no more correct than blind trust of one expert, or another with an opposing position.
Oh, and my apologies if this became rather long winded.
Was this paper peer reviewed?
Answered many times. Yes. See the paper’s location on PNAS, where you can read:
Contributed by Stephen H. Schneider, April 9, 2010 (sent for review December 22, 2009)
Then sort out your misunderstandings about PNAS review policy by visiting the PNAS publication guide for authors here.
RomanM,
This is getting awfully tiresome. Sounds like you want to apply bootstrapping or MonteCarlo techniques. Sure, give it a try and publish! Nobody is claiming this paper, or any paper for that matter, is perfect.
As for you asking “Was this paper peer reviewed?”
You have clearly not read the post above where they state:
“In our study we were subjected to two rounds of reviews by three social scientists and in addition comments from the PNAS Board, causing us to prepare three drafts in response to those valuable peer comments that greatly improved the paper.”
You say, “No, I don’t think that this paper is particularly important nor the ‘findings” inconvenient,”
Uh, huh. Your presence here (with that of Fuller and Mosh) and fuss by the ‘skeptics’ and those in denial about AGW clearly and loudly states otherwise.
You say,
“The groups were chosen in a poorly defined manner, the methods for gathering data were haphazard, the measures of “credibility is not particularly robust “
More unsubstantiated accusations. Accusations and critique that Schneider addressed, and which have been addressed here– and it is obvious that you have not been reading the relevant text. Keeping on repeating the allegations does not make them true RomanN.
And, who are you again? Your friend Mosh suggested that we look up your citations, please provide us with the information to do so. Thanks.
RomanM: Treating the two groups as equal creates its own bias. While valuable enough in its own right, this doesn’t make the authors’ treatment wrong. When examining the extreme values in two groups, balancing the groups defeats the purpose, since it omits some of the highest values. But as we have already established, you disagree about the purpose. What I am saying is let’s look at both perspectives instead of bashing pointlessly away at the one you don’t like. And no, I’m not going to do the analysis myself, since I found the paper more than made its point, and nitpicking over a detail like this adds nothing of real value.
What I like about the paper’s approach is that it makes the “highest-expertise” group independent of whatever methods were used to select the rest of the scientists. You can, for example, reduce the publishing cut-off without affecting the result. Likewise, deriving the list in a completely different manner would produce substantially the same result.
Your comments about peer review once again indicate that if you did read the entire article at the top of this page, then you failed to pay sufficient attention.
I look forward to reading your own article on the subject once you get it past peer review – but as MapleLeaf so eloquently asks: “Who are you?” – wouldn’t want to overlook your contribution on this subject when it arrives.
#159 ML
I didn’t bring up the sub-sampling. The authors claimed they had done that and then you threw it at me as some sort of challenge. I told you one way it could be done, but you brought up an arcane “small sample” remark which indicated that you missed the point. Rather than take a personally aggressive and confrontational stance (such as the one in your latest comment), I politely explained it again.
I gave a reason for why I thought that the paper should not have been published on ethical grounds but the moderator chose to remove as is his right. Further explanation is not warranted here.
“And, who are you again?” This coming from a person whose first name is Maple? I have been identified a number of times on other blogs, but I choose not use my full name generally because frankly I don’t trust some of the individuals on the internet who might decide to abuse my privacy. Threats were issued a while back from greenpeace and although officially withdrawn later, it remains a worry. The existence of the paper in question indicates that someone else may decide to create another list and I don’t wish at this point in time to be on it.
BPL wrote: “My own belief is that if we don’t massively switch from fossil fuels to renewables in the next 5 to 10 years, it will be too goddam late.”
I believe that the legitimate scientific “debate” about AGW is about whether you are right, or whether in fact it is already too late to avoid truly catastrophic consequences not only to the human species but to the entire biosphere, even if we ended all anthropogenic GHG emissions tomorrow.
And that debate is NEVER discussed in the mainstream media, which persists even now in promoting the manufactured, phony “debate” about whether AGW is “real”. And it seems to me that climate scientists themselves are reluctant to discuss it in public.
I certainly hope that you are right, but my sense is that the preponderance of the evidence strongly suggests that it is already too late.
> someone else may decide to create another list and I
> don’t wish at this point in time to be on it.
You’re safe so long as you don’t sign one of those public declarations from which their list was collected — unless you’re on Inhofe’s list of course.
http://650list.blogspot.com/2008/12/names-on-650-list.html
RomanM,
You seem easily offended by my terse tone…think about it though, is it any wonder that I and others have little patience for people who are happy to critique but who are unwilling to make the effort to follow through and advance the science? You seem to think that you could do better or improve upon the paper, so yet again I invite your to do what any credible scientist would do and follow through and publish.
You say,
“but I choose not use my full name generally because frankly I don’t trust some of the individuals on the internet who might decide to abuse my privacy”
Fair enough, same reason here. As I have explained to you already, the only reason I pushed you on providing your name is b/c Mosh told us to look up your citations. How can I do that? If I were you I would have a word with Mosh– he put you in this position.
You say,
“The existence of the paper in question indicates that someone else may decide to create another list and I don’t wish at this point in time to be on it.”
Now you sound like a paranoid conspiracy theorist.
As for your reference to GreenPeace making threats–well, let us not generalize, the offending comment was from a blog post. Anyhow, the text in question was stupid and uncalled for, but I do not think you have reason for concern for your safety, not in the same way that Mann, Jones or Santer do. Please take come time to read this, maybe it will put your mind at ease:
http://weblog.greenpeace.org/climate/2010/04/will_the_real_climategate_plea_1.html
Funny how Inhofe, Limbaugh, Beck and Morano have not made apologies for their inflammatory rhetoric. I hope that you have an issue with their lack of ethics? I may have an opportunity to go into climate science field in the near future, but as for now I am not doing so. Not because of paranoia, but because of the very real threats made against climate scientists. Contrast that with your motivation to remain anonymous–misinterpretation of a paper and a bungled blog post on civil disobedience.
I think that we are done here.
RE #109, Simon, and
You see, mitigating AGW is stopping doing something bad (emitting excessive GHGs), while invading another country’s sovereign territory is doing something aggregious. And it set precedence for others to invade us for the same reason (don’t we also have OT Weapons?), or for perhaps for other reasons in which we are a serious threat to planet earth.
Well, I suppose one could climb up a ladder upside down and backwards.
But here’s a better idea, become as energy/resource efficient/conservative as possible without lowering productivity or living standards (which some suggest can be done by 75% or more reduction in energy demands — see http://www.natcap.org ), then see how much alt energy we need. Not as much as you imagine. And don’t forget solar, which is rapidly becoming more feasible/cost-effective. And micro-hydro, and sensible bio-fuels — you know, the ones that don’t involve more GHGs and toxic pollution than petroleum.
Further to an item I mentioned earlier w/regard to Jame Annan’s analysis of how model ensembles are presented to the general public, I suspect this matter will become the new hobbyhorse of contrarians. Might it be worth getting ahead of inevitable empty rhetoric with some remarks from a scientific perspective?
Latest on “truth-centred paradigm” here:
http://julesandjames.blogspot.com/2010/08/ipcc-experts-new-clothes.html
#164 ML
My final words as well.
It may interest you to know that I have spent forty years in a professional capacity “critiquing” and tearing apart research projects, theses and papers before they were written or published for the pure purpose of advancing the science in a university environment in many distinct disciplines. The reason was if the mistakes were corrected and I could find nothing wrong, the research was on a pretty solid footing. At the time, one generally got a thank you in the result but no name on the paper as seems to be more common today.
Why would I publish anything on this topic? In my area, we don’t publish simple things of this sort, and for me to go through the exercise of collecting the data and re-doing something that I find unethical seems something not even a “credible” scientist would do. I would hope that if Mr. Anderegg decides that what I drew attention to was incorrectly done, he would at least learn that statistical analysis is important enough to consult a good statistician next time. To me, this is advancing the science.
I do not countenance ANY inflammatory rhetoric or violence on either side although I must admit I am more concerned with the ones that are aimed in my direction. Let’s leave the causes and blame for another discussion.
I would suggest that you are over-reacting to the threat in your personal situation. Do what you think you should.
It depends on the expertise and the solidity of its foundations as well as the underlying skills of the individual expert. For all their limitations, experts are usually the best people to tell you what is going on within their expertise. For the strengths , weaknesses and purpose of climate models go to a discussion by a climate modeler, not to a review by an economist like Lomborg.
You may not always agree with Karl Popper but he did set up some lofty ideals*. One of these was that .. if you wish to challenge a theory you should first understand it, then present it in the best possible light and only then point to its shortcomings. The problem for non experts is that they are often incapabable of doing this , or perhaps unwilling. The result is that contrarianism so often starts with a misrepresentation or misunderstanding of what it is trying to contradict.
For numerous examples read RC.
For examples of expertise, resting on weak foundations, look around in forensic science. David MacKay (yes you may have heard of him; see Wikipedia) and Ray Hill have discussed a case here:
http://plus.maths.org/content/beyond-reasonable-doubt
Before anyone should quote this out of context, please note that this was an individual whose knowledge of pathology did not appear to extend to an expertise at determing guilt. Perhaps the court made a mistake in identifying the two kinds of expertise.
————-
* Its not clear that he kept them himself.
Network science demostrates many other examples, at is, toher than the PNS paper being considered, of a giant connected component and sporatic outliers. The giant component is the main body of the subject at hand and the outliers are the largely ignored contrarians.
There are many algorithms which can be, and are, used — depending upon the purpose of the study. Some are mentioned in “The Giant Component: The GOlden Anniversary” by Joel Spencer, Notices of the American Mathematical Society, V. 57 #6 (June/July 2010), p. 720 ff.
[reCAPTCHA seems to prefer an old3er topic, “Jones files”.]
Bob (Sphaerica) — 5 August 2010 @ 3:07 PM gave a, typically, erudite response to Andy@149 (a drive by or hit and run troll) who quoted Feynman– “Science is the belief in the ignorance of experts.” My much more crass response in the form of a question is– Will, do you believe that Feynman was an expert?
Steve
“We trust the aggregate knowledge of experts – what do 97% of oncologists think about this cancer treatment – more than that of any single expert.”
The problem with that kind of reasoning is that those oncologists have between them have been trained and then practiced on large numbers of patients with the affliction(s). Thus, they speak from direct EXPERIENCE of the development, progress and final consequence of the tumour.
Climatologists have partial experience of one planet (patient) which may or may not have a problem. The sanctimonius claim to experience by the climate science community is no more useful to real people than the claims by those who predicted doom as a result of Y2K, SARS, mad cow disease, H1N1 flu . . .
If so, it must have occurred to you that making a claim you don’t want to back up is useless, especially now after so much discussion on this point.
Your description was apparently designed to leave quite a bit to the imagination in any case, and you should reasonably expect that a reader might be left more suspicious than awed by it.
Flip.
Pete50 @171 — Well, Y2K, if not prevented, was (correcly) predicted to result in economic chaos. So organizations of all sorts did the work to preveent that.
Your others examples are matters for epidemiologists, who actually do have all that accumulated experience.
As for climatology, I recfommend you learn some before commenting. Try “The Discovery of Global Warming” by Spencer Weart:
http://www.aip.org/history/climate/index.html
I am not sure what the intend of the authors was in writing this paper ? If their intent was to develop a method for policy makers to better decide on whether or not man has a role in the warming of the late 20th century then I don’t see the point since the more important issue is what the effect of that global warming would be on regional extreme weather events which really affect people’s lives. Here the concensus is not clear for example on the effect of AGW on hurricane intensity as discussed in the Realclimate blog for example. Should we compare the publication list of Kevin Trenberth to that of Chris Landsea ? What about the effect of AGW on storminess or the sensitivity of the models to CO2 doubling ? Should we list the publications of the research groups predicting a 1.5 degree Celsius increase to that of 4 degrees Celsius ? The study is tautological since they have picked the question of whether or not AGW exits. What convinced me of the role of man on global warming was what climatologists , such as those in Realclimate, were saying. I did not check their publication list to decide that AGW is real.
The predictions of doom were countered by billions, if not trillions, of dollars to mitigate against it. It’s a great example of government and business (neither subject to the kind of anti-democratic anti-science crap climate science is subject to, since banks, etc, looked … understood the problem … and spent huge, huge, huge sums to fix it).
Tell the commercial IT people that Y2K was a “fraud”. Ha ha ha ha. You’d be schooled …
Aggressive countermeasures that work are typically help up by anti-science types as evidence that the threat was not real.
Personally, I’m in favor of getting rid of all the international government efforts to diminish the effects of infectious disease. Let billions die, just to show you how idiotically stupid you are!
(actually, I don’t really believe that for this world, only that there were a second one where people like you could ban vaccinations, antibiotics, etc, and live to enjoy your freedom).
One could claim that efforts to save a few lives that cause a tiny increase in the cost of meat production is silly, of course.
Can we feed *you* the prion-infected beef, and will you pay market price, in order to firm up your ideological foundation?
Real threat. 1918. Devastating.
Roman @ 167
“something that I find unethical”
So far the accussations in this direction have proven to be without any substance, mostly through misunderstanding or misapplying research ethics.
Perhaps you’d like to have stab at it?
Pete50 @171,
I think you meant to say that “climate scientists have a….
Also, you may not realize it, but the inappropriate Y2K and SARS analogies are frequently by those in denial about AGW, so you trotting them out here does not do much to further your argument. Has it ever crossed your mind that the reason those particular issues were ultimately not a huge issue (they were definitely not a non-issue as some claim- SARS claimed 775 lives) is because governments took action? Now if you wish to find an earth-like planet where you can see how much of an issue SARS will be if left unchecked, then please go ahead. Until then, please do not continue to suggest that we continue with this very dangerous experiment with the very biosphere which supports us.
I trust and respect the science, research and recommendations made by reputable and highly experienced groups such as NOAA, NASA, NAS etc. But I realize that doing so is an issue for conspiracy theorists and/or those afflicted with Dunning-Kruger.
This is funny … and it’s obvious.
Nothing to see here, RomanM won’t attack the paper professionally, choosing to snarkily proclaim victory while he knows full well that the vast majority of scientists even remotely involved in climate research do, indeed, understand that it’s warming because physics-based predictions of warming mean it must warm.
And it’s not like RomanM is bothering to respond to the criticisms of his supposed debunking. He’s reduced to saying \we don’t publish such trivial stuff in my field, therefore it’s not worth responding formally\.
While basking in Mosher’s \you gotta see this guy’s credentials!\ while refusing to identify who he is, so we can so bask …
RomanM, trash the paper all you want, the reality is that the paper actually bends over backwards to give credibility to the denialist side, and any reasonable evaluation would be far, far harsher than this paper.
You are damned if you do and damned if you don’t with the contrarians. As stated in the Anderegg paper:
“A vocal minority of researchers and other critics contest the conclusions of the mainstream scientific assessment, frequently citing large numbers of scientists whom they believe support their claims”
When contrarians are called on those claims, and the answer is not to their liking, they then cry conspiracy and make wild and fanciful accusations about ‘black lists’ or start nit picking about (probably) inconsequential details to fabricate the impression that the work is seriously flawed etc. Well, if that were the case, please do then explain why this independent study by Anderegg et al. corroborates the findings made by Oreskes (2004) and Doran and Zimmerman (2009) is lost on those whining here and elsewhere? Oh right, silly me, it is all part of a global conspiracy…..
# 162 The Earth and much biosphere will persist has it has in the past, albeit in a different form;it is the human race and some living organisms that cannot adapt to the change that is most threatened. Humans especially.
RomanM:
I must be missing something, but why would you want to do this when the Mann-Whitney U test was already done on the two full populations? What new, interesting information would this provide?
As for the test on the top-50 in each group, I understand the reason for its inclusion very differently. Perhaps the word “sub-sample” misleads, as this is — obviously! — not a random sampling of the full populations, or pretends to be.
It is like wanting to find out which is the best football nation. You do so by making the top national teams play against each other, not so much by fielding teams made up of average citizens of each nation (although that could also make sense under circumstances). Given the motivation of this paper — trying to answer the “whom do you call for advice” question — it is a relevant and legitimate result that so-recruited “team CE” is best by a large margin.
Pete50 – there’s also Venus, Mars, Titan …
Pete50, Um actually, no. We have applied the exact same science to Venus and Mars and gotten good insight. Mars even has it’s own climate model. Moreover, we have 5 billion years of Earth’s history to look at. Just imagine how well your doctor could do in treating you if he/she had been working on you his/her whole life!
Ever wonder what else you don’t know about climate science? Maybe click on the “Start Here” button at the top left of the page and start finding out.
RM 167: Why would I publish anything on this topic? In my area, we don’t publish simple things of this sort, and for me to go through the exercise of collecting the data and re-doing something that I find unethical seems something not even a “credible” scientist would do.
BPL: Put up or shut up.
CAPTCHA: “pillaging it.”
# 165
http://www.fastcompany.com/1677797/china-overtakes-united-states-as-wind-power-leader
You obviously consider yourself to be a more useful expert on these matters. So what advice would you have given?
Re: bse. The meat industry had already tried the experiment with small doses of infected meat and quite a few people in the UK died a horrible death. The incubation period can be very long, so the story is not over. I gather that you would have advocated a more advanced experiment with higher doses for all meat eaters. What is your estimate of the number of survivors?
You left out the asbestos scare. Similar story. Pseudo-experts providing false reassurance followed by casualties.
As for SARS , I suppose that you would have tried a piece of inactivist research on a highly infectious epidemic. Brilliant idea.
Last years flu’ was interesting. The people who died were young. A friend of mine plotted the rise of all reported cases. It appeared to be exponential until it suddenly ceased. Amazing. Flu’ is always mutating and one explanation is that it could have undergone one such mutation after which most of us caught it without developing symptoms.
As for climate, please suggest how a similar mutation might reduce the effects of carbon dioxide.
This discussion carries on as though it were about differences of opinion between different “experts”.
That’s not what is actually going on in the real world.
In the real world, on one side we have “experts” — climate scientists — who do indeed have differences of opinion. Some of the articles posted by the esteemed moderators of this site illuminate the areas where there are legitimate differences of opinion, areas of uncertainty, and even ignorance, within the field of climate scientist.
What we have on the other side of the public “debate” is not another group of “experts”. What we have is a group of cranks, frauds, liars, Madison Avenue propagandists, media personalities and other bought-and-paid-for corporate shills whose job it is to perpetuate the fossil fuel industry’s one billion dollars per day in profit for as long as possible, through deceit, denial and obstruction.
Climate scientists often don’t seem to understand that they are not participating in a normal process of scientific dialectic here — arguing in good faith with other scientists who, like them, are seeking to understand what is really going on, in a contest of testing their ideas against empirical observations of nature.
Rather, they are up against DELIBERATE, INTENTIONAL, KNOWING LIARS, who care nothing for science, who care nothing for “truth”, who are simply doing what they are paid to do — which is to deceive the public so as to delay as long as possible the urgently needed phase-out of fossil fuels.
(I don’t normally bother with ‘comment streams’ but having found reason to float down this one, I pour in a contirbution.)
I have great sympathy with the objections to the article’s supporting analogies (which in choice & application were pretty rubbish – less of the oncology, perhaps more ontology). The main thrust of the article, the idea of using head counts, citation counts, etc to define the robustness of a theory; I find this alarming whatever method is employed.
A few weeks ago a BBC TV programme (title?) argued that public scepticism of AGW was entirely different to scientific scepticism. It demonstrated that the public perception is more affected by pseudo-science than the genuine science. (More to do with “The Great Global Warming Swindle”, Climategate, WUWT, even laughable tabloid nonsense – I hear the present Russian heatwave is being blamed on US heat rays based in Alaska – I kid you not!)
The sceptical scientists featured by the BBC (one was from UAH) were shown as agreeing with their colleagues on all but the level of warming expected or attributed to GHGs.
This being so, would it not be possible for science to present a concensus of “the experts” and narrow the scientific debate in a way that would(I wish) prevent the village idiots piling in as they do at present?
Connecting GHGs to climate change, even connecting to some rough numerical change in average global temperatures, results in the IPCC saying “very likely” and “highly probable” – words which are the kiss of death to any message. Can they be avoided?
CO2 is above 390ppm for the first time in, what, 20 million years? It is cetainly far higher than ever on the known record that is now 800,000 years if not 2.1 million years long.
The global carbon cycle is understood enough to be sure that volcanoes emit far more CO2 than humans but because there are so many humans the rise in CO2 is solely down to mankind. Ditto some other GHGs and these changes in atmospheric composition are making a very significant* change to global climate mechanisms.
“Likely” & “probability” are avoided. Science can get on examining the effects of so much extra IR being trapped in the atmosphere and Jo Public can be told unequivocally that his emissions are messing with climatic mechanisms. And the village idiots, they can be identified by their crazy words. This is a serious business & crazy ideas do requirie labelling as “crazy” & the authors & publishers involved should be embarrassed by the exercise. Make them an entertainment. Make the process fun – Ask yourself, why does WUWT get so may visitors?
*What is so far missing is the simple model/calculation that was used to show that 2xCO2=+2oC. I assume the ones used in the 1950s would still show this “very significant” impact of our emissions.
#45 – I am an engineer. While you you say you don’t mean to be entirely disparaging to engineers (you say you are one yourself,), I wonder how disparaging your intention was.
RE Doug Bostrum
Annan and Hargreaves are getting around. Julia Hargreaves also has a paper examining (among other things)the 1988 Hansen model. Help from Annan, Schmidt, Mann, Foster, and Nieslsen-Gammon was acknowledged. Interesting revisit of the 1988 model from a different angle?
It seems to me that these types of efforts are what the “auditors” could be doing.
And now reCaptcha has upside-down words?
#171–
Multiple “fails”–notably, undefined terms (“those who predicted doom. . .”)
Who made the predictions? Doom to whom?–after all, quite a lot of people actually did die from SARS and H1N1, and some from mad cow. For them, “doom” was in fact the outcome. And Y2K problems were (largely) averted by dint of a lot of work and expense–a climate change analog for which hasn’t really been forthcoming yet.
But the biggest “fail” is the failure to recognize the crucial point, which is that there is a very large body of well-verified knowledge about how climate systems work–knowledge that gives very clear and specific reason to think that yes, Houston, we do indeed have a problem here. That is, returning to Pete50’s oncological metaphor, we may only have one “patient” to observe, but we’ve come to know quite a lot about the relevant “physiology”–and specifically, just why it is that that patient’s fever continues to rise toward dangerous levels.
Quasi-random comparisons to unrelated issues are mere obfuscation of this central fact.
RomanM: Your statistical arguments would carry more weight if you didn’t accompany them with conspiracy theories.
The authors of the paper made a choice, yet you continue to argue that they are wrong, rather than to argue that they made a poor choice, or that their choice was weak and undermined their conclusions.
You seem to be claiming that the authors intended to use your method, or should have done so to match exactly their description – but this is patently untrue. The wording is a little off, but there is no reason to suppose that they didn’t intend exactly what they did.
For completeness, it would have been nice if the authors had used your method in addition to their own. But the space constraints probably didn’t allow such luxuries.
And for all your nitpicking and indefensible complaints about “ethics”, you refuse to see any value in the conclusions. Your bias is showing.
I don’t know what it is that leads to elderly retired types becoming self-declared “sceptics”. You insinuate that it is because publishing scientists want to be published so write what they think editors want to see, but you know that is nonsense. A paper that actually changed the consensus view on climate – that would make a career. But no such paper has appeared. There are less charitable explanations for why elderly people don’t care about the future of the planet, but I don’t want to paint you with generalities. I believe it is true, however, that older people are sometimes more inflexible and intensely dislike change.
So, why don’t you go and prove your sceptical credentials by going and railing against some of the completely outrageous abuses of statistics perpetrated by the denier crowd? Can you put aside your bias?
RomanM: It’s a trivial issue, so ‘publish’ it on this blog – what happens when you compare the top 5% of each group? Do you have any evidence that they are from the same population? It’s your job to demonstrate, not argue without data, that the authors did it wrong. That’s science – reality trumps argument. If you prefer to argue, that’s not science.
Tom Fuller. As a psychologist myself, who has to go through the protection of human subjects/confidentiality/privacy routines, can I simply say – nonsense? If you are concerned about violating privacy, ask that the next time the Cato Institute, the Heartland Institute, etc, publishes a signed statement, that they make the list of signers confidential. And perhas Google Scholar can do the same. So far, I’m not aware of any consequences of a ‘blacklist’, and I’m sure you’d let us know if there have been any. Any investigations? I’m only aware of harassment of prominent members of the 97%, which goes against your thesis.
Given the lack of constructive demonstration that the findings are biased (I agree that they are flawed, and the flaws were reported in the paper itself), and the repetitious nature of the arguments against it, with no presentation that this would make a substantive difference, I think the main point has been demonstrated. The weight of expertise and demonstrated science is on the side of the overwhelming majority.
#171
“The sanctimonius claim to experience by the climate science community is no more useful to real people than the claims by those who predicted doom as a result of Y2K, SARS, mad cow disease, H1N1 flu . . .”
And so you doom yourself to be ignored as a “know nothing idiot with a loud voice”.
I have had much reason to regret my part in solving Y2K issues. To be honest the ungrateful and unknowing idiots who rant on about spending billions on Y2k and “nothing happened” just leaves me bemused.
Yes we spend billions. Yes we “fixed it”, mainly and then we are pillioried for doing exactly what was labelled on the tin.
Consider this. IF we manage to get the climate laws in place and IF we mitigate the worst of the effects and IF geoengineering is successful in reducing CO2 and stabilising the climate.
Then climate scientists in the latter part of the 20th century will know exactly how I feel today over the Y2K fix. Because they won’t be thanked. They won’t be lauded. They’ll be pillioried as overbearing leftist “world power mongers” who spend Trillions on something that “Never Happened”.
That is the thanks the likes of Pete50 will give to climate scientists if they fix this mess. However at least climate scientists will have the pleasure of knowing how many lives they saved. We merely saved lots of companies (and economies), trillions of $..
171 (Pete50),
Yes, thank you. This is a fantastic argument for taking action to mitigate climate change. After all, none of the Y2K, SARS, mad cow disease, or H1N1 flu challenges have resulted in doom, since in every one of those cases governments and businesses took and continue to take concerted and expensive action to control the situations.
Of course, all such action was taken on the advice of experts. The fact that modern civilization has not yet been stupid enough to ignore the experts and potential, predicted, major disasters does not in anyway add support to the idea that we should do so (i.e. ignore the experts).
Oh, and you forgot to add DTD, CFCs and the link between smoking and cancer.
Except that in that last case many people did and do ignore the experts, and as such many more people have and continue to suffer from and die of lung cancer than was/is necessary. So much for the wisdom of ignoring the experts.
[Separately, do I win some sort of prize? My reCaptcha code for this was “Galileo triumphant”. How about an all expenses paid trip to the North Pole to watch the ice melt next summer?]
“… something not even a “credible” scientist would do”
Which of course must be a hint that Roman is not a “credible scientist”, rather a newly minted “citizen auditor”, with an “academic interest” – in line with the theoretical, not realistic, or directly useful and scholarly but lacking in common sense, or practicality meaning of academic.
I find it interesting which articles at RC bring out which contrarian aberrations, this one, examining credibility, attracting particularly humourous in-credibles.
reCaptcha: own treats
A number of people have addressed the issue of relevant expertise by posing some form of the question, “Whose opinion would you rely on to diagnose cancer and recommend an appropriate course of cancer treatment — an oncologist or a cardiologist?”
That’s a good point, but I would suggest it doesn’t really get to the point of what is going on with the public “debate” about AGW.
The right question to ask is: “Whose opinion would you rely on to diagnose cancer and recommend an appropriate course of cancer treatment — an oncologist, or a paid propagandist for the tobacco corporations?”
The public “debate” isn’t really about science at all.
It’s about money. It’s about the fossil fuel corporations’ interest in perpetuating their billion dollars per day in profit for as long as they can get away with it, versus the public interest in phasing out the use of their products as fast as possible.
I don’t see any point in replying to each individual separately since many of the criticisms of my comments are rants with no substantive content and it would be a waste of time to address such nonsense.
However I will make a few remarks on several issues that have been raised.
My criticism of the specific analysis of the top 50 in each is based on the fact that the same analysis applied to separate samples of sizes 472 and 903 from the same population will invariably produce larger means and medians in the 903 size sample. This is not an opinion or a guess. It follows from the mathematics behind the situation. The statistical test used in this case in the paper will produce a “significant” p-value that the larger sample size group comes from a population which produces larger values – this is false since the sample were selected from the same generating mechanism.
You do not need to “try it out” on real data to see if that is the case. However, if you wish to, go ahead. To get something similar to the data in the paper, try generating exponential variables with mean 100 for each of the samples. If you know some probability, you could calculate what to expect: the theoretical mean of the top 50 will be about 388 for the larger group and 324 for the smaller. For the data which has a higher percentage of extreme values, the difference would be greater.
I could suggest to you that you try it on the data from the paper (as you seem to be implying I should to convince you of something which can be demonstrated otherwise). However, maybe you haven’t noticed, but the data is NOT available.
The only semi-cogent suggestion was from #172, stewart longman,
The cogent part is that it makes more sense to compare the top 5% rather than the top 50 individuals in each group. Yes, that would be a better comparison, but that is NOT what the authors did. As a psychologist, you should understand that you do statistical testing to demonstrate that they ARE different, not the other way around. The onus is on the authors to use valid statistical techniques to make their point. In this case, showing that their technique is not valid can be done simply by analyzing what happens when the technique is applied.
As far as publishing this type of criticism, what journal would you suggest? Statistics journals do not publish papers showing that someone misused a statistical technique. Such errors do need to be corrected before someone else decides to copycat and make the same mistake again. The best way to do it is to publish a corrigendum from the authors in the original publication. Only in the case that the authors refuse to do this is there a need for someone else to step in. Given the “climate” at PNAS, I suspect that would be a difficult task for an “outsider” to accomplish.
I suggest that you ask Mr. Anderegg for his opinion on the points I have raised. If he wishes to ask for clarification or to discuss this off the record, I don’t mind the moderators passing my email address on to him.
As an aside, you might try counting the number of ad homs, name callings and disparaging remarks found in this thread… and look at who is generating these.
Does RC have a constructive view on the exchange between Dr Roy Spencer and Christopher Game at http://www.drroyspencer.com/2010/08/comments-on-miskolczi’s-2010-controversial-greenhouse-theory/#comments?
[Response: Spencer is correct. -gavin]
Ever the eternal optimist, I’m starting to wonder whether Tom Fuller’s attitude might not represent the first glimmering of understanding. After all, if being in the “unconvinced” camp were something to be proud of, don’t you think those who self-identify with that position would be trumpeting it from the rooftops? Maybe Tom is starting to realize that it’s not a good thing for scientists to ignore evidence.
Brian (151) what happened to that classic definition of “expert”: a has-been under a lot of pressure?