Does this sound familiar? A quantitative prediction is inconvenient for some heavily invested folks. Legitimate questions about methodology morph quickly into accusations that the researchers have put their thumb on the scale and that they are simply making their awkward predictions to feather their own nest. Others loudly proclaim that the methodology could never work and imply that anyone who knows anything knows that -it’s simply common sense! Audit sites spring up to re-process the raw data and produce predictions more to the liking of their audience. People who have actually championed the methods being used, and so really should know better, indulge in some obvious wish-casting (i.e. forecasting what you would like to be true, despite the absence of any evidence to support it).
Contrarian attacks on climate science, right?
Actually no. This was assorted conservative punditry attacking Nate Silver (of the 538 blog) because his (Bayesian) projections for Tuesday’s election didn’t accord with what they wanted to hear. The leap from asking questions to cherry-picking, accusations of malfeasance and greed, audits, denial, and wish-casting was quite rapid, but it followed a very familiar pattern. People who value their personal attachments above objective knowledge seem to spend an inordinate amount of time finding reasons to dismiss the messenger when they don’t like the message.
Fortunately for Nate, all it took was one day, and reality came crashing down on his critics entire imaginary world.
For climate science, it will probably take a little longer…
Well, you’ve got to realize that most people are allergic to numbers. And they are still granting degrees to people who have not taken a math course since 9th grade.
Run for election to your local school board.
Relevant.
The recent housing bubble also fits this pattern. There was ample evidence that a huge bubble had formed, but you’ve been hard pressed to know this at the time, given the way the media chose to highlight “this time it’s different” arguments.
I was glad to hear Obama, in his “victory speech” on Tuesday night, mention climate change, in no uncertain terms, as a problem needing attention.
What are peoples’ thoughts on the chances of climate legislation in the next four years?
Even more impressive than predicting 50/50 state correctly was how good Nate’s predictions were of the percent of the vote won by Obama in each state:
He was off by less than 1.5% on average.
you could add that unskewedpolls copies the lack of organisation of Morano’s blog … same pattern. Same inability to clearly understand the information and just see that they are angry .
Now, if you will excuse me, I have to apply for retina surgery afterhaving seen this site.
Indeed – just like the whole climate debate – compressed into a few weeks!
But in Nate’s case, his predictions were right.
[Response: Back into your epistemic bubble you go…. – gavin]
What a great graphic!! Can I use it? Can I use it?
[Response: It’s from xkcd. – gavin]
I’m wondering if narcissism, which they’ve found applicable to climate change denialists, also applies to denialists of other or even all stripes.
See: http://watchingthedeniers.wordpress.com/2012/09/13/anatomy-of-motivated-self-importance-too-much-of-me-just-isnt-enough-guest-post/
Climate change denialism is somewhat understandable in this regard — accepting climate science means coming face-to-face with the realization that one is contributing to serious harms to others, including one’s own progeny. That’s a very unpleasant feeling, that may prompt mitigating action on the part of normal people. However, it is untenable for the more narcissitic types — borderline personality disorder, narcissitic personality disorder. I’ve noticed they cause psychological harm to their own children (without admitting it) bec of their psychotic narcissistic condition — which is one of those conditions almost impossible to overcome bec in their view nothing is ever wrong with them, it’s always something wrong with other people.
Gavin, I really, really hope that you don’t mean that this has relevance to climate predictions. Out of hundreds or thousands of election predictions, there would necessarily be a few good ones, and you cherry-picked the best you know of.
[Response: The point of this post is that unwelcome information is almost always greeted with hostility that very quickly morphs to personal abuse in ways that are quite familiar to scientists working in politicised fields. As for the charge of cherry picking, 538 was the only source I spent any time looking at prior to the election – I made no search for the ‘best’ predictions after the fact. But that is pretty much irrelevant to the point I am making here. – gavin]
Nate Silver wasnt a misunderstood genius, and Silver would be the first person to agree with that statement. He didnt do the polling, either. 538.com also wasnt unique, because there were plenty of other websites that aggregated polling results, and they more-or-less agreed with Silver’s estimates. However, when the polling-denialists needed a scapegoat to focus their denial, they chose a prominent target who looked and sounded like the nerdy kid who irritated the cool kids in high school, and pretended that he was the only person who advocated a particular data interpretation. The parallel to the vilification of Michael Mann by climate denialists is almost funny.
#11–Anecdotally, it is easy to find examples of prominent deniers whose egos appear to deserve the adjective ‘monumental.’ But I suspect that it isn’t that they are bothered by the harm that they might cause via their carbon footprint; it’s rather that the allure of publicly blazoning the superiority of their intellect and understanding over a whole field of science and a whole cadre of scientists is irresistible.
(Perhaps I’m too influenced by one particular fellow I often encounter on line who is a denialist not only of climate change, but of HIV/AIDS, quantum mechanics, and who knows what else. Superiority to just one discipline apparently wasn’t enough for him!)
Even if numbers are more reliable, all of the prediction sites (and even the comic above) employed handy visuals, the better for our lizard brains to see them with.
The other relevant point here is that someone ( a single person even) can do great science in a near live time period without peer review. And that climate scientists waiting for peer review articles as a way to cautiously respond to criticism is froth with weakness exploitable by those who could not care less about it. On the other hand, being able to respond to complex current events in lay terms quickly props up the science ridiculed regularly by fake skeptics. The lag in time to respond to contrarian pseudo claims hurts the science confidence reputation and gives a chance for doubt to grip the thoughts actually guiding those who could do something about AGW.
Well, he could have picked George Will’s predictions. After all, we know how sharp he is on Climate Change?
Aggregate polling data pointed to an Obama win. That’s iron-clad proof of massive, nationwide election fraud (don’t ask me to explain why).
Aggregate temperature data shows that the Earth is warming. And of course, that’s ironclad proof of a massive international temperature data manipulation conspiracy (same reasoning as above).
And for folks who haven’t already seen this, here’s something to hit your reality-denying co-workers/family/etc. with — it shows what happens when you average raw (“unskewed”) and homogenized (“skewed”) data from 1 to 40 random rural stations: http://tinyurl.com/ghcn-animation
The interesting thing to me about the abuse heaped on Silver is that other aggregators came in with numbers that predicted an Obama victory by a larger margin or announced a larger degree of confidence in an Obama win. And yet the noise machine settled upon Silver. One blogger even mocked Silver’s build and manner of speaking. It made it clear that the noise machine’s intent was to make an example of an individual in order to intimidate other voices. In climate science, Michael Mann is an example of this. There are thousands of other climate scientists and many thousands of studies limning the extent of AGW, but the noise machine decided to “cull” Michael Mann from the herd.
While Nate Silver was quite close, he was not the only one. Check out Cassidy’s count at the New Yorker:
http://www.newyorker.com/online/blogs/johncassidy/2012/11/cassidys-call-obama-to-win-by-x.html
His only potential error is giving Florida a slight edge to Romney.
Josh Putnam over at frontloading HQ has an almost idential electoral maph, except that Obama wins Florida.
ttp://frontloading.blogspot.com/2012/11/the-electoral-college-map-11612.html
Most of the oundits will say that the key to accurate polling was accurate rebalancing of the polls to the actual electorate. That was the biggest wildcard. Those that most closely matched their polls to the voting demographics, faired the best.
It’s similar to the abuse heaped on economist Paul Krugman. The difference with Nate Silver is that most deniers are now forced to admit Silver was right and they were wrong.
In economics, this ‘Great Recession’ has proven that Krugman and the theories of like-minded economists were correct versus those of the GOP-leaning ‘freshwater’ school. It’s a pretty awkward failing of any macroeconomic theory when you can’t get interest rates or inflation correct, but that hasn’t stopped or slowed down the abuse much less force an acknowledgement of being completely and totally wrong.
Carl #12,
There are only about half a dozen models like Nate Silvers, some of them originating in academia, others in political news blogs.
Eg.
Pollster on the Huffington Post
Vote Tracker on Talking Points Memo
Votamatic http://votamatic.org/
Electoral Vote http://www.electoral-vote.com
Princeton Election Consortium http://election.princeton.edu/
Votamatic, for example, predicted the Electoral College vote precisely right, while Nate Silver did not. All predicted that the President would be re-elected with between 280 and 332 electoral votes – he actually achieved the maximum.
Silver accurately said that the probability of Romney being elected was equal to the probability of the state polls being biased in Obama’s favour.
The “pols” like Karl Rove dissed the “quants” like Silver and ended up with egg on their faces. A poll of the polls of polls would have been interesting.
I thinki Nate Silver has singled out because his prediction appeared in the disgustingly liberal New York Times.
People are wondering why Silver was singled out. Well, isn’t it obvious. People like to vote for a winner, so they had to keep the myth of Romney’s momentum alive. The only alternative would have been to physically keep voters from pulling the lever for Obama.
Obviously, they ultimately decided…better Nate than lever.
[Sorry, it couldn’t be helped!]
“Votamatic, for example, predicted the Electoral College vote precisely right, while Nate Silver did not. ”
Well, Nate Silver’s _mean_ prediction was not precisely right (it was 313.0) but you wouldn’t expect it to be – the possibility for a fractional electoral-vote should be a tip-off about that calculation. Nate actually showed the full result of his Monte Carlo algorithm in an “Electoral Vote Distribution” figure, where 332 is the most likely peak (at about 20%), and 303 is the 2nd most likely at 16%.
I like Zeke’s graph at #6, which is much more informative about the accuracy of Nate’s predictions (Zeke: could you post a link to the source?). It would be even better if each state had little error bars attached. However, it is hard to fully evaluate the accuracy of the overall model, because of the correlated uncertainty between states (eg, the possibility of national or regional bias): if all states were independent, then you’d expect with a good algorithm that 90% of the votes should fall within the 90% bounds, and 10% outside. But because of correlation, it is more complicated. I’m looking forward to Nate’s own self-analysis.
But this above discussion, regarding Monte Carlo and internal correlation and so forth, is exactly why the reality-based community especially appreciates Nate Silver: he didn’t just get the right answer, but he showed why he got it, and what might lead to him being wrong.
-MMM
Brilliant Ray!
MMM,
I created the graph by simply copying the NYTimes election map values and the FiveThirtyEight projections into a spreadsheet. Its available here: https://www.dropbox.com/s/pzz358kopewjblp/Election%20Data.csv
It seems to me that there is a flaw in concluding that the close match on the national result validates Silver. I believe that Silver’s Bayesian prediction gave Romney something like a 20% chance of winning. If Romney had won, something that was entirely possible under Silver’s analysis, would you have concluded that Silver was a fraud? I believe that would be clearly incorrect.
If Silver makes 100 predictions and the distribution of the real-world outcomes, matches his predictions, then one can soundly conclude that he is on to something.
Thus, Zeke’s graph (#6) is a lot more informative than that the outcome of the national election matched the expected value of Silver’s distribution.
[Response:I think this is exactly the point isn’t it? If Silver had been “wrong” — that is, if Romney had won — than those who’d been criticizing him would feel vindicated, and indeed would have called him a fraud. They’d be just as wrong to criticize him in the way that they did, either way.–eric]
Re- Comment by Ray Ladbury — 8 Nov 2012 @ 12:39 PM:
Paronomasia and spoonerism in one= fun!
Steve
I think there is something else going on. Deniers often not only deny the predictions they don’t like, they deny the right of people whom they disagree with to make predictions. In the case of the election, I think one reason Tea Party Supporters and other such people think there is some sort of plot is that they don’t think Obama voters should be allowed to vote. They don’t feel they are “real Americans”. I think that while Obama supporters certainly have questioned the motives of Republican extremists, they have not questioned their right to vote. On the other hand, their opponents regularly engage in voter suppression efforts. An example is billboards describing the penalties for voter fraud put up in minority districts. One interpretation of such efforts would be that the originator of such a billboard is just cynically trying to suppress votes for the opposing party. But i think what may be more correct is that such people really do believe their own propaganda about voter fraud. They don’t think those people should be allowed to vote, so it is easy for them to think of their votes as fraudulent.
Similarly, climate deniers don’t limit their complaints to errors in analysis. They also suggest that climate scientists are doing fraudulent science to maintain their lucrative contracts. But such arguments don’t make any sense at all, as anyone who has thought about it would have to know. A climate scientist who did good science which showed that climate change was not a problem would get supported by the usual sources and would also get a lot of industry support. So a climate scientist whose results showed that there was no problem would have to be crazy to falsify his/her results to show the opposite. Also, the amount of money available to support denial dwarfs that available from the usual sources. It seems to me that anyone who ignores these facts must have questions about the right of climate scientists to do what they do, not only the accuracy of their results.
Re: #24. @Ray
Is there any evidence for this?
or for the assumption that a bias in any kind of election forecast might increase the votes in the direction of the error ?
Or is it that Americans value optimism more than Britishers?
1992 in the UK. The polls were favouring a victory for the Labour Party (LP) led by Neil Kinnoch. Most of the journalists * thought that the conservative John Major would lose. In the end many people thought that a Labour bias in the opinion polls had contributed to a Conservative victory. Since then many politicians in the UK have a slight tendency to be artificially pessimistic before the election, because they hope it will encourage their supporters to make a bit of an effort.
—–
*. There was actually a last minute swing in the polls towards the Conservatives which was not much publicised. In addition the bias itself was later put down to the fact that the opinion polls had included too many Labour supporters who had lost the right to vote by moving.
I suggest you replace the show that graphic via a link to XKCD:
Permanent link to this comic: http://xkcd.com/1131/
Image URL (for hotlinking/embedding): http://imgs.xkcd.com/comics/math.png
This post raises an interesting topic that is not addressed. What are the differences between the inherent predictability of an impending election and the predictability of an impending climate change?
Nate Silver remarked on his blog, when the Obama probability passed over 87% or so, that all of the remaining probabilities for a Romney win were in the fringe of his analysis that assumed a consistent bias towards Democrats in the poll results, all erring in the same direction. At that point other poll aggregators, such as Sam Wang, gave Obama a ~99% chance.
This is similar to how the denier claims of no global warming, or of no anthropogenic influence upon warming, or of low climate sensitivity, depend on all observational data being wrong in the same direction. Silly, really…
Re: 18 Caerbannog – neat graphs, but as linked there is no caption or explanation with the graphs. I could not make out what the lower graph is showing – some kind of difference? Most of the frames I saw had the bottom graph peaking around 1960 and dropping sharply afterward… Color me puzzled.
For those focusing on Nate Silver’s stat work, this article by Michael E. Mann posted on Huffington Post questions his understanding of climate models and their accompanying statistical analysis.
In the article, Mann states, “And so I was rather crestfallen earlier this summer when I finally got a peek at a review copy of The Signal and the Noise: Why So Many Predictions Fail — but Some Don’t. It’s not that Nate revealed himself to be a climate change denier; He accepts that human-caused climate change is real, and that it represents a challenge and potential threat. But he falls victim to a fallacy that has become all too common among those who view the issue through the prism of economics rather than science. Nate conflates problems of prediction in the realm of human behavior — where there are no fundamental governing ‘laws’ and any “predictions” are potentially laden with subjective and untestable assumptions — with problems such as climate change, which are governed by laws of physics, like the greenhouse effect, that are true whether or not you choose to believe them.”
FiveThirtyEight: The Number of Things Nate Silver Gets Wrong About Climate Change http://www.huffingtonpost.com/michael-e-mann/nate-silver-climate-change_b_1909482.html
“A victory for data over punditry” on (Aussie) ABC News http://www.abc.net.au/news/2012-11-08/green-win-for-data-over-punditry/4360072 gives a bit more detail than Gavin’s post, for those who are interested. If you just want a laugh, though, go to http://www.abc.net.au/technology/articles/2012/11/08/3628527.htm
What differentiates Nate Silver’s work from most in the elections field is: (a) he’s sophisticated about quantifying uncertainty, (b) he has his full methodology published at the 538 site, and (c) his projections are posterior densities, such as the Electoral College distribution he gave on his site, or the distribution of expected Republican versus Democratic Senate seats, post election. The latter is especially powerful … These are not point estimates with confidence intervals and such, but full density estimates. He obtains his results by sampling from existing polls, weighting their sampling by how they have done historically. He adjusts things here and there.
Not only was his overall outcome impressive, but he also called an amazingly accurate number of Senate seats, and (I think, but don’t know) in the House. The Florida outcome came down on the side of his projection just by luck, but, nevertheless, his projections knew it would be close.
I’m reading Professor Myles Allen, “Liability for climate change– Will it ever be possible to sue anyone for damaging the climate?”, NATURE, 421, 27 Feb 2003. His Figure 1 shows an empirical likelihood density very like the Silver posterior densities. Professor Allen has recently expanded on that theme in a recent article included in Munich Re’s “Liability for Climate Change” report, an article called “Attributing extreme weather events: Implications for liability.”
Re: Comment by Jim Prall — 8 Nov 2012 @ 5:41 PM (#35)
My above post was a quickie re-post (I put that plot up in an earlier thread here), so I sorta skipped out on the documentation this time.
The lower half (lower plot) shows the number of selected stations that actually reported data for any given year. The #stations that reported raw data is shown in red; the #stations that reported adjusted/homogenized data is shown in green.
Since most stations don’t have a continuous record from 1885-present, the number of stations will in general decline as you go back or forward in time from the 1950-1981 baseline period. Also, stations may not report data for every month of the year — if a station reported data for 6 months of a given year, I counted it as “half a station” for that year.
In my experimentation with techniques to “showcase” the robustness of the global-average temperature results, I found that it is also important to show the actual number of stations reporting data for each year. That way, you can correlate “noisiness” of the results with the actual #stations reporting. I’ve found that once you get to 30+ stations scattered around the world, the global-average temperature trend settles down very nicely to the NASA results. (30 out of thousands of stations — not bad!)
Some additional background for those who might have questions about that animated GIF. I selected rural stations at random via mouse-clicks on a global-map GUI I cobbled together with the help of the QGIS app (www.qgis.org). As each station was selected (from random mouse-clicks all over the globe), I updated my global-average computations with that station’s data. Each frame in the animation shows results updated with data from 1 additional station’s data (40 frames in the animation — I started with 1 station and increased the station count by 1 per frame up to 40 stations).
Raw data results are plotted in red; Homogenized results in green. The official NASA/GISS land-temperature results are plotted in blue for comparison purposes.
I should note that the plot was the result of my “first attempt” to put together an animation like that. No McIntyre-style “noise hockey-stick cherry-picking”. Just took what I got on my first try at picking random statios and uploaded the results. Stations were pre-screened on the basis of data-record length only. I wanted decent global coverage for the entire 1885-present time period.
In my experimentation with the data, I found that it was virtually impossible to get results inconsistent with the NASA results — rural stations, urban stations, raw data, adjusted data — once you average data from a few dozen stations scattered around the world, everything settles right down to something that looks very much like the NASA land-temp. results.
Those who want to experiment with the software I wrote/stole to generate that plot can download a zip file containing software and data from http://tinyurl.com/WattsBusterProject
Disclaimer: The software is a rough “proof of concept” package — Depending on your computer skills, it could be a real chore to set up — you will need to deal with Unix command-line stuff, compile from C++ source, understand TCP ip-address/port settings, install a bunch of other supporting software, etc. It’s definitely not a turnkey “plug-and-play” package.
Not all number are alike; see Dylan Mathews in WP at http://www.washingtonpost.com/blogs/ezra-klein/wp/2012/11/07/which-pollsters-to-trust-in-one-chart/
Nate Silver uses the correct polls. He pays attention to data quality.
Correction: The baseline period should be 1951-1980.
Since you and Nate are in the same town, it might be worth while to invite him to lunch/over to your offices to compare notes. Seems that his good grasp of statistical nuance would put him in good stead to be interested and understand the physical theory and data driven climate models that you and your colleagues utilize, and more importantly to appreciate the differences between your and his work. Seems like it would be worth doing simply because by reaching out and exposing him to the transparency and subtleties of the scientific method, he could become a real ally in the future.
Seems we’ve had a good couple of weeks, as exemplified by the Frum quote @2. One of the major attacks on climate science has been the denigration of models, now in less than two weeks two very public model predictions based upon science have proven spectacularly true. The second was Nate’s polling predictions. Even more remarkably (and relevant to climate science) were the predictions of the strength, path and effects of superstorm Sandy. As early as six days prior to landfall ECMWF had it nailed, and within a couple of days all the other major models had converged on similar solutions. So now the public has seen that mathematically rigorous modeling can in fact beat human gut instinct. So denigrating climate change, by dismissing the models as garbage in garbage out will now fall on a bunch of deaf ears.
Nate Silver was “singled out” because he works for the New York Times. He’s been doing this analysis using more or less the same methods for several years without drawing an inordinate amount of attention. But only during the last election cycle has he done it under the Times banner.
I think that Leonard@30 makes a good point, which the initial article also describes. This is not just about denial – it is about the more or less vicious personal attacks that go along with that denial. Those vicious attacks are a form of bullying behavior. Bullies usually pick out an easy and high profile target, and then attempt to make an attack unpleasant enough to deter others from supporting the same point of view, which is invariably a point of view that operates against the interests of the bullies. In the case of the outcome of the election, there was a moment of truth beyond which the bullies’ position was embarrassingly untenable, which is the source of the humor here.
In the case of anthropogenic global climate change, there will not be one “moment” of truth. By the time the effects of AGCC are so obvious that they visibly support its already undeniably high degree of scientific certainty, it will be too late to do much to stop AGCC’s worst effects. I think some AGCC “deniers” actually know all of this. The real strategy is to attempt to “adapt.” This will be much easier for those with a lot of money, which perfectly describes those who continue to make millions from selling and burning carbon-based fuels. So in the end, we have the age-old problem of the concentration of wealth and therefore power into the hands of a limited elite, which in this case has morphed into a survival strategy for that elite in an increasingly uncertain future. The driver of this behavior is ultimately fear. Unfortunately, mankind has reached the point where this fear-sustained trade in carbon is actually the cause threatening the failure of all human society. So I go by what I learned at an early age: always stand up to bullies (Michael Mann is an inspiring example of this). Bullies are NEVER as tough as they would have you believe – that’s why they resort to bullying.
Election predictions/forecasts can be verified in almost real time (if you think geologically…). Nate got it right and good for him. Climate predictions/foreasts, on the other hand, will only be verified long after anybody posting on this forum is dead.
[Response: not true. Predictions related to the impact of pinatubo, post 1984 trends, the ‘satellite cooling’ mismatch, lgm tropical sst, water vapor increases, ocean heat content etc have all been made and verified within a short time period. Admittedly not as short as a single day, but your claim it all lies beyond our lifetime is nonsense. – gavin]
.
Thomas,
One needs to be carefule before proclaiming victory. Early on, only the ECMWF forecast the left turn into the Mid-Atlantic states. The other models were predicting that the storm would continue on the more typical path out to sea. The closer the storm came to landfall, the more the models converged. This is typical modelling short-term events; the closer the event is to occurring, the more likely the models are to converge and predict accurately. Check out the comparitive forecasts between ECMWF and GFS from 8 days and 5 days out:
http://theweathercentre.blogspot.com/2012/10/east-coast-may-be-threatened-by-sandy.html
http://hurricanetrack.com/tag/ecmwf/
#30 & #45–It seems to me, this morning at least, that one of the most crucial defining characteristics of all this is the will to believe what one wishes were true–aka, “argument from consequences” and “intellectual dishonesty.”
Kevin,
So true. This occurs more frequently that most people care to admit. The election was a microcosm of the rest of the world, where Obama supported followed the 538 and democrat-favoring polls, while the Romeny supported gravitated towards Rasmussen and the like. Different groups will even pull out parts of a report showing their own viewpoint, while opponents point to other portions. Objectivity gets lost in polarization.
One important point that should be noted: Nate created his model before the election season began, and then left it alone. He did not indulge in post-hoc tinkering.
And that’s how he was able to make a 100% correct electoral college call. It’s also what gave his predictions weight, and absolved him of partisanship.