You’ll find an excellent talk at edge.org given by Dr. Phillip Tetlock on political forecasting. He’s a psychology professor from the University of Pennsylvania who is known for a book he wrote several years ago called Expert Political Judgement: How Good Is it? How Can We Know? He conducted a twenty-year study in which he interviewed 284 experts, everyone from Marxists to free-marketeers, journalists, economists, governmental officials, and professors, asking them to make political predictions about the future on a wide range of issues. How often were they right?
It turns out that their expert predictions are only slightly better than chance and sadly, even basic algorithms which simply extrapolate trends consistently beat them. During his talk he made a lot of great points.
There’s a question that I’ve been asking myself for nearly three decades now and trying to get a research handle on, and that is why is the quality of public debate so low and why is it that the quality often seems to deteriorate the more important the stakes get? … There is quite a bit of skepticism about political punditry, but there’s also a huge appetite for it. I was struck 30 years ago and I’m struck now by how little interest there is in holding political pundits who wield great influence accountable for predictions they make on important matters of public policy.
When hearing that, I leaned back in my chair and thought, “Yeah, well, they’re not in the business of offering objective information or helping us make informed decisions. They’re propagandists. They’re more like salespeople. No, take that back. They’re more like entertainers who cheerlead for a particular side.” Then a few minutes later he confirms my thoughts.
It’s easy for partisans to believe what they want to believe and political pundits are often more in the business of bolstering the prejudices of their audience than they are in trying to generate accurate predictions of the future. […] So, we found three basic things: many pundits were hardpressed to do better than chance, were overconfident, and were reluctant to change their minds in response to new evidence. That combination doesn’t exactly make for a flattering portrait of the punditocracy.
It’s nearly impossible to predict what the future will bring. Especially in the long term. Apparently political experts can make accurate short-term predictions, but we’re all blind as to what will happen in the distant future.
How do people react when they’re actually confronted with error? You get a huge range of reactions. Some people just don’t have any problem saying, “I was wrong. I need to rethink this or that assumption.” Generally, people don’t like to rethink really basic assumptions. […] If you have a theory how world politics works that can lead you to value avoiding one error more than the complementary error. You might say, “Well, it was really important to bail out this country because if we hadn’t it would have led to financial contagion. There was a risk of losing our money in the bailout, but the risk was offset because I thought the risk of contagion was substantial.” If you have a contagion theory of finance that theory will justify putting bailout money at risk. If you have a theory that the enemy is only going to grow bolder if you don’t act really strongly against it then you’re going to say, “Well, the worst mistake would have been to appease them so we hit them really hard. And even though that led to an expansion of the conflict it would have been far worse if we’d gone down the other path.” It’s very, very hard to pin them down and that’s why these types of level playing field forecasting tournaments can play a vital role in improving the quality of public debate.
I find myself wrong about politics all the time. I was against the GM bailout but that seemed to work out alright. We were paid back and millions of jobs were saved. I was honest about it and said, “Well, I was wrong.” But I could see a free-marketeer say, “Yeah, but that just kept a bad company in business. If we would’ve let them go under, a new, better company would have taken its place. We can’t pick winners and losers.” Then I think, “Yeah, OR a foreign competitor could’ve come in and taken over, leaving all those workers unemployed and in need of government assistance.” Here’s the problem – a lot of political and economic ideas are non-falsifiable. What would need to happen in order to convince you that you’re wrong? If the economy thrives then you’ll say, “See, the free market works!” But when things go south, and a bailout was successful, we’ll hear, “Well, it would’ve been even better had we left things alone.”
That’s the thing about politics – you can interpret the same event in so many different ways. Events are spun every which way, and when pundits and experts are wrong, nobody seems to care. I have a lot of respect for the social sciences, but as a scientist, a lot of the prediction making of political “experts” is no different to me than astrology. That goes for every side of the aisle. And don’t get me started on political blowhards.
Pundits and politicians have to give off this sage aura that they’re in control, that they know what’s going on, and that we need to follow them. They’re sleazeballs and their feigned confidence is divisive. I think it’s all bad theater.
I’m a concerned citizen who tries to stay informed, but it’s almost impossible to get good information. We drown in misleading polls, loaded statistics, and rhetoric. I remember once trying to learn what’s wrong with our healthcare system. You know what happens when you Google it? You get floods of articles from MSNBC, Fox News, and ABC News, all saying the same few things. None of them teach you how things work, or compare our system to what other countries are doing. I literally searched hundreds of pages. Who has time to sift through all that garbage? I eventually got fed up and quit. I was hoping to find a professor somewhere who has spent his life studying healthcare systems, unbiased and willing to teach me how it all works. By chance I stumbled upon a Coursera offering, a full course on what I was looking for. I plan to sign up but it hasn’t started yet. It’s called Health Policy and the Affordable Care Act. Finally, someone without an agenda who will teach me what’s going on in detail.
It’s hard to find quality information. As Noam Chomsky points out, media sources are not out to inform, they’re out there to make money.
You’d think with the internet we’d have an easier time finding information, but that’s not always the case. There are so many blogs and websites who just make stuff up, and news sources aren’t reliable either. It’s really no wonder why we have such trouble discussing important issues. The incentives for our leaders and media sources are all screwed up.