Post-Script to ‘On Bernie, Dispositional Bias & Emergence’
due to a sort of critical mass of pareidolic convergence of things around issues & ideas in my previous post:
wanted to purge some of them by way of this follow-up post, cuz cartharsis
Nate & Tyler talk prediction, probability & this aberrant moment
this is a long (& very good) meandering discussion, touching on issues in my original post more or less obliquely at various points.
transcript here:
no1 knows wtf is going on, but pundits & politicians rarely ever have
(added bolding to facilitate skimming of long quotes)
“I’m not sure I fully understand politics right now,” the former Labour prime minister told The Guardian and The Financial Times.
Blair said there appeared to be a “desire to choose people who are going to rattle the cage” in both Britain and the United States.
“And it’s partly also about social media, which is itself a revolutionary phenomenon which can generate an enormous wave of enthusiasm at speed,” he said.
“When I first started in politics, these things took so long to build up momentum; your decision points were well before that moment was achieved. But it’s also a loss of faith in that strong, centrist progressive position and we’ve got to recover that.”
…
“One of the strangest things about politics at the moment — and I really mean it when I say I’m not sure I fully understand politics right now, which is an odd thing to say, having spent my life in it — is when you put the question of electability as a factor in your decision to nominate a leader, it’s how small the numbers are that this is the decisive factor.
(h/t WEMPE)
So are the predictors getting worse? Or is the political world getting more unpredictable? Both, although one should be cautious about assuming that there ever was a golden age of political predictability. First, on the predictors: changes in the technology of media and the status of celebrity are steadily driving forecasters further from accuracy, even as technology increases our assumption that accuracy should be possible. More than a decade ago, the professor Philip Tetlock, of the Wharton School, who specializes in decision-making and social and cultural psychology, published “Expert Political Judgment: How Good Is It? How Can We Know?” It was based on a long-term study of two hundred and eighty-four people who make their living “commenting or offering advice on political and economic trends.” He asked them a range of questions, such as “Would the United States go to war in the Persian Gulf?” He amassed more than eighty thousand predictions and then waited for history to yield a verdict. In his now-somewhat-famous conclusion, Tetlock reported that human beings who hold forth on the state of the world to come are, by and large, “poorer forecasters than dart-throwing monkeys.”
…
Since then, …those dynamics have become more pronounced. A predictor who confines himself to a one-year horizon would barely be able to forecast the results of Super Tuesday, much less the election, because the campaigns have lengthened; Ted Cruz entered the race in March, 2015, nearly six months earlier than the equivalent point, in September, 2003, when John Kerry announced his candidacy. (Why? Partly to raise more money.) Second, as more readers move to the Web, writers have adopted the click-bait pose of total certainty, instead of permitting qualifications and unknowns. (Tetlock’s advice? Beware of “experts who say ‘moreover’ more often than they say ‘however.’ ”) Moreover, the emphasis on celebrity has grown, too, as rising generations reports measurably greater desire to be famous than previous cohorts did.
But the element that has caught forecasters most off-guard this year has been a more fundamental change — a seeming decline in the predictability of politics because of a shift in how institutions shape outcomes. In an admirable explanation of where he and others went astray, Silver wrote that he was overly skeptical of Trump’s prospects because he “assumed that influential Republicans would do almost anything they could to prevent him from being nominated.” He relied on the presumption, once sound, that the Party itself would make decisive choices, in the form of endorsements and funding, that would bless some candidates and doom others. The parties, it turns out, are weaker and more out of touch than observers understood.
^NB here: centralized, top-down institutions (e.g. political parties) are inflexible & inorganic, complicated but not complex, artificially stabilized by force, & thus fragile. The fragility is masked, and compounded, by the artificial (& temporary) braces of hierarchical scale, bureaucracy, concentrated wealth & political power.
A human leg left in a supportive brace for too long will suffer muscle atrophy & loss of bone density from deprivation of the regular small stressors & variability which build muscle strength & bone stability. After a while, a level of stress which would ultimate strengthen a healthy leg will snap the atrophied leg. The brace was counterproductive (even if necessary for some length of time).
Similarly, an organization comprised of organisms but buffeted by such artificial braces (& thus deprived of regular stresses, failures & reformations), will eventually succumb to stresses to which it can not healthily respond due to inflexibility & inexperience with stress-response. An emergent wave of organic volatility can snap it like the over-supported leg.
Behold the Republican Party.
Worth noting that Philip Tetlock (whose work demonstrating the perennial incompetence of pundit predictions is quoted in the article above) has focused his more recent work on a small class of forecasters who can actually reliably outperform random odds in their predictions. ^this is a decent primer for that work. As is his appearance on Econtalk.
Berned Out
Since my original piece on Bernie etc, his campaign both achieved the ‘biggest primary polling upset in history’ & yet has fallen into a nearly insurmountable deficit in the primary race. Proving that, even as reliably reputable & dispassionate long range forecasts generally hold up , anomalous events & dislocations of record proportions must be expected in the current environment.
Recriminations like the embedded article above abound, but like most any emergent phenomenon in a complex system, it is likely overdetermined / multicausal, & not tractable by ordinary analysis.
I did come across one forecaster (^) who nailed the historically aberrant Michigan results like no other, using an ‘experimental’ statistical model relying heavily on social media & web data. Unfortunately (at least for his near-term fame & fortune prospects), the model badly missed the following week’s contests (at least in the cases where it diverged from polling averages, which was supposed to be it’s great competitive advantage re: the Michigan coup). Again, see previous 2 paragraphs.
Another forecast (^) was brought to my attention (again, h/t WEMPE), which was able to retroactively explain > 90% of the variance in Hillary Clinton’s vote share up through March 8 with “a simple model based on two predictors — the racial composition of the Democratic primary electorate and a dummy variable for region”. The model’s estimated standard error is +-5%, which is decisive in a close race, but less meaningful in terms of delegate distribution — and in any case, impressive for such a simple model. And while the actual March 15th results of NC and OH (vs the model’s forecast) both exceeded this error margin by about double, the average error across all March 15th contests was just within it (4.6 ppt).
Not bad, & lends more credence to the power of heuristics & simple rules in accounting for complex behavior (more on this below). But the model is possibly overfit & we’ll see how it holds up across regions (i wouldn’t bet more than lunch on a winner it picks with less than a 10% margin of victory).
The Sage Works in Mysterious Ways
as i was trawling youtube for lectures re: statistical pitfalls & misuse of models this past friday night, i clicked one titled Use and Misuse of Probability from the Abu Dhabi NYU campus. This and the black swan artwork alone shouldve been enough of a give-away, but i was still surprised to find that the lecturer was The Sage Himself — (surprised mostly cuz i thought i had exhausted the youtube quarry of his lectures).
besides the beauteous backdrop, Nassim’s customary sprawling lecture addresses myriad issues germane to this piece, this blog & my thought in general. but a few points are particularly salient here.
We had, luckily, a model error here. Visibly, my computer doesn’t work, so I’m gonna have to maneuver with slides. Which is fun because it can show the fragility of technology — that gains in efficiency always come with increases in fragility.
~clearly an aside, but a meaningful one & a genuine concern of Nassim’s, which foreshadows a deeper point to come.
~pertinent concepts re: ‘measurement’ (particularly of risk, but germane to probabilistic forecasts in general)
@ 14:30:
~Nassim’s pet concept of 2 classes of domains in the real world with regard to probabilities: Mediocristan & Extremistan
One I call Mediocristan because it’s the collective impact of the regular that determines the properties. In the second one, the exception plays a disproportionate role in determining the properties.
^ this is an important distinction because in Mediocristan, things are normally distributed & have linear responses (can be modeled by classical statistics etc), but in Extremistan outliers have a greater impact than common ‘median’ values & responses are non-linear (non-parametric, geometric, exponential, etc), thus intractable by classical statistics. Much trouble comes from believing a system belongs to the former when it really resides in the latter.
As Nassim describes around 21:50, complex systems such as most socio-economic domains, as it turns out, are more natural to Extremistan.
the bulk of the rest of the middle of the talk covers relevant but somewhat more technical issues regarding the Central Limit Theorem & where it applies or doesn’t, probability distributions and which are tractable by common statistical tests & assumptions, moral hazard, meta-probability as it relates to errors in measuring rare events, etc.
@ 56:45:
~ extension of the notion of Extremistan & ‘the 4th quadrant’ where unbounded exposure (i.e. unlimited risk) intersects with Extremistan (where rare events account for most of the impact on / behavior of the system, & thus statistics breakdown).
@ 1:02:30 (during Q&A):
~ crucial discussion of Gerd Gigerenzer’s research on heuristics & implicit/practical knowledge vs ‘rational’ / ‘propositional’ knowledge — with evidence that the former are more familiar with & robust to ‘4th quadrant’ dynamics, as it is the crucible in which they were forged & refined over generations, by experience. As promised, this relates to the above re: ‘simple rules’ & heuristics.
@ 1:15:12:
The internet has caused crises to be even — more fat-tailed than before, you can see it in the data…
…
I don’t like connectivity. Maybe people like it, it’s nice we’re all here but it’s not good for stability. Flat world is not good for stability of the world. … Flat world causes more of these winner-take-all fat tails.
…
^ Among the most crucially relevant to my prior post.
The following question & answer re: Libya is also directly on theme, as he describes even the major agents of that crisis ascribing 0 probability to the outcome which ended up taking place just a week later (when this lecture was filmed).
Then he wraps it all up nicely for us:
@ 1:17:45:
In a complex system with lots of interdependence… incidentally, a complex system has one rule that, to me, is central and why it generates fat tails. It’s because Central Limit doesn’t work, because one of the requirements of Central Limit Theorem is what? Independence. You don’t have independence.
If I buy a book and you buy a book [due to online rating or popularity increasing] you don’t have independence of observation, alright. So this interconnectedness is destroying independence. So it’s making random variables that peak, go much higher, and the other ones die. So you have positive feedback effects that are monstrous. So that lowers [predictability] monstrously.
(^followed up on @ 1:22:00)
ok think that about covers the minimum
