In times of increasing dynamic and common insecurity one is led to search for experts and order forecasts. That one enters thereby an uncertain field by itself, seems clear to everybody. However, only a few politicians and business leaders, the primary clients of forecasts, are aware of the fact how low the accuracy of forecasts actually is.
Philip E. Tetlock brought new insight and disenchantment to this topic. He asked more than 200 experts for their forecasts on political and economic developments, collected more than 80.000 data points and checked them 20 years later against what really happened. The picture, which came out, was extremely sobering.
The very most of the forecasts did not realize.
25% of the ones labelled as absolutely sure did not realize.
15% of the forecasts seen as absolutely unrealistic have become a reality nevertheless.
There were also enlightening insights on the experts themselves. Two types of experts could be identified. The first one, the foxes among the experts, take more details and information into account and see also nuances and ambiguities. The second group of experts, the hedgehogs, have one favored dogma or theory and process information more strongly in light of this particular perspective. It could be proven that foxes are delivering relatively more accurate forecasts as opposed to hedgehog experts.
In the media and the private consulting market, however, there is another dynamic at play. Here, the hedgehogs get more attention, namely those experts, who polarize with single-minded forecasts and put forward pointedly strong optimism or pessimism.
Tetlock not only uncovers the deficits in the expert world, but also works actively to improve the forecasting. For this, he founded the initiative “Good Judgement”: https://www.gjopen.com/
More to the results of the study in the book by Philip E. Tetlock „Expert Political Judgement. How Good is it? How can We Know?” (2006):
Link to a video with Philip E. Tetlock: