So you want to be a superforecaster?

On my travels to and from the fabulous Kilkenomics Festival this weekend, I read by Philip Tetlock and Dan Gardner. It’s a very interesting book. If you liked Daniel Kahenman’s , Nate Silver’s and Gerd Gigerenzer’s , you’ll like this book too.

[amazon_image id=”184794714X” link=”true” target=”_blank” size=”medium” ]Superforecasting: The Art and Science of Prediction[/amazon_image]

Essentially the book describes the outcome of the research project that followed on from Tetlock’s famous demonstration in that experts are not good at predictions. One group of experts failed to do better than random guesswork – and did worse for long term forecasts. Another group did better than this but could rarely beat a simple rule of thumb such as ‘predict no change’ or ‘extrapolate the trend’. The follow up project was stimulated by the introspection of the US intelligence community post 9/11, looking to see whether it would be possible to make forecasts of political or economic events any better than the earlier dispiriting results suggested.

The bulk of the book explains that yes, it is, but it’s hard work requiring techniques and habits that help people avoid the normal cognitive short cuts (‘fast thinking’) we humans take. For example, make sure you start with what the authors call ‘the outside view’. Looking at the drop in popularity of a prime minister after an election? Start out by seeing what has happened to the ratings in the past. Watch out for the ‘bait and switch’ habit of answering an easy question rather than the hard one. Break up complex questions into smaller questions to narrow the territory of your ignorance. Take as many different perspectives as you can. Consult others and welcome diverse views – be on the alert for groupthink.  Be prepared to change your mind. Be alert to conclusions based on your strong feelings or beliefs about an issue (the Keynesians vs Austerians debate is singled out as one where the participants are captive to their prior beliefs). All of the tips are gathered in a how-to-be-a-superforecaster appendix to the book.

Summarised like this, it sounds obvious perhaps. But the book is stuffed with examples demonstrating how hard it is to put the advice into practice. Indeed, the experiment showed that some people can do far, far better than the majority, including ‘experts’ – but there are not many of them. They have specific characteristics: for example, clever (but not Mensa geniuses), open-minded, self-critical, numerate, comfortable with probabilities, willing to change their mind – plus determination. Tetlock is still looking for volunteers to have a go (

The book ends with a discussion of two critiques. One is whether Nassim Taleb’s Black Swans imply superforecasting is a chimera – if history moves in jumps because a black swan appears, that must be unforecastable. The other is Daniel Kahneman’s hypothesis that even superforecasters will lose their mojo because their very success will make them as vulnerable to the same cognitive patterns as the ‘experts’ who did no better than randomness or algorithms; we are all vulnerable to complacency or ‘fast thinking’. Tetlock concludes that history does show that the possibilities for the future are radically open but nevertheless argues that his results show that: “People can, with considerable effort, make accurate forecasts about at least some developments that really do matter.”

He does, for me, describe his results convincingly, although the ‘considerable effort’ bit seems a huge barrier to seeing superforecasting habits spreading more widely. But it doesn’t matter if you end up agreeing more with the critiques; this is still a very useful guide to cultivating your own good cognitive habits and critical thinking abilities. I’ll be trying to put the lessons here into practice myself – although not to the extent of starting anything foolish like macroeconomic forecasting.


What’s new about nudges?

As I prepare to lecture on behavioural economics in public policy tomorrow, I’ve been riffling through the books I have in the room here & am struck by how similar they all are. The most recent is Richard Thaler’s , which is as the subtitle suggests about the debate within economics about foundational assumptions as well as the content of behavioural economics. It’s very good and accessible. Daniel Kahneman’s Thinking Fast and Slow is still my favourite. There’s , of course, by Thaler and Sunstein. The Dan Ariely books – I have here – are jolly reads. Mullanaithan and Shafir’s is an important book. A more scholarly volume is Eldar Shafir’s edited volume . Somewhere too in here is Colin Camerer’s . And I have by Julian LeGrand and Bill New, somewhat cautious in its embrace of behavioural policies.

[amazon_image id=”1846144035″ link=”true” target=”_blank” size=”medium” ]Misbehaving: The Making of Behavioural Economics[/amazon_image]  [amazon_image id=”0141040017″ link=”true” target=”_blank” size=”medium” ]Nudge: Improving Decisions About Health, Wealth and Happiness[/amazon_image]  [amazon_image id=”0007256531″ link=”true” target=”_blank” size=”medium” ]Predictably Irrational: The Hidden Forces that Shape Our Decisions[/amazon_image]  [amazon_image id=”0141049197″ link=”true” target=”_blank” size=”medium” ]Scarcity: The True Cost of Not Having Enough[/amazon_image]  [amazon_image id=”B00P6ZJ6LS” link=”true” target=”_blank” size=”medium” ]Government Paternalism: Nanny State or Helpful Friend?[/amazon_image]

The thing that strikes me looking at these volumes covering a dozen years is how much overlap there is in the material they cover, often exactly the same examples. Does this mean behavioural economics has not really moved on from the central insight about people’s inability to calculate probabilities as if they were Mr Spock from Star Trek, and hence the “irrationality” of much decision-making concerning uncertainty and predictions of the future? The big questions I have  – and can’t readily find the answers to – are:

1. What do we know about the boundaries between situations in which people are “rational” and “irrational” (ie individual choices lead to the market outcome as predicted by conventional economic theory, or not) – is it as simple as any situation involving having to predict the future?

2. What do we know systematically about the interaction between individual decisions and social influencing (other than that it exists)?

3. What do we know about whether people adjust over time to ‘nudge’ policies and change their behaviour, as they do to any traditional economic policies?

If those who have read more of the behavioural literature know where I can find relevant material, I’d be very grateful.


Behavioural economics, 1892 version

Well I’ve read Paul Mason’s and can’t write about it because I’ve agreed to do a review elsewhere, and I’ve read Dani Rodrik’s and can’t write about that because it’s embargoed until the autumn.

[amazon_image id=”1846147387″ link=”true” target=”_blank” size=”medium” ]PostCapitalism: A Guide to Our Future[/amazon_image]  [amazon_image id=”0393246418″ link=”true” target=”_blank” size=”medium” ]Economics Rules: The Rights and Wrongs of the Dismal Science[/amazon_image]

So I’m going to offer instead another thought about by Alfred and Mary Marshall, which has been my bedtime reading for a few days. One of the striking features is that so many of what seem to be recent insights in economics are there in Marshall, in crystal clear English. For example:

“An increase of income nearly always causes pleasure; but the new enjoyments which it provides often lose quickly much of their charm. Partly this is the result of familiarity, which makes people cease to derive much pleasure from accustomed comforts and luxuries, though they suffer great pain from their loss.”

Doesn’t this sound just like modern behavioural economics?

[amazon_image id=”1932512136″ link=”true” target=”_blank” size=”medium” ]Elements of Economics of Industry[/amazon_image]

I noticed Brad DeLong coincidentally also writing about Marshall – his earlier Economics of Industry with Mary Marshall.


Be happy! (Or else…)

It has been a busy and dyspeptic week. by William Davies has been the perfect accompanying reading material. The enthusiasm of many of my fellow economists for behavioural economics has made me increasingly uneasy. This is, after all, a profession strongly  inclined towards social engineering, and I’ve written here before about the likelihood that nudges are seen as an exciting new tool for this job. After all, they seem to work, and as even the status quo is a nudge, why wouldn’t you design better nudges to deliver better outcomes?

[amazon_image id=”1781688451″ link=”true” target=”_blank” size=”medium” ]The Happiness Industry: How the Government and Big Business Sold us Well-Being[/amazon_image]

eloquently reinforces my suspicions. It locates the fashion for “well-being” in the long tradition of making the internal world measurable and reducing questions of morality and political choices to scientific decisions. Economics, rooted in , plays a leading role in the story, as do the successive waves of management science from Taylorism on. Indeed, in management, the growing surveillance of employees’ ‘well-being’ by wearable devices is the latest version.

Davies points out there is an inconsistency at the heart of this: “Workplaces put a growing emphasis on community and psychological commitment, but against longer term trends towards atomization and insecurity. We have an economic model which mitigates against precisely the psychological attributes it depends upon.” Yet the emphasis on resilience or mindfulness puts all the onus on the individual to adjust: “one progressive route would involve changing [the] context. But another equivalent would be to focus on changing the way it is experienced.”

He is also critical of the economists’ use of the idea of revealed preference: that you can infer somebody’s inner preferences or desires from their choices, usually their choices about what to spend their money on. Shopping speaks louder than words. Perhaps wearables that can measure heart rate or sweat will replace money as the best revealed preference metric, but meanwhile what someone spends is a readily-measurable indicator, easier to count and compare than what people say about their emotions. “This granted money an exceptional psychological status, as it allowed others to peep into people’s private desires.”

Disliking money as a metric, Davies is therefore also critical, as many people have been, of using the technique of contingent valuation to put monetary values on, say, the impact of an environmental disaster. “What we witness in this sort of example is economics becoming used as a basis for broad public agreement well beyond the limits of the market place,” Davies writes. He’s in good company. and are among those who dislike the use of money as a measure of non-monetary values, such as nature, or relationhips, or civic virtue.

However, this seems to me distinct from the reductionism of the behavioural economists and psychologists. It is one answer to the question of how you resolve conflicts when there is no market: if you have to make interpersonal comparisons, how should you go about it? Or, in the words of a well known survey article, is some number better than no number? If you want to calculate compensation after an oil spill, how else could you go about it? So I am far more comfortable with these valuation techniques than I am with the happiness tendency.

On the latter, my instincts are with Davies: ” Behaviourism stretches Bentham’s dream of a scientific politics to its limit, imagining that beneath the illusion of individual freedom lie the cold mechanics of cause and effect, observable only to the expert eye.” When I teach my students behavioural economics – and they’re very interested in it – I ask them to look at this Adam Curtis blog, From Pigeon to Superman and Back Again. While not dismissing the policy sense of some nudges, beware economists who know how to make you happy and beware even more bosses demanding it of you.


(Ain’t) Misbehaving

Despite having read plenty of the behavioural economics books, of course I had to read by Richard Thaler, one of the first people to introduce and then popularise (through in particular) the introduction of psychological empiricism into economics. Nor do I regret it. It is a very good read. Although it goes over much familiar territory, it’s very interesting to read Thaler’s account of how a highly resistant discipline became accepting and then positively enthusiastic about behavioural models. Too enthusiastic – but more on that later.

[amazon_image id=”B00SSKM714″ link=”true” target=”_blank” size=”medium” ]Misbehaving: The Making of Behavioural Economics[/amazon_image]

combines a broadly chronological account of Thaler’s career and work with a highly accessible explanation of what behavioural economics is, how it differs from the previously conventional kind, and the evidence from psychology about how people make decisions. The book starts by explaining why economists had adopted an unrealistic model of rational choice, and why it made economics so powerful: “That power derives from the fact that economics has a unified, core theory from which nearly everything follows.” Certainly early resistance to ‘behavioural’ assumptions tended to be that these derived from an ad hoc list of patterns of choice with no theory behind them, never mind that rational choice is ad hoc with respect to the facts. This seems to be hard for some economists still to accept perhaps because – as Thaler recounts – economists make choices far more often in conformity with their own models than do other groups of people. Misbehaving tells of a survey conducted among wine connoisseurs designed to explore how people regard sunk costs and opportunity costs, in which the people who gave the ‘correct’ answer were economists.

The book has lots of examples that will be useful to people teaching behavioural economics, including classroom experiments. I also very much enjoyed all the anecdotes, like the story of a vigorous debate with Richard Posner at a conference on law and economics, or a session on behavioural finance that had smoke coming out of Merton Miller’s ears. Resistance among distinguished economics professors who had built their glittering careers on rational choice models is, of course, entirely rational. Less rational, more human, was the behaviour of a group of University of Chicago economics faculty in selecting their offices in a brand new building.

Behavioural economics is now one of the most popular areas of the subject, and seminars on behavioural papers are packed. Sometimes it seems pretty much everyone I know has a new paper applying behavioural insights to their own sub-field. Perhaps this is just me being contrarian, but the new embrace by economists makes me uneasy. This is not just because of the well-known debate about paternalism (as discussed by Gilles St Paul in or Julian LeGrand and Bill New in ) It is because the sight of economists delighting in a new tool to engineer society is alarming – it’s the same old reductionism in more fashionable clothes. I happened to read this morning this essay by historian Ian Beacock on Arnold Toynbee. This quotation jumped out: “We’ve begun to treat vexing social and political dilemmas as simple design flaws, mistakes to be rectified through a technocratic combination of data science and gadgetry.”

I’m 100% in favour of empiricism. Why would you not do ‘what works’? But the behavioural rules of thumb are in danger of being seen as a new policy gadget.

[amazon_image id=”0691128170″ link=”true” target=”_blank” size=”medium” ]The Tyranny of Utility: Behavioral Social Science and the Rise of Paternalism[/amazon_image]   [amazon_image id=”0691164371″ link=”true” target=”_blank” size=”medium” ]Government Paternalism: Nanny State or Helpful Friend?[/amazon_image]