Party animal that I am, I’ve been spending my spare time in Stockholm this week (where I’ve been doing a couple of events for the Institutet for Naringslivsforskning) reading Oskar Morgenstern’s [amazon_link id=”B00177CAI0″ target=”_blank” ]On The Accuracy of Economic Observations[/amazon_link]. The 1963 edition of this 1950 book was just sent to me by a friend who had read a draft paper of mine that will be out quite soon, which refers to the book in a quotation from a recent paper by Charles Manski.
[amazon_image id=”0691003513″ link=”true” target=”_blank” size=”medium” ]On Accuracy of Economic Observations[/amazon_image]
As so often when you start researching something, it turns out there’s nothing completely new. Morgenstern wrote trenchantly about some of the things that have been bothering me about the way economists use economic statistics. For example: “A significant difference between the use of data in the natural and social sciences is that in the former the producer of observations is usually also their user.” Economic data on the other hand are collected by many hands, often consist of time series, and are processed by statisticians. Economists rarely think hard enough about their numbers – a problem made all the worse by the easy availability of statistics to download from handy websites and pour into a software package that will churn through regressions and generate test statistics.
Morgenstern also notes the strong incentives many ‘creators’ of economic data have to give misleading responses to survey questions. What’s your income? What price do you charge for this service, oh oligopoly provider? What is the level of your GDP, oh Greek government? “‘Strategic’ considerations play havoc with reliability.” Even if all respondents are well-intentioned, the sampling error, general mistakes in data collection and processing and so on mean we place a ludicrous amount of confidence in any and all economic statistics. “Three or four digits is probably the maximum accuracy of primary data that ever needs to be considered in the vast majority of economic arguments,” he wrote.
Gratifyingly, he likes the technique of spectral analysis of time series, which I did a lot of in my PhD days, because they arrange the data in frequency bands. “There can hardly be any doubt that the powerful new techniques of spectral analysis will put the study of economic fluctuations on a new basis,” the book says. Alas, it didn’t catch on except among a small number of time series anoraks. The chapter I’ve just finished concludes that economics still needs to learn to live with the fundamental importance of errors in the data, and include them in its theories – just as modern physics has. Alas, this too has not caught on among economists. All of which raise the question – why not? Why haven’t economic statistics improved, it seems, since 1960?
My favourite chart at the moment is the Bank of England’s GDP forecast in the Inflation Report. February’s showed a 90% chance that UK annual GDP growth was between 1% and 5% at the start of 2015, taking into account simply past experience of revisions of the data, and not any of the additional potential sources of error. This is the difference between the standard of living doubling in 70 years and 14 years, or in other words vast. Something to bear in mind for anyone making any last minute decisions about how to vote today on the basis of parties’ economic claims.
I dear to disagree (to an extent). Economic data did become better. To give a reacent example: the recent Eurostat ‘nowcasted’ HICP estimate is two weeks earlier than the official estimate and even more recently measures of core inflation have been added to the provisional estimate, the ‘nowcast’ seems to be of pretty good quality. While the official estimate is much faster than it used to be, too (seems a nice topic for a masters thesis: delve through 40 last week of november issues and using the statistical data at the end to check out when statistics became available, might become an interesting publication surely when (I believe) Friedman’s idea that statistics are notoriously late is used as a background). Natonal accounts nowadays also include sectoral data on wealth and debts, a breakthrough improvement which made the work of Piketty possible in the first place (though there of course *always* measurement errors and conceptual issues, hey, it’s science!). Also, monetary and GDP statistics are nowadays consistent while international comparison has improved – a lot. Labour market statistics are increasingly not just about the level of unemployment but also about flows of labour into and out of the labour force and into and out of work (these last data are often also included in textbooks). This is all about macro, of course, but these statistics did improve. Well, back to my sixteenth century Frisian farmers, tryting to improve my data on their lending and borrowing… (possible, by the way, because all kind of determined genealogists have put scores of transcribed sixteenth century juridical deeds on the net…).
I probably overstated the picture 🙂 but there does seem to me to be a valid point about why economists don’t take measurement error seriously whereas most scientists do. You know far, far more about national accounts data than many economists running regressions using the data.
Pingback: Why haven’t economic statistics improved? | Homines Economici
Good to see Morgenstern surfacing again, I liked many of his ideas. Data of course refers to what is past, if applied to what is present it may need adjustments but if used to attempt to predict the future has serious risks.
Good to see Morgenstern surfacing again, I liked many of his ideas. Data of course refers to what is past, if applied to what is present it may need adjustments but if used to attempt to predict the future has serious risks.