The increasing use of algorithmic decision making raises some challenging questions, from bias due to societal biases being baked in to training data, to the loss of the space for compromise (due to the need to codify a loss or reward function in a machine learning system) that is so important in addressing conflicting aims and values in democratic societies. The broader role of models as a means of both understanding and shaping societys is one of the themes of my most recent book, Cogs and Monsters, in the domain of economics. In particular, I wanted to expose in economic modelling the reflexivity involved in being a member of a society analysing the society in order to try to change the society – when its other members may well alter (in often-unanticipated ways) the behaviour that was analysed – because they are subjects, not objects.
Well, all of this is the subject of Erica Thompson’s excellent book Escape from Model Land: how matehmatical models lead us astray and what we can do about it. It focuses on the use of algorithmic models, and has a wide range of examples, from health and epidemiological modelling to climate projections to financial markets. It covers as well as the reflexivity some familiar challenges such as performativity, non-linear dynamics, complex systems, structural breaks and risk vs ‘radical’ uncertainty. The ultimate conclusion is the need to be duly humble about what models can achieve. Alas, people – ‘experts’ – seem to all too often get caught up in the excitement about technical possibilities without the thoughtfulness needed to make decisions that will affect people’s lives in important ways.