Performativity and Models (barely scratching the surface)

Andrei Boutyline mentions something important in the comments to the Krugman on Modeling post.

“…another line of criticism of formal models should be mentioned too: the performativity thesis. I am not sure I can do it justice, but as far as I understand the performativity thesis, it claims that, if powerful enough actors adopt certain simplifying models of understanding the world, the modeled actors will modify their behavior to better fit the model. The clearest example of thesis I’ve encountered is with the introduction of rankings of law schools (here, I may be wrecking an argument of Wendy Espland’s). The rankings intended to capture the criteria of law schools that the schools valued, but had to make simplifying assumptions to focus on common quantifiable elements. This had the effect of creating a strong incentive for schools to focus on specifically those areas–eventually at the cost of the unquantified ones. So, the introduction of simplifying (in this case, not into policy but into incentive structures) had the effect of transforming the world. I think this may be a better depiction of what Krugman’s “humanists” actually fear about these models.

The concept of performativity is a can of worms, but it needed to be brought up, even if I can’t say everything that needs to be said about it in one blog post.  There is much to like about Espland’s article, academic rankings are fine example of how something “intended” to describe the world, can change it.  Interestingly, widespread knowledge or use of a mathematical model, or an individual’s overconfidence in a model, can make the model more accurate, or make it less accurate, or both in different ways depending on the context.  But all this is true of ideas that are less formal than mathematical models.  In fact, academic rankings are not a mathematical model in the sense that Krugman or I am talking about.  Perhaps people using mathematical or statistical models are more prone to overconfidence, but I’m not sure…

A discussion of this topic with economists requires mention of Keynesian beauty contests, and the Lucas Critique.  The former is used to help explain bubbles.  The latter is used to argue that macroeconomic models require microfoundations, in other words, more (not less) formal mathematical modeling, to complement and combine with statistical analysis.

Of course, sociologists have been thinking along these lines long before performativity became a popular buzzword.  As W.I. Thomas said, “If men define situations as real, they are real in their consequences.”