Donella Meadows’s book Thinking in Systems: A Primer (pp. 89-90) contains a rather interesting critique of econometric models and their limitations in explaining and predicting what happens in the world. While Meadows acknowledges that econometric models are more useful than what she calls “event-event analysis” (e.g. explaining event A, such as stocks going up, with event B, such as U.S. dollar falling), but notes that the fundamental limitation of models that strive to uncover statistical links between different types of flows (e.g. income, savings, investment, government spending, interest rates, output, etc.) is that, first, econometrics overemphasizes flows (because that’s where the most variation happens) and underestimate stocks (e.g. total physical capital), and second, there is no fundamental reason to expect that any flow bears a stable relationship to any other flow. While a statistical correlation can be detectable for a brief period of time, feedbacks and changes in the underlying system’s structure would make the econometric model worthless.

The example Meadows uses is trying to predict a temperature of a room based on correlations of heat flows in and out of the room, without knowing anything at all about how thermostats operate. One could probably easily find an equation that tells how the in- and outflows of heat have varied together in the past, because thermostat means that they are being governed by the same stock (temperature of the room). However, the equation would hold only as long as system’s structure changes. If someone opens a window or improves the insulation, or forgets to pay the heating bills, the equation would be worthless.

What’s more, the econometric result would tell little about how to change the system. It would only tell about system’s behavior and how they used to correlate with each other, but very little about the underlying structure.

Hence, Meadows asserts, such “behavior-based econometric models are pretty good at predicting the near-term performance of the economy, quite bad at predicting the longer-term performance, and terrible at telling one how to improve the performance of the economy” (p. 90).

References

Meadows, D. H. (2009). Thinking in Systems: A Primer. London: Earthscan.