Thursday, October 27, 2011

Not Science.

When you have to constantly "recalibrate" your model to get it to predict existing behavior you're not doing science or even understanding the underlying mechanism of the phenomena you're tyring to model. No, you're simply engaging in the type of thinking that if you make the bamboo plane a little bit bigger it'll attract John Frum and his silver eagles.

Poor modeling, it's not just for Global Warming anymore.


The problem, of course, is that while these different versions of the model might all match the historical data, they would in general generate different predictions going forward--and sure enough, his calibrated model produced terrible predictions compared to the "reality" originally generated by the perfect model. Calibration--a standard procedure used by all modelers in all fields, including finance--had rendered a perfect model seriously flawed. Though taken aback, he continued his study, and found that having even tiny flaws in the model or the historical data made the situation far worse. "As far as I can tell, you'd have exactly the same situation with any model that has to be calibrated," says Carter.

That financial models are plagued by calibration problems is no surprise to Wilmott--he notes that it has become routine for modelers in finance to simply keep recalibrating their models over and over again as the models continue to turn out bad predictions. "When you have to keep recalibrating a model, something is wrong with it," he says. "If you had to readjust the constant in Newton's law of gravity every time you got out of bed in the morning in order for it to agree with your scale, it wouldn't be much of a law But in finance they just keep on recalibrating and pretending that the models work."

And why yes, these are the exact same models that the political class uses to justify how they can run all the businesses in a country better than those that actually own and work in said companies.

Well, they use these models when they're not guided by their keen sense of social justice.

Via the Instapundit.

Update: Oh and also from Prof. Reynolds is this little article. For liberals, income inequality is the new global warming.

Well, Science says we have to do what our political masters tell us to.

No comments: