“To the degree big data can seem magical it can also be spectacularly misleading. Is this not clear? Perceiving magic is precisely the same thing as perceiving the limits of your own understanding.
“When correlation is mistaken for understanding, we pay a heavy price. An example of this type of failure was the string of early 21st century financial crises in which correlations created gigantic investment packages that turned out to be duds in aggregate, bringing the world to indebtedness and austerity. Yet few financiers were blamed, at least in part because the schemes were complex and automated to such a high degree.
“Naturally, one might ask why big business data is still so often used on faith, even after it has failed spectacularly. The answer is of course that big business data happens to facilitate superquick and vast near-term accumulations of wealth and influence.”