Statistical alchemy
 | This article includes a list of general references, but it lacks sufficient corresponding inline citations. (May 2016) |
Statistical alchemy was a term originated by John Maynard Keynes to describe econometrics in 1939.
The phrase has subsequently been used by Alvan Feinstein to describe meta-analysis. It is generally regarded as a deprecatory term which undermines attempts to present such activities as meeting the rigorous standards of science.
Econometrics
[edit]Keynes (1939) wrote a review of Jan Tinbergen's Statistical Testing of Business-Cycle Theories. Although he praised Tinbergen for his objectivity, he however depicted his methodology as "black magic" which he regarded as essentially untrustworthy. He was unpersuaded that "this brand of statistical alchemy is ripe to become a branch of science" (emphasis in the original).[1]
Often this metaphor is seen as a way of suggesting that econometricians were following a foolhardy pursuit comparable to the alchemical quest of turning base metal into gold. However G. M. P. Swann points out that Keynes was well aware that such eminent early scientists as Isaac Newton. He rather proposes a more nuanced interpretation of the metaphor as referring to the Alkahest, a universal solvent, which, it was claimed could turn stone into water. He claimed that by restricting econometrics to theory, mathematics and statistics, econometricians had discarded other important applied techniques. Although Ragnar Frisch had made warnings about this, these had been subsequently ignored by other econometricians who had ended up claiming that econometrics constituted a universal solvent.[2]: 25
Meta-analysis
[edit]Feinstein (1995) published "Meta-analysis: statistical alchemy for the 21st century" where he claimed that in meta-analysis scientific requirements had been removed or destroyed, eliminating the scientific requirements of reproducibility and precision. This was equivalent to a free lunch, comparable to the alchemical transmutation of base metals to gold. Detourning the adage concerning the combination of apples and oranges, Feinstein suggested that meta-analytic mixtures were so heterogeneous that they might be better described as "combining rotten fruits". He argues that meta-analysis violates the Bradford Hill criteria of consistency as inconsistencies are ignored or buried through the process of agglomerating the data.
References
[edit]- ^ Hendry, David (1980). "Econometrics-Alchemy or Science?". Economica. 47 (188): 387–406. CiteSeerX 10.1.1.683.2527. doi:10.2307/2553385. JSTOR 2553385.
- ^ Swann, G. M. P. (2006). Putting Econometrics in Its Place: A New Direction in Applied Economics. Cheltenham: Edward Elgar Publishing.
- Feinstein, A (Jan 1995). "Meta-analysis: statistical alchemy for the 21st century". Journal of Clinical Epidemiology. 48 (1): 71–9. doi:10.1016/0895-4356(94)00110-c. PMID 7853050.
- Keynes, J. M. (1939). "'Professor Tinbergen's method'". Economic Journal. 49 (195): 558–568. doi:10.1093/ej/49.195.558.