One of my favorite self-refuting statements is “There is not such things as truth.”

Is that true?

More seriously, once you start paying people to do things, they tend to do them with some enthusiasm. This is the disaster now internationally with academic work and publishing, where careers are made by quantity publishing and there is little quality evaluation. Of course, there is this absurd mythology of “refereed journals” that Americans and Brits go on about, but it is difficult not to laugh at their sincerity on the issue. OF COURSE 90% IS CRAP: WHY WOULDN’T IT BE? After all, nobody did anything to stop it!

The Ioannidis meta-study was covered in New Scientist magazine when it first appeared in 2005. It followed another survey by the Health Partners Research Foundation in Minneapolis, which sent questionnaires to researchers listed by the US National Institutes of Health. 3247 were returned. The results were published in Nature (vol 435, p 737). 15.5 per cent admitted changing their design, methodology or results to suit a sponsor, and 6 per cent admitted suppressing data. More than a quarter owned up to inadequate record keeping, and 10 per cent confessed to inappropriately giving credit to an author. See:

http://www.newscientist.com/article/mg18625033.400.html

In 2006, an Editorial in New Scientist lamented that corporate-sponsored research has undermined the standards of evidence-based medicine:

http://www.newscientist.com/channel/opinion/mg19025472.500.html

The peer-review system is subject to confirmatory bias too, as this 1977 experimental study demonstrated:

http://www.stern.nyu.edu/~wstarbuc/Writing/Prejud.htm

But what made me laugh out loud was the reader comment in this blog post about genetic bias.

http://www.nytimes.com/2007/09/16/magazine/16epidemiology-t.html?_r=1&ref=magazine&pagewanted=all

The following three quotes from the article are interesting.

“In January 2001, the British epidemiologists George Davey Smith and Shah Ebrahim, co-editors of The International Journal of Epidemiology, discussed this issue in an editorial titled ‘Epidemiology — Is It Time to Call It a Day?’ They noted that those few times that a randomized trial had been financed to test a hypothesis supported by results from these large observational studies, the hypothesis either failed the test or, at the very least, the test failed to confirm the hypothesis: antioxidants like vitamins E and C and beta carotene did not prevent heart disease, nor did eating copious fiber protect against colon cancer.”

“As John Bailar, an epidemiologist who is now at the National Academy of Science, once memorably phrased it, ‘The appropriate question is not whether there are uncertainties about epidemiologic data, rather, it is whether the uncertainties are so great that one cannot draw useful conclusions from the data.’ “

“Richard Peto, professor of medical statistics and epidemiology at Oxford University, phrases the nature of the conflict this way: ‘Epidemiology is so beautiful and provides such an important perspective on human life and death, but an incredible amount of rubbish is published,’ by which he means the results of observational studies that appear daily in the news media and often become the basis of public-health recommendations about what we should or should not do to promote our continued good health.”