This is the age of wisdom but some would say it is also the age of foolishness. Newspapers, television, radio and of course the Internet bombard us with information at an unprecedented rate, but when it comes to scientific issues the quality of the information is variable. Television doctors entice us with claims of breathtaking breakthroughs, global warming is hotly debated, evolution is questioned and the peer-reviewed literature, our supposed gold standard, brims with flawed studies.
The challenge is to separate fact from folly, foolishness from wisdom. The 2013 Lorne Trottier Public Science Symposium aims to do just that. The Symposium’s four invited speakers have all forged stellar careers based on separating the wheat from the chaff and will explore diverse areas of science as they answer the common question, “Is that a fact?”
As part of the Symposium, on Oct. 29, Dr. John Ioannidis, C. F. Rehnborg Professor in Disease Prevention at Stanford’s School of Medicine and one of the world’s foremost experts on the credibility of medical research will deliver the lecture “Improvements in Reproducible Research,” in which he will discuss serious flaws in much of today’s research and the measures that can be adopted to strengthen the reproducibility of research findings. Ioannidis’s PLoS Medicine paper, “Why Most Published Research Findings Are False,” is the most accessed article in the history of the Public Library of Science.
Ioannidis will be sharing the podium with Tim Caulfield who will deliver the lecture “The Truth About What Makes Us Healthy”
Oct. 29, 5:30 p.m. – 7:30 p.m.; Centre Mont-Royal (1000 Sherbrooke West). Free admission, no reservations are required. For more information on the Lorne Trottier Public Science Symposium, click here. You can also call 514-398-2852 or send an email.
How can most published medical-research findings be false, when the great majority are supervised by experienced scientists and vetted by other experts in the field?
For many research questions, the odds of success are slim when we start our scientific investigation, so even with the best efforts and best intentions, the chances of getting a conclusive, correct answer and making major discoveries are low. In other cases, biases and errors are very frequent and they increase substantially the rate of false inferences.
You’re a medical researcher. How have your colleagues reacted to your claim that most of their studies are flawed?
There has been a spectrum of positions taken. I think the majority of scientists understand now that this is a serious problem and they have also been convinced of its magnitude based on extensive empirical evidence that has been accumulated on very diverse disciplines. There are always some dissenters and debate about these issues with serious arguments and data is always welcome.
It’s been nearly a decade since your famous PLoS Medicine paper appeared. Do you think the situation has gotten any better since then?
There are several improvements that have been implemented or discussed/considered, including wider adoption of large-scale collaboration, more stringent statistical analyses, wider adoption of a replication culture, improvements in reporting of research, registration of studies and protocols, sharing of raw data and analysis codes, reproducibility checks, and many others. Not all of them have been widely adopted and there is wide variability across specialties and disciplines. It is likely that some fields are making much more progress than others.
How can the public possibly figure out which research findings are reliable and which aren’t, if even peer-reviewed academic journals get it wrong so often?
Often it is not easy for the average reader to understand what is happening. Some hints are to look at whether this is a first, small study or multiple, large studies have shown the same results; whether there is an experimental design (e.g. randomized trial) rather than observational data; whether there is obvious conflicts of interest (e.g. who sponsored the study); whether the data are publicly available or at least available to all scientists who want to check them; and whether the results are interesting for the sake of curiosity or they also claim that the average reader should act upon them and change his/her life. The vast majority of scientific results touted in media will not require immediate changes in one’s life. Given that science is the best that has ever happened to humans, public media have a very unique opportunity to sensitize their wide audiences to these issues and training them on how to interpret scientific results with due caution.