It seems that every week there is a new study that correlates some food to bringing you better health or closer to death. How much trust should we put in each study, and how much certainty is there? Media reports on scientific studies often use the word “could” or “may.” “Coffee could reduce your risk of cancer.” “Tea may be linked to increased metabolism.” Vox created the following chart that shows discrepancy among studies in linking various foods to cancer.
Why is there so much disagreement among studies? Shouldn’t science be free from the ambiguity that we associate with literature? The science taught to us in our K-12 years is science that has been peer-reviewed and scrutinized for over a hundred years. This gives the illusion that all science is set in stone.
The purpose of scientific studies is to describe novel information. Any new discovery is then scrutinized by scientific peers, and only time will tell whether the results stand. We often hear about exciting new studies, but we never hear back on whether the results have stood their ground. This can lead to the public believing wrong information. An extreme example is the MMR vaccine controversy. In 1998, Andrew Wakefield claimed that the MMR vaccine was linked to autism by publishing in The Lancet, a British medical journal. The paper turned out to be fraudulent, eventually resulting in Wakefield losing his medical license.
The MMR vaccine controversy is an extreme case of false scientific data. Wakefield was found to have altered the data to support his conclusions. In reality, intentional data manipulation is rare, so why do so many studies like in the picture above have differing results. Honest scientific work, especially in areas like medicine, will result in conflicting conclusions.
The ideal way to perform science is to utilize a fully-controlled experiment where only one variable is tested while every other factor is kept constant. This is easy in theory, difficult in practice, and impossible in medicine. If we wanted to test whether increased consumption of red meat leads to increased risk of cancer, we would not be able to design an ideal experiment. The test subjects would be people, and we cannot control the variation among people. Normally, scientists will randomly divide their test subjects into several groups and give each group a different amount of red meat to eat. They will then assess the group’s incidence of cancer over a long period of time. Some people will have an inherently greater risk of cancer due to their home environment, diet, or genetics. Scientists cannot exactly control for this. By randomly dividing people into groups, they can minimize the effect of these uncontrollable variables. If each group has an equal amount of disturbance by the uncontrollable variables, they will essentially "cancel" each other out.
Because each scientific lab will be using a different random group of people, there will be different results. Sometimes, if a food or substance has a strong enough effect on some quality of health, many labs will obtain similar results. If it has a weak effect, there will be mixed results.
What does this mean for the average person who is seeking to benefit from scientific conclusions? Newer science is not necessarily more accurate than older science. Scientific conclusions that have stood the test of time and peer review are the ones most likely to be true.
