How do you know that the expensive supplement that you have purchased, or the new diet is helping your health?
Seen the latest on mushroom extracts or fish collagen? Big promises and big prices and all “scientifically validated”1,2. I won’t even mention the diet wars. How do you know if an intervention that is “backed by the latest research” will actually work for you? This is something I have been pondering from a personal and a clinical perspective. Saturated fat may clog your arteries,3 but dairy may prevent certain cancers in some people.4 What is going on in science? What is N-of-1 Citizen Science tracking anyway?
Science. The word itself evokes promise of the truth, of researchers pursuing effective treatments for the scourges of life. Science has been defined as “a system of knowledge covering general truths…as obtained and tested through scientific method;5” while the scientific method is the “pursuit of knowledge involving the recognition and formulation of a problem, the collection of data through observation and experiment and the formulation and testing of hypotheses.6”
Ahh, the promise of science and the secrets of life. Where else would a bright-eyed girl look but to medical school? How wonderful was the promise of science and the guidance by learned teachers into the world of molecules and life. Science and medicine that had respectively proven that the world was round, and that Consumption was caused by a microbe named Mycobacterium tuberculosis.
How Scientific Discoveries are Announced
After being peer-reviewed by a panel of scientists, most scientific research is published in academic journals. Once accepted the scientific paper often forms the basis of further hypotheses and scientific argument. The journals are then rated for relevance (impact factor) based on the number of times the papers in the journals were cited (referenced) by other papers. The published papers provide a vast body of information, that I search almost every day when trying to answer a question such as- do mushroom extracts improve brain health?
So, why do you need to be your own researcher?
Let’s look at a number of considerations, starting with the process of publishing medical research.
I would posit that most scientist are of good intent. They are generally employed by academic institutions, such as universities or industrial producers with the intention of researching particular hypotheses or producing cures. University researchers are usually required to publish a certain number of papers every year outlining their findings. Universities may also be partnered with industry and receive a lot of funding from these organizations. These institutions are then able to pay their researchers, get more published papers, attain further academic credence and enroll more students. Sweet; maybe.
Many of the publishers of medical journals are for-profit-businesses and charge a considerable sum to read each of the papers in their journals, limiting access to researchers in academic institutions!
The next consideration is bias. To suggest that bias is fully excluded in scientific methodology would be naïve.7 Even researchers acknowledge that “our talent for jumping to conclusions makes it all too easy to find false patterns in randomness, to ignore alternative explanations…to ceaselessly lead ourselves astray without realizing it.8” These same researchers are looking at research and data analysis techniques to mitigate this problem and in a future world, humans may be excluded from drawing conclusions from large data sets.8 However, we all need to admit that there is bias in science and in ourselves.
Now let’s look at funding and kudos. This takes the form of direct funding by organizations with financial interest in the outcome of the research and the personal cost to those scientists who are brave enough to go against the prevailing understanding, some would say dogma; think of Galileo – and yes, it is still happening today, albeit in a different guise9. The editor of Lancet, one of the oldest and most prestigious medical journals has stated: “The case against science is straightforward: much of the scientific literature, perhaps half, may simply be untrue. Afflicted by studies with small sample sizes, tiny effects, invalid exploratory analyses, and flagrant conflicts of interest, together with an obsession for pursuing fashionable trends of dubious importance, science has taken a turn towards darkness10.”
It has been said that if you read two clinical papers per day, by the end of the year, you will be years behind in your medical reading.11 Furthermore, most doctors have limited training in interpreting scientific papers, where trial teams often have their own statistician analyzing the vast amount of data collected. It is not always practical to thoroughly review the analysis and resulting conclusions.
Hence doctors generally take advice from guidelines and organized ongoing education. Guidelines and education are often compiled by professional organizations consisting of doctors and industry, who generally also rely on experts and specialists in a particular field to provide final approval. These specialists will also not be able to go through the vast amount of material in just their field and are reliant on other specialists, guidelines and meetings. While the days of the pharmaceutical rep are all but over, pharmaceutical (and supplement) companies spend billions of dollars on marketing through value-added online services, “facilitating links with clinical trials, journals and knowledge opinion leaders.12”
Types of Studies
Without going into too much detail, human clinical studies look at the effect of pharmaceuticals (as well as medical devices and surgery etc.) while epidemiological studies research patterns in populations.13
Epidemiological studies try to answer questions by comparing lifestyle, exposure to toxins and diet, to the risk of disease. These are generally the studies that look at lifestyle factors in chronic disease. For example, does eating vegetables prevent memory loss?14 There are numerous hurdles in these type of studies: they are often based on questionnaires (participants need to be truthful and have good memories) and the results may be misleading (people who eat more vegetables, also exercise more, so could it be the exercise and not the vegetables preventing memory loss)? Researchers attempt to adjust for these confounders, but what if there are confounders we don’t yet know about?
Clinical studies are easily applied to a particular intervention that is foreign to the body e.g. Exenatide for diabetes.15 Answering the question of whether Magnesium is useful for sleep16 is less clear cut. Why is this you ask – as you should?
- The Magnesium study was done as an interventional study – you give a compound and study the effect. Consider, however, that every human being already has some Magnesium in their body compared to no human being naturally having Exenatide.
- The researchers did adjust for known confounders, such as dietary intake of Magnesium. However, this would be affected by diet recall, quality of food and absorption of Magnesium amongst other factors; and levels of Magnesium where not tested.
With both types of studies, groups rather than individuals are studied. Taking the Exenatide example, it has been calculated that four people need to take the drug, for one person to benefit.17 This is known as the Number Needed to Treat.
So how do you know whether you are among the four people who do not respond? How do you know that changes you might see, are due to the intervention or some other effect (e.g. you started going to bed earlier as well as take the Magnesium?
Further, applying these types of methodology to well-being and chronic conditions ignores the fact the humans are not genetically identical and present with a myriad of varying environmental factors affecting genetic expression. In particular, Epidemiological studies need to be interpreted with care, so as not to “result in more harm than good.18”
It is not unreasonable to suggest that a spinning a wheel would produce an intervention for the individual that is just as useful as a scientific study. As medicine moves from treating mainly acute illness and injury to that of “unexplained” chronic disease and unwellness, an alternative model of inquiry and research is needed.
Taking Back Science and Democratizing Health
If the science of health and well-being is so conflicted, how does an individual and their health practitioner move forward?
N-of-1 trials in which there is only one person being studied are gaining interest as possibly “the ultimate strategy for individualizing medicine.19”
What if this was taken further? Consider a method whereby the individual is the scientist and the subject; apologies to all researchers who are shuddering at this heresy. My countering argument is that if we are all biased and if science has taken a “turn towards darkness”10, the way forward is with Citizen Science.
Technology might provide access to clinical experience (standing on the shoulder of the giants that have gone before us) and data science, usefully progressing the health of an individual. Notwithstanding the Heisenberg uncertainty principle (observer effect), specific outcome measures are now trackable by available technology and can be analyzed against user-defined interventions. Interventions would be based on known physiological pathways, clinical experience, clinical trials and population studies. This is already being undertaken with recent examples including Swarthmore College (www.mementolabs.io) and Salk Institute (https://mycircadianclock.org/).
Why track your own health data?
While not discounting the placebo effect, as so eloquently stated by Mathew Walker author of Why We Sleep, “One practice known to convert a healthy new habit into a permanent way of life is exposure to your own data.20”
While any health metric is trackable (weight, resting heart rate, fasting blood glucose, happiness, what level of Sudoku is completed), an overall health indicator, Heart Rate Variability (HRV) is now easily accessible and accurate.21 HRV can potentially be used to asses nutritional influences on health.22 More interestingly, HRV not only correlates with aging, but is a marker for health and wellbeing.23 This is indeed very exciting for this nerdy practitioner!
I am looking to reclaim the promise of science and the exploration of life. Medicine is changing and innovative new solutions are promising to by-pass the roadblocks separating patients and practitioners from their data and the outcomes they seek. Wearable medtech such as the Oura Ring are providing individuals with data on a level previously thought of as science-fiction. Online services are starting to bypass practitioners, promising a means for individuals to collate and track their own data, including that from wearables and functional testing.
N-of-1 Citizen Science Tracking
So, now you can answer the question: Does Mushroom Extract improve my brain function?
1. Liu D, Nikoo M, Boran G, Zhou P, Regenstein JM. Collagen and gelatin. Annual review of food science and technology. 2015;6:527-557.
2. Phan C-W, David P, Sabaratnam V. Edible and Medicinal Mushrooms: Emerging Brain Food for the Mitigation of Neurodegenerative Diseases. Journal of medicinal food. 2017;20(1):1-10.
3. Siri-Tarino PW, Sun Q, Hu FB, Krauss RM. Saturated fatty acids and risk of coronary heart disease: modulation by replacement nutrients. Curr Atheroscler Rep. 2010;12(6):384-390.
4. Jeyaraman MM, Abou-Setta AM, Grant L, et al. Dairy product consumption and development of cancer: an overview of reviews. BMJ open. 2019;9(1):e023625-e023625.
5. Science. Merriam-Webster Inc. The Merriam-Webstar.com Dictionary Web site. https://www.merriam-webster.com/dictionary/science. Accessed December 12, 2019.
6. Scientific method. Merriam-Webster Inc. The Merriam-Webster.com Dictionary Web site. https://www.merriam-webster.com/dictionary/scientific%20method. Accessed December 12, 2019.
7. Korteling JE, Brouwer A-M, Toet A. A Neural Network Framework for Cognitive Bias. Front Psychol. 2018;9:1561-1561.
8. Nuzzo R. How scientists fool themselves – and how they can stop. Nature. 2015;526(7572):182-185.
9. Stacey M. The Fall and Rise Of Kilmer McCully. The New York Times Magazine. https://www.nytimes.com/1997/08/10/magazine/the-fall-and-rise-of-kilmer-mccully.html. Published 1997. Accessed December 12, 2019, 2019.
10. Horton R. Offline: What is medicine’s 5 sigma? The Lancet. 2015;385(9976):1380.
11. Kendrick M. Dr Malcolm Kendrick: the True Causes of Heart Disease are Not What You Think [Internet]: The Fat Empoero; 2019. Podcast: 1:44
12. Stone K. Pharma Marketer Strategies to Reach Doctors. the balance. https://www.thebalance.com/pharma-market-strategies-to-reach-doctors-2663218. Published 2019. Accessed December 14, 2019, 2019.
13. Röhrig B, du Prel J-B, Wachtlin D, Blettner M. Types of study in medical research: part 3 of a series on evaluation of scientific publications. Dtsch Arztebl Int. 2009;106(15):262-268.
14. Wu L, Sun D, Tan Y. Intake of Fruit and Vegetables and the Incident Risk of Cognitive Disorders: A Systematic Review and Meta-Analysis of Cohort Studies. The journal of nutrition, health & aging. 2017;21(10):1284-1290.
15. Klonoff DC, Buse JB, Nielsen LL, et al. Exenatide effects on diabetes, obesity, cardiovascular risk factors and hepatic biomarkers in patients with type 2 diabetes treated for at least 3 years. Current medical research and opinion. 2008;24(1):275-286.
16. Abbasi B, Kimiagar M, Sadeghniiat K, Shirazi MM, Hedayati M, Rashidkhani B. The effect of magnesium supplementation on primary insomnia in elderly: A double-blind placebo-controlled clinical trial. J Res Med Sci. 2012;17(12):1161-1169.
17. Meloni AR, DeYoung MB, Han J, Best JH, Grimm M. Treatment of patients with type 2 diabetes with exenatide once weekly versus oral glucose-lowering medications or insulin glargine: achievement of glycemic and cardiovascular goals. Cardiovascular Diabetology. 2013;12(1):48.
18. Farmer R. The problems with some epidemiological studies. Maturitas. 2007;57(1):11-15.
19. Lillie EO, Patay B, Diamant J, Issell B, Topol EJ, Schork NJ. The n-of-1 clinical trial: the ultimate strategy for individualizing medicine? Per Med. 2011;8(2):161-173.
20. Walker M. Why We Sleep: Unlocking the Power of Sleep and Dreams. In: Penguin Books; 2017.
21. Kinnunen HO, Koskimäki H. 0312 The HRV Of The Ring – Comparison Of Nocturnal HR And HRV Between A Commercially Available Wearable Ring And ECG. Sleep. 2018;41(suppl_1):A120-A120.
22. Young HA, Benton D. Heart-rate variability: a biomarker to study the influence of nutrition on physiological and psychological health? Behav Pharmacol. 2018;29(2 and 3-Spec Issue):140-151.
23. Kemp AH, Koenig J, Thayer JF. From psychological moments to mortality: A multidisciplinary synthesis on heart rate variability spanning the continuum of time. Neurosci Biobehav Rev. 2017;83:547-567.