In recent months, findings from several studies—and the media attention they’ve garnered—have created a stir in the natural products industry.

The scenario has played out like this: Emerging research calls into question a mainstream supplement with seemingly impressive science behind it. Headlines declare that the once-wonderful nutrient is now, well, not so wonderful. Sales of the supplement in question fall off. Fellow scientists scramble to restore confidence, pointing out flaws in the new research. The media frenzy dies down. Consumers and retailers are left vaguely unsettled, but mostly reassured… that is, until the pattern repeats again. 

How does one distinguish good science from “bad science”? When armed with the proper knowledge, it is possible to distinguish a meaningful clinical study that has utilized rigorous methodology, from a potentially flawed one that has generated results that are questionable, or just plain false.

Here are six questions to ask to evaluate the strength of a particular piece of research:

1. Who funded the study? Corporations, non-profit organizations, and governmental agencies like the National Institutes of Health are the primary sources of funding for scientific research. While a corporate-funded study might seem suspect, corporations seeking to determine the efficacy of their own products have funded some of the most important nutrition research conducted to date. So don’t discount a study solely on that basis.

2. Who is doing the research? The affiliations of the researchers are typically found at the end of the study, near the references. Look for appropriate credentials and a lack of undue corporate influence.  For example, a supplement company seeking to evaluate the efficacy of their product can outsource the research to an independent lab, so as to avoid preferential bias when interpreting the outcome.

3. Is the study published in a peer-reviewed journal? An ad placed by a manufacturer in a trade journal might have the appearance of a well-executed study, but that does not make it sound. Credible research should always be published in a scientific journal that is “peer-reviewed.” Peer reviewed means that before publication, appropriate experts examined the research and determined it to be unbiased and credible.

4. What type of study is it? The “gold standard” for research is a study that is randomized, placebo-controlled, and double-blinded. This means that patients are randomly assigned to either a treatment group (receiving the nutrient being studied) or a control group (receiving a placebo or alternate standard of care). Neither the subjects nor the researchers know which group is receiving treatment and which is being used as a control.

5. Did the trial include a large enough sample size? “Sample size” refers to the number of patients who take part in the study. Large sample sizes are generally more representative of the general population and produce statistically stronger results. If a corporation funds the study, it’s also important that the study be conducted on an unaffiliated group of patients, and not on company employees.

6. Was the dosage adequate or the interpretation appropriate? The dose studied should not far exceed what is appropriate, nor should it be too low to have a meaningful clinical outcome.  Also, ensure that the study’s conclusion actually pertains to the intervention in question.

So, the next time the media circus comes to town over controversial new research, ask some questions before drawing any conclusions. The answers will determine the degree of confidence you can place in the study, and will help you make informed decisions about your own health, the products you carry, and the health of your customers. WF

Energized by a desire to unite public health initiatives with the growing field of Complementary and Alternative Medicine (CAM), Eliza Leggatt is a gifted communicator and health educator. Eliza draws upon a wealth of experience ranging from the United States Marine Corps to motherhood. During her time in the Marine Corps, Eliza earned a meritorious promotion upon graduating from recruit training in Parris Island, South Carolina. She was honored after graduating from the Naval War College in Newport, Rhode Island, where she earned a higher GPA than any previous graduate. She saw and experienced firsthand the prevalence and consequences of Post Traumatic Stress Disorder (PTSD) affecting our servicemen and women, and entered the civilian world with a renewed enthusiasm for educating about Natural Living. Intense in her zeal, yet gentle in her approach, Eliza empowers others on their health journeys with compassion, candor, and a unique ability to convey complex scientific principles in a down-to-earth, accessible manner.

Posted 4/16/14