Hello Wine & Wellness Fans!
It's easy to get confused and sometimes paralyzed by many of today's wine and health consumer articles. The same publication or online journal often writes that wine is great for your heart one week 🥰, then the next week just one glass could cause heart issues😖!
No more! In this article, we show you how to identify quality studies in consumer health articles. We get nerdy 🤓 on details, but break down methods for qualifying studies into simple steps so YOU can decide what information is factual and what, if any, lifestyle changes you may want to make.
Here's to becoming your own best wine, wellness, and health guide!
Kathy and The Natural Wine School team
Journalism vs Science: How to Identify Quality Studies in Health Articles
By Kelly Scanlon, RD for The Natural Wine School
With all the conflicting health messages out there, it can be difficult to know just who to trust. Sources cited can sound very official, but how do you know if the information is factual and if it is, how far should you go to change your lifestyle based on this new information?
To begin, we must first understand a bit about journalism and science. Journalism has many goals such as informing the public of current events and news, sparking debate, and teaching us about the world around us.
But, another main goal of journalism is to entice readers to consume the articles they produce.
For blogs, websites, newspapers, and magazines to be competitive, they need to capture the attention of the audience and entice them to continue reading. Only those with the most followers/ readers reap the rewards.
Secondly, we must understand science, how “facts” come about, and why science messages seem to be changing day-to-day. Let’s start with a refresher on the scientific method we learned in school:
Step 1: Start with a Question. What is it you want to know or would like to find out?
Step 2: Research the topic to see what information already exists
Step 3: Form a hypothesis (make an educated guess what the outcome will be based
on the research you have available)
Step 4: Test your hypothesis by designing and performing an experiment. An
experiment is needed to make a discovery and to prove something as fact. This
is not the same as researching which is investigating materials to reach new
Step 5: Analyze your findings.
Step 6: Report your conclusions
to support existing information.
As you can see, determining a fact takes a lot of time, effort, and of course funding!
It may be a surprise, but the field of nutrition (including wine) is still in its infancy. Research is being done, but much of this work is still on the frontier and not yet conclusive.
In journalism, it’s a race to be the first to report the news, so journalists often report inconclusive findings and tidbits of information with creative liberties and since the headlines appear on familiar sites, they are often deemed trustworthy and taken as the truth.
Identifying Quality Studies
The most reliable research has gone through much testing which involves double-blind randomization and control groups. Double-blind is when neither the tester nor subject knows the treatment or intervention until the trial is over. Being human, it is only natural for us to have a bias that can heavily influence the interpretation of the experiment which is why this randomization is needed.
As this is the case, it is important to look at the Design of the study.
If you see Randomized Controlled Trial (RCTs), Meta-Analysis of RCTs, or Control Groups you can feel reassured it is higher quality evidence. If the studies are not random, such as in Case/Observational Studies (which don’t include interventions or experiments) they are less reliable.
But why are there so few good quality studies for nutrition?
Unfortunately, it would be nearly impossible for both the experimenter and experimented to be “blind” as even if you had a blindfold on, you are likely to know what you are eating or if you are drinking alcohol!
Thus, many conclusions in nutrition and health research are based on observational studies which are performed using diet recall (participants reporting what and how much they ate or drank- which can vary greatly!) and noticing what medical conditions they wind up with in the future.
It is hard to draw exact conclusions based on this as so many other factors such as genetics, environment, sleep, etc come into play.
Hierarchy of evidence pyramid. Reprinted from “Options for basing Dietary Reference Intakes DRIs) on chronic disease endpoints: report from a joint US-/Canadian-sponsored working group,” by EA. Yetley, 2017, Am J Clin Nutrition, page 11.
What to Look for in a Study
No matter which website you read, the best way to determine if the information is reliable is to read the source, the journal article itself. You don’t have to be a researcher to begin reading research articles.
In most cases, start with the abstract of the research paper (a general overview of the study and its purpose) and read the conclusion to get a simplified summary of the study. This is a great way to prime yourself into further scientific literature once you are more comfortable with the field.
There are a few questions you can ask, which will help you determine if the information is valid and applicable to you.
1) What is the design of the study?
a. Is it an RCT or Case-Control Study?
1. If RCT - Reliable Source of Information
2. If CCS - The information may be less reliable or overall inconclusive. Further
research is likely needed.
2) Who’s Who?
a. Who wrote it?
b. Who published it?
c. Who funded it? Those who pay, will often have an interest in the outcome or be
seeking a certain result.
3) What are the Results?
a. How large was the treatment effect? Was it a big group or just a few individuals?
b. How precise were the results?
4) Will the results help?
a. Do you share characteristics with the subjects of the study? Or are they a
different age, gender, and have different health conditions than you?
b. Were all the outcomes considered?
c. Are the benefits worth the harms or costs?
Let’s compare the following articles for practice:
The first article states that drinking alcohol may decrease patients’ chances of developing cataracts that require surgery, but the study does not “establish causation — only an association linking cataracts and alcohol consumption”. Essentially there seems to be a correlation but further research is required. The tone of the article is neutral, remains scientific, and points out the shortcoming of the study as well as a detailed summary of the results.
- of lower quality evidence;
- subject to bias; and
- has not yet undergone intervention that would lead to it become a scientific fact suitable for textbooks.
The larger the impact you think it could have on your life, the more you should read to expand your knowledge of the subject!
It can be a time-consuming process if you don’t understand terminology or statistics and you may need to look up words as you go along, but this will help you develop your analytical skills and in time you’ll get quicker.
And remember, if it is feeling outside your scope, it is best to seek an expert in the subject.