Tag Archives: bad science

Poor Study + Poor Reporting = Misinformed Public

Over the past week, several stories like this one have shown up in my Google News feed under “wine”.  At first, I ignored them, making the assumption that the NYT or some other bastion of reporting had taken a study out of context and it would die off, but it hasn’t.

The news stories all pretty much say the same thing.

Never mind those aerobics classes and boring raw vegetables. Normal-weight women may stay in shape by sipping a glass of Pinot Noir, if one long-term study is to be believed.

The research, reported by the Daily Mail, focused on nearly 20,000 women who were asked about their drinking habits. The researchers found that participants who drank one or two glasses of wine per day gained fewer pounds than women who drank soft drinks or mineral water.

I see several gigantic issues with the study based on this information, so I went to see if I could find the actual text of the paper.  I found it, but can only access the abstract (the article is on a pay to read site.)

Some of my issues with the study:

  • A survey for both alcohol consumption and weight?
  • How did they control for caloric intake?
  • Did they normalize anything (or take into account) other common factors among women wine drinkers, such as wealth, education, diet?

The abstract doesn’t say, so all we can assume is that these are issues, as they are not addressed (where smoking and geography are).  It does mention adjusting for non alcohol energy intake and physical activity, but I don’t see how you do that on a survey.  Self reporting is highly variable and unreliable.

It is nice to see that n is greater than say, 18, which explains in part the statistical significance.  However, statistical significance doesn’t mean causation.  To their credit, the authors of the study don’t make that claim in the conclusion, saying:

Compared with nondrinkers, initially normal-weight women who consumed a light to moderate amount of alcohol gained less weight and had a lower risk of becoming overweight and/or obese during 12.9 years of follow-up.

I wouldn’t be surprised at all to find a very well done study that says the same thing, but making the leap from that relationship to ‘wine makes you skinny’ is a big problem with the reporting on the story.

No reporter asks the very basic questions I just asked.  They just report that drinking wine will make you skinny because it’s science.  They also don’t address the serious questions of physiological explanation, only to say that there might be one.  so we end up with the story suggesting the study claims a relationship where it doesn’t, and no explanation of what the real implication or value of the study might be.

Fact is, you can not make the connection based on the information provided that drinking wine is related in any way to fitness, health, or weight gain.  Frankly, it’s far more likely wine consumption is a result of the causal factor as well.

This is all too common in both science and journalism, and leads to a misinformed public.  It is another example of failure of the press.


Bad Study

I don’t understand how this crap gets published.

Experts say a spectrum of natural sleeping and waking rhythms exists, ranging from extreme morning people to extreme “night owls.”

A new study examines how morning people compare with night owls on a strength test and looks at what other physiological processes may contribute to their performance.

Researchers tested participants’ leg muscle strength at various points in the day. They looked at nine “early birds” and nine “night owls,” who were classified as such based on a questionnaire.

Surprisingly, morning people’s strength tends to remain constant throughout the day, but night owls have peak performance in the evening, said researchers from the University of Alberta in Canada.

“We thought that morning people would be better at this in the morning, but they never changed,” said study co-author Olle Lagerquist, a Ph.D. candidate in neurophysiology at the University of Alberta.

n=18?? Self classified??  Please.  Absolutely meaningless, other than perhaps pointing out that this might be something that should be studied using a meaningful study.  This one, though, should have been completely ignored by the press.  One more lesson, I guess, that science you see in the mainstream press should be examined with a critical eye (global warming, HFCS, etc)

By the way, I went from being a night owl to morning person.  It wasn’t that hard, and made a pretty big difference in my productivity.  While I tend to lose steam in the afternoon (which is when I usually train, so I’m a big fan of Monster), I often perk up late at night and can be pretty creative. 

One other thing:  don’t let this kind of article keep you from finding what’s optimal for you.

Via Lifehacker

%d bloggers like this: