Tag Archives: Low Carb Daily

Lies, Damn Lies and Statistics

The title of this article refers to the phrase Mark Twain popularized, that he attributed to Disraeli. There is a touch of irony in that no one can actually pinpoint where Disraeli said any such thing. But you get the point.

Sometimes, even when its good news for low carbers, the stats are manipulated in such a way to mask the true advantage.

Dietary science is the bastard son of science, with correlation in cohort studies trumpeted as “proof” by the media. A randomized controlled study is a better model as it helps eliminate various types of bias.

We were all excited to see the new study that compared low fat to low carb dieting, and found that cardiovascular risk factors were not greater with a low carb diet … in fact, they were lower! And this was a randomized controlled trial, with over 300 participants who were followed over a two year period; that’s a very long time in dietary science!

But in scouring the web for more authoritative voices to add to our new low carb news aggregator LowCarbDaily.com, I found something shocking. From a layman’s perspective, the study is horribly flawed, and grossly underestimates the beneficial impact of a low carb diet. How so? The researchers used a statistical method called “Intention To Treat” (ITT). This method uses the results from all the participants, even those that drop out, diluting the true impact of the diet on those that continue to follow it. As Pål Jåbekk writes in the Ramblings of a Carnivore blog:

Now, intent to treat analysis is a perfectly fair method to use. But it means that the results cannot tell us which dietary approach is the more effective. If we want to know which diet causes the greatest weight loss we must look at the data from the participants that actually followed the diet, and only those. What the results of this recent study tells us, is the effect of being put on a diet as opposed to the effect of following one. Is it really so bloody impossible to include data on compliers vs non compliers? As interesting as it is to know the effect of being put on a diet I for one would also like to know the actual effect of following the diets.

Jåbekk links to a paper by Richard D Feinman on ITT where the curious practice is further explained:

In ITT, the data from all subjects who are randomized to treatment are analyzed regardless of whether subjects followed the protocol or not (“analyze as randomized”). At first hearing, the idea of ITT is counter-intuitive if not completely irrational – why would you include in your data, people who are not in the experiment? – suggesting that a substantial burden of proof rests with those who want to employ it. No such obligation is usually met and, particularly in nutrition studies, such as comparisons of isocaloric weight loss diets, ITT is frequently used without justification. ITT analyses are typically reported in a way that implies that they have the final say on efficacy and it is even argued that, once assigned to an experimental group, all data must be included in the analysis even if subjects do not comply with the protocol.

Feinman’s article includes this table, showing how ITT masks the true response of those that follow a low carb diet in two prior studies:

Table 1

Weight Loss in Diet Comparisons and the Effect of Analysis.

Data for 12 months Weight Loss (kg)

With Drop-outs SD Only Study Subjects SD

Foster, et al. low carb 4.4 6.7 7.3 7.3
low fat 2.5 6.3 4.5 7.9
difference 1.9 2.8

Stern, et al. low carb 5.1 8.7 7.3 8.3
low fat 3.1 8.4 3.7 7.7
difference 2 3.6

Feinman Nutrition & Metabolism 2009 6:1   doi:10.1186/1743-7075-6-1

Keep in mind that in these prior studies, the end result reported in the paper was that weight loss was just about equal with either a low carb or low fat diet. That’s the message if you use the numbers that include the people who dropped out of the diet, in the “With Drop Outs” column, where the low carb group in the “Foster, et. al.” study only lost 1.9 kg more than the low fat group. But look at the column titled “Only Study Subjects”, comparing those that actually followed the low carb or low fat diet, and you find that the low carb dieters actually lost 2.8 kg more than the low fat dieters (47% more weight). For the “Stern, et. al.” study, we find even greater numbers: a difference of 2 kg between the diets using ITT and 3.6 kg when counting those that actually followed the diet plans. That’s 80% more weight loss. Feinman continues:

Thus, the conclusion that weight loss is the same at 1 year on low-carbohydrate diets and conventional diets comes from an ITT analysis and, as stated, is misleading. Because dieters and practitioners reasonably want to know the potential of a diet, it seems that authors must be very circumspect about describing results. The ITT analysis, again, only answers the question about assignment to a diet in a particular experimental setting, and does not address the question as to which is the more effective diet if adhered to. The fact that it is acknowledged that the substantially greater improvement in plasma triglycerides on the low carbohydrate diet compared to the low fat diets persisted for one year should have been taken as a sign that it would be surprising if the diets were the same.

While the news about the good results was welcome, it is disheartening to see that the results are still not being reported in an easy to understand manner. It appears the new study did not include the results only from the people who adhered to the diet, but only reported the “Intention to Treat” numbers.

New Site: Low Carb Daily

Low Carb Daily is a new concept in Low Carb news; an aggregator that previews the most recent stories in the Low Carb blogosphere. The entire article is not reprinted, just the headline and lead paragraph. Links to the original article allow you to click through and read what interests you, on the original author’s site. This provides the easiest way to review the dozens of personal stories, articles and news generated every day in the Low Carb world.

Several features make the site unique:

  • Site Search: Search the latest articles for any current topic
  • MegaSearch: Search all Contributor’s Sites, and only their sites, for any topic, past or present
  • Categories: Sort all articles by the author’s category

The front page always displays the most recent articles, but you can sort the collection several different ways. The search method works, of course, but you can also display articles written by members of a particular category.

There are six categories based on the author. Advocates are low carb enthusiasts who contribute a tremendous amount of information to the low carb world. The Low Carb News category tracks the latest news in popular media about low carb topics, direct from Google News. Medical Blogs presents low carb articles written by medical doctors. In a similar vein, the Researchers category includes those with doctorates, or other advanced degrees. Personal Sites presents the posts by bloggers on their own low carb journey and Recipes shows you the latest from the low carb recipe sites.

Low Carb Daily also features a new Forum, not tied to a specific company or organization. Charter members can help design and build the community as an independent forum. As the site is brand new, there is no activity there yet.

Low Carb Daily is ad-supported, with free membership. Membership is only required to post in the Forum, but all of the rest of the site’s features can be enjoyed anonymously. Advertising on the site is intended to be non-intrusive and clearly marked. A Google ad is placed at the top of the page, and an Amazon Associate store features books and products for low carbers. Two ads appear in the sidebars for products from the store, clearly marked as “Shop!” items.

Low Carb Daily is sponsored by Low Carb Age and HOSTkabob.