Nutrition

August 2, 2014

Protein Timing Revisited: The Importance of Objectivity in Drawing Evidence-Based Conclusions

The beauty of peer-reviewed research is that it’s self-correcting. Scientists publish papers and then the scientific community scrutinizes the methodologies and conclusions employed. When appropriate, criticism is levied on a study and discussion/debate ensues. This process should be welcomed by researchers; it’s what pushes science forward and enhances our understanding of a given topic.

As many of you know, an area of research that I’ve recently been pursuing is the effects of protein timing on muscle strength and hypertrophy. In 2013 I collaborated with my good friend and colleague Alan Aragon to publish a review on the topic titled, Nutrient Timing Revisited: is there a post-exercise anabolic window?. In short, our review concluded that while muscle is sensitized to anabolism in the post-workout period, current evidence does not seem to support the existence of a narrow “window of opportunity.” I discussed the practical implications of the review in a previous blog post.

The paper stirred a lot of controversy. After all, the concept of an anabolic window of opportunity for nutrient consumption had been somewhat of a sacred cow in the field. Until publication of our review it was widely taken as gospel that you had an hour or less to take advantage of this narrow window; if you missed the window, muscular gains would be compromised.

One of the staunchest proponents of the nutrient timing paradigm is Dr. John Ivy, a professor at the University of Texas and noted sports nutrition researcher. Dr. Ivy literally wrote the book on nutrient timing with publication of his popular text, “Nutrient Timing: The Future of Sports Nutrition” back in 2004 It therefore was no surprise that Dr. Ivy took issue with our paper in an article he published in the American Journal of Lifestyle Medicine.

I welcomed Dr. Ivy’s criticism of our paper. As stated, critical debate of research is what drives science forward. But debate is a two-way street. I felt his critique was heavily biased and that he cherry-picked research to substantiate his claims.

As such, I provided a point-by-point rebuttal to Dr. Ivy’s critique in this blog post. In an effort to be fair and balanced, I emailed Dr. Ivy and gave him the opportunity to write a response to my comments. I offered to post anything he wrote unedited on my blog; unfortunately he never responded to my offer.

Subsequently, I co-authored a meta-analysis on protein timing with Alan Aragon and another good friend and colleague, James Krieger, titled The effect of protein timing on muscle strength and hypertrophy: a meta-analysis. Results showed a small effect for protein timing on hypertrophy, but virtually the entire effect was explained by an increased protein intake in the timing group. I discussed the study in detail in this blog post.

This week Dr. Ivy took to the airwaves to levy criticism of our meta-analysis. Appearing on the Superhuman Radio Network, Dr. Ivy called our paper “flawed” and took several of our methods to task (the discussion of our paper comes in at about the 22 minute mark).

Suffice to say, it’s my view that Dr. Ivy’s criticisms lack merit. What follows is a point-by-point refutation of Dr. Ivy’s claims.

First, Dr. Ivy mentions that “good studies were eliminated from the analysis” In particular, he specifies two studies that didn’t make the cut because they did not provide enough protein for inclusion in the analysis. He goes on to states that “…these studies showed significant increases in strength and muscle mass, so I don’t know how you can say that the protein wasn’t effective.”

I’ll start by saying that meta-analysis is only as good as the data it analyzes. Thus, rigid inclusion/exclusion criteria must be established to focus the analysis on the topic at hand.

Our inclusion criteria mandated that studies had to provide subjects with at least 6 grams of essential amino acids (EAAs). This cutoff point was determined from research showing that a ~6 g dose is required produce a marked increase in net protein balance – double the magnitude compared to a 3 g dose. If we had allowed inclusion of lesser amounts of post-workout EAAs intake it could just has easily been claimed that any negative findings would be attributed to insufficient protein provision.

That said, Dr. Ivy makes a fair point here since some potentially relevant studies were omitted from analysis. So let’s look at the two studies he mentioned.

In perhaps the most heralded study by protein timing proponents, Esmarck et al. randomly assigned 13 elderly men (average age 74 years) to perform a resistance training protocol 3 days a week for 12 weeks. The only variation in the protocol was that subjects consumed 10 grams of protein (a combination of skimmed milk and soy protein) either immediately following or 2 hours after the exercise bout. Results showed that muscle cross sectional area and mean fiber area of the quadriceps increased by 7 and 22 %, respectively for the group that received protein immediately post-exercise while the group that delayed protein intake showed no increases in fiber hypertrophy. On the surface, these findings would appear to provide compelling evidence in support of a narrow anabolic window of opportunity. Nail-in-the-coffin evidence, right?

Not so fast.

It is highly curious that the delayed-intake group saw *no* gains in muscle growth over a period of 12 weeks regimented resistance training simply because they waited 2 hours to consume protein. Zero! Considering that virtually every resistance training study ever done shows significant hypertrophy in untrained subjects after 3 months of regular lifting, the results must be viewed with skepticism. Moreover, these results were achieved with a dose of just 10 grams of protein (including lower quality soy protein), which equates to ~3 grams of EAA –an amount that as mentioned promotes only half the increase in protein synthesis as our required 6 gram dose. Add to this the fact that elderly subjects tend to be protein insensitive and generally need a ~40 gram dose to fully stimulate muscle protein synthesis and the findings are even more suspect. I’ll also note that the study had a very small sample size (only 7 subjects in the immediate provision group and 6 in the delayed consumption group), limiting statistical power. All told, it’s hard to make a case that this is nail-in-coffin evidence in favor of protein timing.

The other study we omitted because protein intake fell below the 6 gram threshold was carried out by Holm et al., who evaluated the effects of protein timing in 29 postmenopausal women over 24 weeks of resistance training. After exercise, subjects consumed a supplement containing either 10 grams of protein or placebo in double-blind fashion. Results showed that group who received the protein dose after exercise displayed greater gains in lean body mass compared to the placebo group. Interestingly, MRI analysis conducted at the 12-week midway point showed no differences in hypertrophy between groups (unfortunately the MRI was not repeated at the end of the study so it can’t be determined whether results would have diverged over time). While on the surface this study does provide some support for a beneficial effect of timing, it is important to note that total daily protein intake was ~10 grams greater for the timed group, and this confounding variable could have been responsible for any differences in muscle mass as opposed to a direct impact from timing.

Dr. Ivy also neglects to mention that two other studies were ultimately omitted from our analysis because we were unable to obtain sufficient data to compute an effect size. One of these studies by Bemben et al. showed no differences from a timed dose of protein compared to a non-timed dose. The other study by Burk et al. actually found a superior response when protein was consumed early in the day and late in the evening as opposed to before and after a workout.

Considering all the above, it is a huge stretch to claim that our inclusion/exclusion criteria unfairly biased results. The two studies excluded because of low protein provision had substantial limitations that deserve real scrutiny, and when factoring in the other two studies excluded for lack of usable data the net effect is at best a washout.

Next, Dr. Ivy stated we included studies that were not truly looking at nutrient timing but rather looking at differences between supplements post-exercise. He states these studies didn’t eliminate supplements post-exercise they just used different types of supplements. To quote Dr. Ivy: “They used studies that did not restrict eating post-exercise. For example, they would use studies where one group was given a protein supplement and the other was given an isocaloric carbohydrate supplement.”

While Dr. Ivy’s basic point is true, I’m not clear as to what the beef is here? Our inclusion/exclusion criteria specified we’d include any and every randomized controlled study where at least one treatment group consumed a minimum of 6 g EAAs < 1 hour pre- and/or post-resistance exercise and at least one control group did not consume protein < 2 hours pre- and/or post-resistance exercise. It really doesn’t matter whether or not protein timing was the primary objective of the research hypothesis per se. As long as a study employed timing in the manner specified by inclusion/exclusion criteria, then it provides a valid basis for evaluating the effects of timing. If I'm missing something here I'd be happy to hear Dr. Ivy's rationale.

Dr. Ivy goes on to question our statistical methods. He states: “In a meta-analysis you do what’s called an effect size on each of the independent studies. There were certain studies like the one by Cribb and Hayes which have dramatic differences between the groups that had supplementation around the workout versus in the morning and evening. When I looked at the effect size I thought it would be huge but it wasn’t. So I’m not sure how they actually measured the effect size, which can affect the results as well.”

This is a curious comment as the statistical calculation of effect size is clearly stated in the methods section of our paper. The formula is plainly presented so that anyone can easily perform their own calculations. We’re certainly happy to address any discrepancies in statistical outcomes based on hard data that conflicts with we found. However, it’s unscientific to simply dismiss our findings because they don’t conform to individual expectations based on intuition.

Finally, Dr. Ivy insinuates that we included a lot of poor quality studies and states if you look at lean body mass gains in the “6 or 7 good studies” — the ones he considers to be “well controlled” – then a clear superiority emerges for protein timing.

While Dr. Ivy is certainly entitled to his opinion as to what represents a “good” study, I’ll note that we did a quality analysis using the PEDro scale. The average PEDro score of the studies was 8.7 (out of a possible 11) indicating a very high degree of quality. I have scrutinized the quality between studies showing an effect of protein timing versus those that did not, and see no major differences between the two. Again, if he feels that the pro-timing studies are of sufficiently higher quality than those showing no effect then he needs to provide specifics. It’s not enough to make a passing statement such as this without backing up claims with supporting evidence.

Final Thoughts
As mentioned in a previous post, I have a great deal of professional respect for Dr. Ivy. He is an esteemed researcher and has a long history of publishing quality studies. However, a researcher’s body of work doesn’t provide a mandate to accept his opinion as fact.

When scrutinizing research, it is imperative that we take an objective approach; personal bias should never enter into the analysis. If Dr. Ivy wishes to criticize our meta-analysis, he must do so in the context of the overall body of literature rather than with evidence that selectively supports his opinion. Otherwise, the critique comes across as a skewed attempt to discredit the study for the purpose of confirmation bias. I’ll again put the offer out there for Dr. Ivy to write a response, which I will publish unedited in its entirety.

I’ll conclude by noting that I will soon be collaborating on a protein timing study that should help to fill in some of the many gaps in the literature, with data collection set to begin next month. Neither I nor my co-authors have any stake in the topic. I’ll be happy to consider changing my opinion on the topic in the face of compelling evidence.

Stay tuned…

References
Bemben MG, Witten MS, Carter JM, Eliot KA, Knehans AW, Bemben DA. The effects of supplementation with creatine and protein on muscle strength following a traditional resistance training program in middle-aged and older men. J Nutr Health Aging. 2010 Feb;10(2):155–159

Børsheim E, Tipton KD, Wolf SE, Wolfe RR. Essential amino acids and muscle protein recovery from resistance exercise. Am J Physiol Endocrinol Metab. 2002 Oct;283(4):E648-57.

Esmarck B, Andersen JL, Olsen S, Richter EA, Mizuno M, Kjaer M. Timing of postexercise protein intake is important for muscle hypertrophy with resistance training in elderly humans. J Physiol. 2001 Aug 15;10(Pt 1):301–311

Holm L, Olesen JL, Matsumoto K, Doi T, Mizuno M, Alsted TJ. et al. Protein-containing nutrient supplementation following strength training enhances the effect on muscle mass, strength, and bone formation in postmenopausal women. J Appl Physiol. 2008 Jul;10(1):274–281