The ketogenic diet and dodgy systematic reviews
Writing about web page http://www.ncbi.nlm.nih.gov/pubmed?term=henderson%20CB%20ketogenic%20diet
I recently came across studies of the ketogenic diet for treating childhood epilepsy. This is a high fat, low carbohydrate diet that mimics starvation and increases the level of ketones in the blood, which is thought to reduce seizures in some way. Nobody has a clear idea of how this might work, but the diet is widely used for epilepsy that does not respond to drugs, perhaps largely because of a lack of other treatment options. The origin of the ketogenic diet seems rather murky, as it was first promoted in the 1920s, building on the idea that fasting could control seizures. The lack of theoretical basis doesn't mean it's wrong, of course.
Anyway, what provoked my interest was a systematic review of the evidence for the ketogenic diet (Henderson et al, Journal of Child Neurology 2006; 21(3): 193-198). This apparently found pretty good evidence of the diet's effectiveness; an odds ratio for >50% seizure reduction of 2.25 (95% confidence interval 1.69-2.98). But looking a bit closer reveals some problems: there wasn't a non-ketogenic diet control group, and the comparison reported is between children who stayed on the diet and those who discontinued it. Read on a bit more and it becomes clear that a major reason for discontinuation was treatment failure (i.e. <50% seizure reduction). So the comparison is between a group many of whom had >50% seizure reduction, and a group many of whom had <50% seizure reduction. Not surprisingly, the reduction in seizures was found to be bigger in the former group. Selecting the groups partly by the outcome that they experienced has introduced serious bias into this comparison.
Abstract is copied below:
The evidence base for the efficacy of the ketogenic diet was assessed among pediatric epileptic patients by application of a rigorous statistical meta-analysis. Nineteen studies from 392 abstracts met the inclusion criteria. The sample size was 1084 patients (mean age at initiation 5.78 +/- 3.43 years). The pooled odds ratio, using a random effects model, of treatment success (> 50% seizure reduction) among patients staying on the diet relative to those discontinuing the diet was 2.25 (95% confidence interval = 1.69-2.98). The reasons for diet discontinuation included < 50% seizure reduction (47.0%), diet restrictiveness (16.4%), and incurrent illness or diet side effects (13.2%). The results indicate that children with generalized seizures and patients who respond with > 50% seizure reduction within 3 months tend to remain on the diet longer. Although no class I or II studies have been published regarding the efficacy of the ketogenic diet, this meta-analysis shows that current observational studies reporting on the therapeutic effect of the ketogenic diet contain valuable statistical data. Future observational studies should aim for long-term follow-up, patient dropout analysis, and improved seizure type characterization.
I will update when I've got the full paper.