Paper 1: Does obesity shorten life?
The doi for this paper is https://doi.org/10.1038/ijo.2008.82.
What is the paper about?
The paper is a commentary on the application of the potential outcomes framework in the context of observational research. It takes the specific example (mind experiment) of the reported effects of obesity on mortality and gives a narrative critique of the claims from the vantage point of causal inference.
What did they do?
The authors introduce the findings of observational obesity studies and then define the causal question that is being asked. They highlight consistency violations as the chief issue and explain how this leads to a cascade of problems that compromise the inference.
A mind experiment
The paper starts with a mind experiment. Three randomised trials are run to estimate the effect of obesity on mortality. The end use is to inform a policy for reducing excess death attributable to obesity. The first study randomises a mandated exercise program as the intervention group, the second randomises a diet, the third randomises both. In all three studies, the control group is behaviour as usual.
Predictably, there is variation in the results. In all, the distribution of BMI and rate mortality are lower in the intervention group. However, the mortality varies across the studies. The implication is that the effect of obesity is unknown, but to inform a policy decision, simply pick the study/intervention with the greatest reduction in mortality.
An observational study is then run using population-level data on BMI, death and risk factors on lifestyle and death. In this study, the effect of obesity on mortality is reported.
The authors raise the question of how it is possible that the observational study appears more suitable than the RCT in answering causal questions on the health effects of obesity.
Why the paradox?
Simply, RCTs compare the distribution of the outcome under differing experimental conditions. In the mind experiment these conditions were physical activity, diet or both relative to behaviour as usual. The effects are attributable to the intervention, not obesity. In contrast, an observational study allows you to compare the mortality rate based on the definition of obesity, e.g. \(BMI \gt 30\) adjusted for relevant factors. Implicit in the observational result is the notion of some, likely multi-factorial and individual-specific, intervention that reduces weight to normal levels. And, as the Prince of Denmark said, there’s the rub.
If they are to provide a single estimate for the effect of obesity, the observational data analyst has to
- assume that all interventions on BMI have the same effect on mortality or
- consider a complex intervention that changes the determinants of BMI in proportion to the distribution of those determinants in those who are not obese in the population
Both of these have problems; (1) is unlikely to be true (2) is unlikely to be feasible.
Formalisation
We know that causal inference requires consistency, exchangeability and positivity.
As alluded to, the authors argue that consistency (the causal contrast involves two or more well-defined interventions1) is often neglected. In observational studies, we do not know how a unit reached a given state (e.g. what was the intervention that resulted in unit \(i\) achieving a BMI of \(20\)?). Therefore, we are left with a vague notion of what the causal contrast is.
In an RCT, we have exchangeability, which allows us to interpret the associational relationship between the intervention and outcome as a causal relationship. In an observational study, we rely on expert knowledge to identify and adjust for confounding in an attempt to approximate exchangeability, more specifically conditional exchangeability. When the definition of the intervention is vague, adjustment to achieve conditional exchangeability is complicated by virtue of the vast array of factors that could influence both obesity and death.
Assuming that conditional exchangeability can be achieved, we need representation within each strata in order to produce estimates of the treatment effect that do not resort to extrapolation, e.g. see doi.org/10.1214/13-AOAS630. Following on from above, a large adjustment space increases the likelihood of sparse data, which increases the risk of positivity violations.
What did they conclude?
- That you need to watch out for observational studies where the intervention is vaguely (if at all) defined because the causal effect may not be meaningful.
- You cannot talk about or compute causal effects in absence of a well-defined intervention.
- Exchangeability and positivity are both potentially impacted by vague intervention definitions.
- Fancy statistics cannot fix problems with consistency.
- To inform policy, it could make more sense to focus research questions on modifiable lifestyle factors (something you can intervene on) than on a particular characteristic, such as BMI.
What was unclear?
Nothing, although discussion may bring some questions to light.
The authors do not give any mention to other causal inference frameworks and how these may have differing perspectives on the authors claims.
Footnotes
\(Y(a)\) is consistent with the observed \(Y\) if \(Y(a) = Y\) when unit receives intervention \(A=a\).↩︎