Causal Sensitivity Analysis for Decision Trees

Abstract

Ventilator assignments in the pediatric intensive care unit (PICU) are made by medical experts; however, for some patients the relationship between ventilator assignment and patient health status is not well understood. Using observational data collected by Virtual PICU Systems (VPS) (58,772 PICU visits with covariates and different ventilator assignments conducted by clinicians), we attempt to identify which patients would derive the greatest clinical benefit from ventilators by providing a concise model to help clinicians estimate a ventilator's potential effect on individual patients, in the event that patients need to be prioritized due to limited ventilator availability.

Effectively allocating ventilators requires estimating the effect of ventilation on different patients; this is known as individual treatment effect estimation. However, we only have access to non-randomized data, which is confounded by the fact that sicker patients are more likely to be ventilated. In order to reduce bias due to potential confounding to estimate the average treatment effect, propensity score matching has been widely studied and applied to estimate the average treatment effect, which matches patients from treated group with patients from control group based on similar conditional probability of ventilator assignment given an individual patient's features. This matching process assumes no unmeasured confounding, meaning there must be no unobserved covariates influencing both treatment assignment and patient's outcome. However, this is not guaranteed to be true, and if it is not, the average treatment effect estimation using propensity score matching approach can be fragile given an unmeasured confounder with strong influences.

Rosenbaum and Dual Sensitivity Analysis is specifically designed for potential unmeasured confounder problems in propensity score matching, assuming confounder's existence it evaluates how "sensitive" the treatment effect estimation after matching can be. This sensitivity analysis method has been well-studied to evaluate the estimated average treatment effect based on propensity score matching, specifically, using generalized linear models as the propensity score model.

However, both estimating treatment effect via propensity score matching and its sensitivity analysis have their limitations: first, propensity score matching only helps in estimating the average treatment effect, while it does not provide much information about individual treatment effect on each patient; second, Rosenbaum and Dual Sensitivity Analysis only evaluates the robustness of estimated average treatment effect from propensity score matching, while it cannot evaluate the robustness of a complex model estimating the individual treatment effect, such as a decision tree model.

To solve this problem, we attempt to estimate the individual treatment effect from observational study, by proposing the treatment effect tree (TET) model. TET can be estimated through learning a Node-Level-Stabilizing decision tree based on matched pairs from potential outcome matching, which is a matching approach inspired by propensity score matching. With synthetic data generated to mimic the real-world clinical setting, we show that TET performs very well in estimating individual treatment effect, and the structure of TET can be estimated by conducting potential outcome matching in observational data.

There is a matching process in TET estimation, and to evaluate the robustness of the estimated TET learned through potential outcome matching in observational data, we propose an empirical sensitivity analysis method to show how sensitive the estimated TET's structure and predictive power can be in situations with strong levels of confounding described by Rosenbaum and Dual Sensitivity Analysis. We use the same synthetic dataset with different levels of confounding encoded as boolean confounders to experiment with this sensitivity analysis method. We show the experimental results of estimating TET from observational data, as well as their performances in sensitivity analysis. The experimental results show that with strong covariates setting, the estimated TET from observational data can be very stable against strong levels of confounding described by Rosenbaum and Dual Sensitivity Analysis encoded as boolean confounders.

In this work, we propose TET model for individual treatment effect estimation with observational data, we show that TET can be learned from matching individuals based on potential outcome. We designed an empirical sensitivity analysis method to evaluate the robustness of TET with different levels of confounding described by Rosenbaum and Dual Sensitivity Analysis, and the experimental results show the learned TET can be stable against strong levels of confounding.