Methods for combining experimental and population data to estimate population average treatment effects

Seminar Series

Friday, October 15, 2021 - 03:30
Elizabeth Stuart, PhD

Abstract: With increasing attention being paid to the relevance of studies for real-world practice (such as in education, international development, and comparative effectiveness research), there is also growing interest in external validity and assessing whether the results seen in randomized trials would hold in target populations. While randomized trials yield unbiased estimates of the effects of interventions in the sample of individuals (or physician practices or hospitals) in the trial, they do not necessarily inform about what the effects would be in some other, potentially somewhat different, population.  While there has been increasing discussion of this limitation of traditional trials, relatively little statistical work has been done developing methods to assess or enhance the external validity of randomized trial results.  This talk will discuss design and analysis methods for combining experimental and population data to assess and increase external validity, as well as general issues that need to be considered when thinking about external validity. Implications for how future studies should be designed in order to enhance the ability to estimate population effects will also be discussed.

Speaker: Elizabeth Stuart, PhD
Departments of Mental Health, Biostatistics & Health Policy & Management
Associate Dean – Bloomberg School of Public Health
Johns Hopkins University

Bio: Trained as a statistician, my primary research interests are in the development and use of methodology to better design and analyze the causal effects of public health and educational interventions. In this way I hope to bridge statistical advances and research practice, working with mental health and educational researchers to identify and solve methodological challenges. I am particularly interested in the trade-offs in different designs for estimating causal effects, especially in terms of improving internal validity of non-experimental studies and external validity of randomized studies. This translates into two primary research areas. First, one of my primary research areas is in the use of propensity score methods for estimating causal effects in non-experimental studies (essentially as a tool to improve internal validity and reduce confounding). My interests in this area include providing advice for researchers in terms of best practice for estimation, diagnostics, and use of propensity score methods. This also includes investigation of how to handle complexities in propensity score methods, including multilevel data settings, covariate measurement error, and complex survey data. My second primary research area is in methods to assess and enhance the external validity (generalizability) of randomized trial results and enable policymakers to determine how applicable the results of a particular randomized study are to their own target population. I also have interests in handling complexities in randomized experiments, in particular missing data and non-compliance. The applied areas I focus on include autism, the long-term consequences of adolescent substance abuse, education, mental health services and systems, and the effects of health care reform models on mental health service use.

For Zoom information, please contact Terry Hales at