Synthetic Control Groups: An introduction to key concepts, recent extensions, and a hands-on application

Abstract

Social scientists often look to policy change as a “natural experiment” that provides the opportunity to assess the causal effect of a policy treatment. For example, you might have data on an outcome both before and after an intervention for a “treated” unit and other "untreated" units. However, simply being untreated does not guarantee that those untreated units will serve as a valid control group for treated. Synthetic control methods use data on untreated units to produce a weighted control group that is more likely to serve as a valid control. These methods have become increasingly popular and can allow for causal inference in many settings where inference could not typically be done. This workshop introduces synthetic controls and will demonstrate a novel extension that exploits a machine-learning, data-driven approach that should be widely applicable to social scientists.

In our work, we study the causal effects of Colorado’s recreational marijuana law on the sales of other legal psychoactive substances that might serve as complements or substitutes for marijuana. To do this we employ a novel extension of the synthetic control estimator. Synthetic control estimators are weighted combinations of untreated groups that are designed to serve as a control group. We extend the estimator most typical synthetic control estimator by incorporating a LASSO into the way the weights for the untreated groups are constructed. The data underlying our analysis come from a retail grocery store scanner database and DEA prescription drug monitoring data. We use detailed product codes to classify the sales of alcohol and tobacco products into a set of homogenous product categories. Then we construct a weekly state level time series for each alcohol and product category. In addition, we construct a weekly state-level time series for the sales of a large set of other product categories that are unlikely to be affected by the availability of legal recreational marijuana in any state. The alcohol and tobacco products in Colorado are potentially treated goods observed before and after Colorado legalized marijuana. The goal of our project is to estimate the counterfactual time series of psychoactive substance sales that would have prevailed in Colorado in the post-periods if the state had not legalized marijuana.

The time series of the sales of alcohol, tobacco, and other products in other states represent a very large set of candidate comparison groups. The Synthetic Control Using Lasso (SCUL) approach is a machine-learning, data-driven way to comb through a very large set of candidate comparison time series, exclude a large number of candidates that are very different from the treated time series, and construct a weighted combination of a small number of candidates that closely resembles a target series. We use cross validation to choose the LASSO penalty parameter and to guard against overfitting the pre-treatment data. Constructing our synthetic control group using lasso has a few advantages over the traditional synthetic control estimator. The first is that the synthetic control can be constructed in a setting where there is a larger candidate set of control states and products than there are observations. This is a common occurrence in many "big data" settings. A second is that our estimator reduces researcher degrees of freedom by automating the model selection process. In general, the estimator allows for a comparison interrupted time series research design and should be broadly applicable to any research design where there are either a small number of treated units or where there are a larger number of candidate controls than observations.

The results of our analysis suggest that Colorado’s recreational marijuana law did affect the sales of other legal psychoactive substances. Some products appear to be substitutes for legal marijuana and others seem to be complements. In particular, we find that the law reduced sales of hard liquor and malt liquor and increased sales of cases of light beer. The recreational marijuana law did not appear to affect sales of a many other alcohol and tobacco products. And it also did not appear to affect the volume of prescription opioid use in Colorado.