Workaround for randomized experiments
A new statistical tool can help researchers get meaningful results when a randomized experiment, considered the gold standard, is not possible.
Randomized experiments split participants into groups by chance, with one undergoing an intervention and the other not. But in real-world situations, they can’t always be done. Companies might not want to use the method, or such experiments might be against the law.
Developed by a researcher at The University of Texas at Austin, the new tool called two-step synthetic control adapts an existing research workaround, known as the synthetic control method.
The traditional synthetic control method creates synthetic control groups from the data, in place of real ones. The groups are weighted statistically and compared with a group undergoing an intervention.
But the synthetic control method does not perfectly apply to all situations, especially ones in which the intervention group is different from control groups, according to Kathleen Li, an assistant professor of marketing at the McCombs School of Business. In these scenarios, the method’s lack of flexibility could lead to less accurate results.
“Our framework allows managers and policymakers to estimate effects they previously weren’t able to estimate accurately,” said Li, who developed the tool along with Venkatesh Shankar of Texas A&M University. “They get a more precise estimate that can help them make more informed decisions.”
The study, published in advance online in the journal Management Science, offers a two-step synthetic control approach: First, it determines whether the traditional synthetic control method applies to a given case. If it does not, the second step uses a more flexible framework that allows weighted controls to differ from 100% or to shift the control group up and down.The researchers tested the new method on a real-world situation by looking at sales of tampons: how they responded in 2016, when New York repealed a sales tax on them. More