The Importance of Base-Rates in Differential Impact: A Bail Reform Case-Study

Researchers commonly seek to understand whether a policy has differential impact on different subgroups, yet the choice of differential impact metric can have a huge effect on interpretation. Commonly used metrics, such as comparing treatment effects in magnitude or as a percentage of the control mean, frequently do not correspond to the question of interest. If the relevant question is how the intervention affected potential beneficiaries of the reform, a different base rate must be used. We demonstrate how to implement a base-rate adjusted impact metric in a difference-in-differences setting using Philadelphia bail reform as an example.