Empirical Model Discussion

Difference-in-Difference Models

Helen Schneider
Spring 2025

1
 Empirical Model (cts)
• Multivariate linear regression
– Omitted variable bias
• Multivariate linear regression with interaction variable
• Two-part model
• Probit
• Ordered probit
• Log transformations
• Time series
• Difference-in-Difference models

2
 Experiments
• A control group would confront the status quo and an experimental group would face
policy change
– Note: in a randomized trial 2 groups should not differ in any consistent fashion
but rather only by the flip of a coin to eliminate bias
• Example: CA welfare trial: 1/3 of the families receiving welfare were receiving
existing benefits and 2/3 were assigned to experimental group and received 15%
lower benefits. Results: 15% reduction in the befits guarantee resulted in 10% higher
employment in the treatment group relative to that of the control group
• Example: A four-year study in Tennessee randomly assigned children in kindergarten
through third grade into classes of different sizes. The results of this experiment
indicated that the smaller class sizes improved test scores (Krueger, 1999).
– However, about 10% of the students in the study switched classes during the
course of the experiment.
• Example: RAND Health Insurance Study

3
 Quasi-Experiments (Natural Experiments)

• Changes in policy that create nearly identical treatment

and control group for studying the effects of the change
(i.e. randomization is created by external forces)
• Example: OR decreased welfare payments and we
observe more hours worked after the policy change
– Economic boom: hours worked increased everywhere
– Control group: WA
• RAND vs. OR health insurance experiment

4
 Time Series
• Your state introduced a tax cut in the year 2023. You are interested in seeing
whether this tax cut has led to increases in personal consumption within the state.
You observe the following information:

Year Consumption in your state

2020 300

2021 310

2022 320

2024 350

• Your friend argues that the best estimate of the effect of the tax cut is an increase in
consumption of 30 units, but you think that the true effect is smaller, because
consumption was trending upward prior to the tax cut. What do you think is a better
estimate?

5
 Diff-in-Diff
• Suppose that you find information on a neighboring state that did not change its tax
policy during this time period. You observe the following information in that state:

Year Consumption in your state Consumption in neighboring state

2020 300 260

2021 310 270

2022 320 280

2024 350 300

• Given this information, what is your best estimate of the effect of the tax cut on
consumption? What assumptions are required for that to be the right estimate of the
effect of the tax cut? Explain.

6
 Difference-in-difference models
• Maybe the most popular models in applied economics today

• Attempts to mimic random assignment with treatment and “control”

sample
• Review: interaction variables
– Interaction=SNAP*South

7
 Problem set up
• Collect data before and after policy intervention
– E.g. welfare reform
– Kansas tax cuts
– Marijuana legalization
• One group is ‘treated’ with intervention
• Have pre-post data for group receiving
intervention
• Can examine time-series changes but, unsure
how much of the change is due to secular
changes 8
Y

Time series = Yt2-Yt1

True effect = Yb-Ya

Yt1

Ya

Yb

Yt2

t1 ti t2
time
9
 Time series: no control

Policy change occurs at time period t1

•True effect of law
– Ya – Yb
•Only have data at t1 and t2
– If using time series, estimate Yt1 – Yt2
•Bias of simple time series: overestimate the
effect of the policy
•Solution?
10
Difference-in-difference models
• Use time series of untreated control group
to establish what would have occurred in
the absence of the intervention
• Key assumption: the time trend without
policy is the same for both groups/states

11
Y
Treatment effect=
(Yt2-Yt1) – (Yc2-Yc1)

Yc1

Yt1

Yc2

Yt2
control

treatment

t1 t2
time
12
 Difference in Difference
Before After
Change Change Difference
Group 1 Yt1 Yt2 ΔYt
(Treat) = Yt2-Yt1
Group 2 Yc1 Yc2 ΔYc
(Control) =Yc2-Yc1
Difference ΔΔY
ΔYt – ΔYc
13
 Assumptions
• In contrast, what is key is that the time
trends in the absence of the intervention
are the same in both groups
• If the intervention occurs in an area with a
different trend, will under/over state the
treatment effect

14
 Basic Econometric Model
• Data varies by
– state (i)
– time (t)
– Outcome is Yit
• Only two periods
• Intervention will occur in a group of
observations (e.g. states, firms, etc.)

15
 Empirical Model
• Three key variables
– Tit =1 if obs i belongs to treatment state
– Ait =1 in the periods when treatment occurs
– TitAit -- interaction term, treatment states after
the intervention
• Yit = β0 + β1Tit + β2Ait + β3TitAit + εit

16
 Yit = β0 + β1Tit + β2Ait + β3TitAit + εit
Before After
Change Change Difference
Group 1 β0+ β1 β0+ β1+ β2+ β3 ΔYt
(Treat) = β2+ β3
Group 2 β0 β0+ β2 ΔYc
(Control) = β2
Difference ΔΔY = β3

17
Application: Hospital Supply of Charity Care

• Unit of analysis: hospital

• Data: OSHPD 2006-2010
• Policy change: Since 2008 all nonprofit hospitals have to
report charity care to the IRS
• Variables: hospital status, year (after 2008), supply of
charity care
. summarize nonprofit charity y2008

Variable Obs Mean Std. Dev. Min Max

nonprofit 1,359 .7579102 .428506 0 1

charity 1,359 2855.041 6414.316 -629.575 77800
y2008 1,359 .4966887 .5001731 0 1

18
Difference-in-Difference in STATA

. gen interaction= nonprofit* y2008

. regress charity nonprofit y2008 interaction

Source SS df MS Number of obs = 1,359

F(3, 1355) = 12.78
Model 1.5369e+09 3 512306505 Prob > F = 0.0000
Residual 5.4336e+10 1,355 40100288.2 R-squared = 0.0275
Adj R-squared = 0.0254
Total 5.5873e+10 1,358 41143453.6 Root MSE = 6332.5

charity Coef. Std. Err. t P>|t| [95% Conf. Interval]

nonprofit 1376.792 557.0168 2.47 0.014 284.0827 2469.501

y2008 532.028 699.1757 0.76 0.447 -839.5564 1903.612
interaction 1009.868 802.8607 1.26 0.209 -565.1166 2584.853
_cons 1161.637 481.4495 2.41 0.016 217.1699 2106.104

19
 Difference-in-Difference
• Stata command to calculate means before
and after intervention for both treatment
group (nonprofit hospitals) and control
group (for-profit hospitals)
– .collapse(mean) charity, by (y2008, nonprofit)

y2008 nonprofit charity Differences

0 0 1161.637

0 1 2538.429 1541.896

1 0 1693.665 532.028

1 1 4080.325 1009.868

20
 Add other controls
. regress charity nonprofit y2008 interaction urate poverty staffedbeds trauma e
> r hhi

Source SS df MS Number of obs = 1,345

F(9, 1335) = 44.09
Model 1.2776e+10 9 1.4195e+09 Prob > F = 0.0000
Residual 4.2982e+10 1,335 32196039.8 R-squared = 0.2291
Adj R-squared = 0.2239
Total 5.5758e+10 1,344 41486238.5 Root MSE = 5674.2

charity Coef. Std. Err. t P>|t| [95% Conf. Interval]

nonprofit -464.4042 532.144 -0.87 0.383 -1508.334 579.5254

y2008 241.5076 890.7953 0.27 0.786 -1506.004 1989.019
interaction 1175.398 720.7339 1.63 0.103 -238.4964 2589.292
urate 69.18827 98.14149 0.70 0.481 -123.3401 261.7166
poverty -84.92048 61.47245 -1.38 0.167 -205.5136 35.67264
staffedbeds 16.80893 1.126938 14.92 0.000 14.59817 19.01969
trauma 1293.309 191.7536 6.74 0.000 917.1382 1669.481
er 1411.658 508.0261 2.78 0.006 415.0414 2408.274
hhi .0912637 .5668932 0.16 0.872 -1.020835 1.203362
_cons -1619.427 930.6734 -1.74 0.082 -3445.169 206.3143

21
 Application: Minimum Wage
•2021 Nobel Prize: David Card “for his empirical contributions
to labour economics” and Joshua D. Angrist and
Guido W. Imbens “for their methodological contributions to the
analysis of causal relationships”
•Card & Krueger (1994) Minimum Wages and Employment: A
Case Study in a Fast Food Industry in New Jersey and
Pennsylvania. The American Economic Review 84(4): 772-
793.
•Abstract: On April 1, 1992, New Jersey's minimum wage rose from $4.25 to $5.05 per
hour. To evaluate the impact of the law we surveyed 410 fast-food restaurants in New
Jersey and eastern Pennsylvania before and after the rise. Comparisons of employment
growth at stores in New Jersey and Pennsylvania (where the minimum wage was
constant) provide simple estimates of the effect of the higher minimum wage. We also
compare employment changes at stores in New Jersey that were initially paying high
wages (above $5) to the changes at lower-wage stores. We find no indication that the rise
in the minimum wage reduced employment.
22
 Book Recommendation
• Jonathan Haidt (2024) The Anxious
Generation, Penguin Press, NY
• Thomas Piketty (translated by Steven
Randall) (2022) A Brief History of Equality,
Harvard University Press, Cambridge, MA

23