Chapter 11

Experiments and Quasi-Experiments

Multiple Choice

1) The following are reasons for studying randomized controlled experiments in

an econometrics course, with the exception of which?

a. At a conceptual level, the notion of an ideal randomized controlled experiment

provides a benchmark against which to judge estimates of causal effects in
practice.
b. When experiments are actually conducted, their results can be very influential, so
it is important to understand the limitations and threats to validity of actual
experiments as well as their strength.
c. Randomized controlled experiments in economics are common.
d. External circumstances sometimes produce what appears to be randomization.

2) Program evaluation

a. is conducted for most departments in your university/college about every seven

years.
b. is the field of study that concerns estimating the effect of a program, policy, or
some other intervention or “treatment.”
c. tries to establish whether EViews, SAS, or Stata works best for your econometrics
course.
d. establishes rating systems for television programs in a controlled experiment
framework.

3) In the context of a controlled experiment, consider the simple linear regression

formulation Yi = β 0 + β1 X i + ui . Let the Yi be the outcome, Xi the treatment level, and ui
contain all the additional determinants of the outcome. Then

a. the OLS estimator of the slope will be inconsistent in the case of a randomly
assigned Xi since there are omitted variables present.
b. Xi and ui will be independently distributed if the Xi are randomly assigned.
c. β 0 represents the causal effect of X on Y when X is zero.
d. E (Y | X = 0) is the expected value for the treatment group.

4) In the context of a controlled experiment, consider the simple linear regression

formulation Yi = β 0 + β1 X i + ui . Let Yi be the outcome, Xi the treatment level when the
treatment is binary, and ui contain all the additional determinants of the outcome. Then
a differences estimator
calling β1

1
 a. makes sense since it is the difference between the sample average outcome of the
treatment group and the sample average outcome of the control group.
b. and β 0 the level estimator is standard terminology in randomized controlled
experiments.
c. does not make sense, since neither Y nor X is in differences.
d. is not quite accurate since it is actually the derivative of Y on X.

5) The following does not represent a threat to internal validity of randomized controlled
experiments:

a. attrition.
b. failure to follow the treatment protocol.
c. experimental effects.
d. a large sample size.

6) The Hawthorne effect refers to

a. subjects dropping out of the study after being randomly assigned to the treatment
or control group.
b. the failure of individuals to follow completely the randomized treatment protocol.
c. the phenomenon that subjects in an experiment can change their behavior merely
by being included in the experiment.
d. assigning individuals, in part, as a result of their characteristics or preferences.

7) The following is not a threat to external validity:

a. the experimental sample is not representative of the population of interest.

b. the treatment being studied is not representative of the treatment that would be
implemented more broadly.
c. experimental participants are volunteers.
d. partial compliance with the treatment protocol.

8) Assume that data are available on other characteristics of the subjects that are relevant to
determining the experimental outcome. Then including these determinants explicitly
results in

a. the limited dependent variable model.

b. the differences in means test.
c. the multiple regression model.

2
 d. large-scale equilibrium effects.

9) All of the following are reasons for using the differences estimator with additional
regressors, with the exception of

a. efficiency.
b. providing a check for randomization.
c. providing an adjustment for “conditional” randomization.
d. making the differences estimator easier to calculate than in the case of the
differences estimator without the additional regressors.

10) Experimental data are often

a. observational data.
b. binary data, in that the subject either does or does not respond to the treatment.
c. panel data.
d. time series data.

11) With panel data, the causal effect

a. cannot be estimated since correlation does not imply causation.

b. is typically estimated using the probit regression model.
c. can be estimated using the “differences-in-differences” estimator.
d. can be estimated by looking at the difference between the treatment and the
control group after the treatment has taken place.

12) Causal effects that depend on the value of an observable variable, say Wi,

a. cannot be estimated.
b. can be estimated by interacting the treatment variable with Wi.
c. result in the OLS estimator being inefficient.
d. require use of homoskedasticity-only standard errors.

13) To test for randomization when Xi is binary,

a. you regress Xi, on all W’s and compute the F-statistic for testing that all the
coefficients on the W’s are zero. (The W’s measure individual characteristics of
individuals, and these are not affected by the treatment.)
b. is not possible, since binary variables can only be regressors.
c. requires reordering the observations randomly and re-estimating the model. If the

3
 coefficients remain the same, then this is evidence of randomization.
d. requires seeking external validity for your study.

14) The following estimation methods should not be used to test for randomization when Xi is
binary:

a. linear probability model (OLS) with homoskedasticity-only standard errors.

b. probit.
c. logit.
d. linear probability model (OLS) with heteroskedasticity-robust standard errors.

15) In a quasi-experiment

a. quasi-differences are used, i.e., instead of ∆Y you need to use (Y after − λ × Y before ) ,
where 0 < λ < 1 .
b. randomness is introduced by variations in individual circumstances that make it
appear as if the treatment is randomly assigned.
c. the causal effect has to be estimated through quasi-maximum likelihood
estimation.
d. the t-statistic is no longer normally distributed in large samples.

16) Your textbook gives several examples of quasi-experiments that were conducted. The
following is not an example of a quasi-experiment:

a. labor market effects of immigration.

b. effects on civilian earnings of military service.
c. the effect of cardiac catheterization.
d. the effect of unemployment on the inflation rate.

17) A repeated cross-sectional data set

a. is also referred to as panel data.

b. is a collection of cross-sectional data sets, where each cross-sectional data set
corresponds to a different time period.
c. samples identical entities at least twice.
d. is typically used for estimating the following regression model
Yit = β 0 + β1 X it + β 2W1,it + ... + β1+ rWr ,it + uit .

4
18) For quasi-experiments,

a. there is a particularly important potential threat to internal validity, namely

whether the “as if” randomization in fact can be treated reliably as true
randomization.
b. there are the same threats to internal validity as for true randomized controlled
experiments, without modifications.
c. there is little threat to external validity, since the populations are typically already
different.
d. OLS estimation should not be used.

19) Experimental effects, such as the Hawthorne effect,

a. generally are not germane in quasi-experiments.

b. typically require instrumental variable estimation in quasi-experiments.
c. can be dealt with using binary variables in quasi-experiments.
d. are the most important threat to internal validity in quasi-experiments.

20) Heterogeneous population

a. implies that heteroskedasticity-robust standard errors must be used.

b. suggests that multiple characteristics must be used to describe the population.
c. effects can be captured through interaction terms.
d. refers to circumstances in which there is unobserved variation in the causal effect
with the population.

21) If the causal effect is different for different people, then the population regression
equation for a binary treatment variable Xi, can be written as

a. Yi = β 0 + β1 X i + ui .
b. Yi = β 0,i + β1,i X i + ui .
c. Yi = β 0 + β1,i X i + ui .
d. Yi = β 0 + β1Gi + β 2 Dt + ui .

5
22) In the case of heterogeneous causal effects, the following is not true:

a. in the circumstances in which OLS would normally be consistent (when

E (ui | X i ) = 0 ), the OLS estimator continues to be consistent.
b. OLS estimation using heteroskedasticity-robust standard errors is identical to
TSLS.
c. the OLS estimator is properly interpreted as a consistent estimator of the average
causal effect in the population being studied.
d. the TSLS estimator in general is not a consistent estimator of the average causal
effect if an individual’s decision to receive treatment depends on the effectiveness
of the treatment for that individual.

23) One of the major lessons learned in the chapter on experiments and quasi-experiments is
that

a. there are almost no true experiments in economics and that quasi-experiments are
a poor substitute.
b. you should always use TSLS when estimating causal effects in quasi-experiments.
c. populations are always homogeneous.
d. the insights of experimental methods can be applied to quasi-experiments, in
which special circumstances make it seem “as if” randomization has occurred.

24) Quasi-experiments

a. provide a bridge between the econometric analysis of observational data sets and
the statistical ideal of a true randomized controlled experiment.
b. are not the same as experiments, and lessons learned from the use of the latter can
therefore not be applied to them.
c. most often use differences-in-differences estimators, which are quite different
from OLS and instrumental variables methods studied in earlier chapters of the
book.
d. use the same methods as studied in earlier chapters of the book, and hence the
interpretation of these methods is the same.

25) The major distinction between the experiments and quasi-experiments chapter and earlier
chapters is the

6
a. frequent use of binary variables.
b. type of data analyzed and the special opportunities and challenges posed when
analyzing experiments and quasi-experiments.
c. superiority of TSLS over OLS.
d. use of heteroskedasticity-robust standard errors.

7
8