Article

American Review of Public Administration

2022, Vol. 52(5) 382–397
The Effect of Civilian Oversight on Police © The Author(s) 2022
Article reuse guidelines:
Organizational Performance: A Quasi- sagepub.com/journals-permissions
DOI: 10.1177/02750740221098344

Experimental Study journals.sagepub.com/home/arp

Jisang Kim1

Abstract
Although civilian oversight has been introduced to make the police accountable to citizens and perform better, its effective-
ness has not been thoroughly investigated yet. To better understand civilian oversight of the police, this study explores its
theoretical basis as an accountability mechanism and suggests how it holds police agencies accountable even without direct
disciplinary authority. The data from Law Enforcement Management and Administrative Statistics (LEMAS) survey, the
Uniform Crime Report (UCR) program, and the Fatal Encounters (FE) database are analyzed to estimate the effect of civilian
oversight on police organizational performance, using a propensity score matching analysis. Among the dependent variables
included in the analysis as performance indicators, civilian oversight turns out to have a significant influence only on police
agencies’ clearance rates. The police agencies with civilian oversight have lower clearance rates by 2.71 percentage points
on average compared to those without civilian oversight. The implications of civilian oversight’s negative impact on police
organizational performance are further discussed.

Keywords
civilian oversight, accountability, performance, administrative procedures, citizen participation, propensity score matching

Introduction of civilian oversight in holding the police accountable for

the work they perform (Kerstetter, 1985; Skolnick & Fyfe,
Accountability initiatives accompanying citizen involvement 1993; Walker, 2006). Yet, the questions remain largely unan-
can be commonly found among modern democratic govern- swered if civilian oversight with no direct authority to oblige
ments (Ferdik et al., 2013; Prenzler, 2011). Especially in the police agencies can indeed hold the police accountable to the
policing context, civilian oversight has been introduced to community they serve and whether civilian oversight can
make the police accountable to citizens and perform their make an actual difference in police organizational perfor-
functions better (Clarke, 2009; Skolnick & Fyfe, 1993). To mance. Especially with the ongoing Black Lives Matter
fulfill such goals, civilian oversight generally has focused movement inflicted by police misconduct and the record-low
on establishing the procedure that enables citizens to public confidence in the police, the current study sheds light
provide public input regarding police operations (De on such a timely topic of police accountability and civilian
Angelis et al., 2016; Ferdik et al., 2013; Walker, 2001). oversight in an attempt to answer the questions above.
Often, civilian oversight has been regarded as a solution Admittedly, there has been comparably little systematic
for the long-standing issues of police misconduct, and it research on the effectiveness of civilian oversight in improv-
even has become recognized as an essential component of ing police performance to be more desirable to citizens (De
police reform necessary for recovering the public’s trust in Angelis et al., 2016). Except for the recent research investi-
the U.S. policing system in more recent years (President’s gating the relationship between accountability and police per-
Task Force on 21st Century Policing, 2020). Nevertheless, formance (e.g., Ali & Nicholson-Crotty, 2021; Ali & Pirog,
civilian oversight has constantly faced challenges even with 2019), the existing research tends to provide sporadic
all its promises (De Angelis et al., 2016; Wilson &
Buckler, 2010). Civilian oversight’s capability in dealing
with police misconduct has been doubted because of the 1
University at Albany – SUNY, Albany, NY, USA
lack of civilians’ experience and expertise in police work
Corresponding Author:
(Kerstetter, 1985). Similarly, civilian oversight initiatives in Jisang Kim, University at Albany – SUNY, 135 Western Ave, Albany, NY
the United States have confronted severe resistance by offi- 12203, USA.
cers and police unions questioning the validity and efficacy Email: [email protected]
Kim 383

evidence not sufficiently supporting the arguments of either a theoretical framework to examine the effect of civilian
proponents or opponents of civilian oversight. The previous oversight on police organizational performance and the
literature (e.g., De Angelis, 2009; Kerstetter & Rasinski, hypotheses based on its theoretical propositions. More
1994; Wells & Schafer, 2007; Worrall, 2002) has shown details on the propensity score matching method utilized in
how civilian oversight influences the perception of individual the analysis and its merits in this research are presented.
citizens or police officers either positively or negatively. After reporting the result of the analysis, the findings of
Nevertheless, they have not explicitly investigated how this research and the implications are discussed.
their findings are related to the broad range of other critical
aspects of police organizational performance. More recent
studies have come to focus on “public and police officer
Civilian Oversight and Police Accountability
safety” (Ali & Nicholson-Crotty, 2021) and “racial disparity” Civilian oversight is frequently referred to as terms such as
(Ali & Pirog, 2019) in police misconduct. Still, there remains “citizen oversight,” “citizen review,” or “external review”
a gap to fill in our understanding of civilian oversight as an (Alpert et al., 2016; Ferdik et al., 2013; Walker, 2001).
accountability mechanism and how it functions to induce Regardless of the terminology applied, civilian oversight
changes in police agencies. can generally be defined as a mechanism of facilitating civil-
Thus, the current study examines the quantitative impact ians’ involvement in holding the police accountable to the cit-
of civilian oversight on police organizational performance izens they serve (De Angelis et al., 2016; Ferdik et al., 2013;
by looking into police agencies’ number of arrests, crime Walker, 2001).
reduction, clearance rates, use of deadly force, and homicide Typically, civilian oversight has been understood to focus
of police officers. More specifically, it attempts to address the on the “citizen complaint process,” providing feedback to law
following questions: “Is civilian oversight effectively func- enforcement agencies based on the complaints filed by citi-
tioning in the policing context?” and “Does civilian oversight zens (De Angelis et al., 2016). However, civilian oversight
make a difference in police agencies to yield better organiza- agencies in the United States are now known to play a
tional performance as theorized?” A quasi-experimental much broader range of roles in practice. Some civilian over-
method is employed for causal inference to answer these sight agencies may still play their traditional role of review-
questions. Based on Bovens (2007, 2010), this article first ing or monitoring compliant investigations of the police
theorizes how civilian oversight renders administrative internal affairs, but others have started to conduct investiga-
accountability relationship with the police. Then it hypothe- tions of complaints or allegations of misconduct independent
sizes that civilian oversight influences police agencies to from the police (De Angelis et al., 2016; Finn, 2001; Walker,
perform better in a way more desirable to the community 2001). Furthermore, some oversight agencies are even autho-
they serve. For the analysis, the data retrieved from the rized to play more active roles, such as reviewing and assess-
Law Enforcement Management and Administrative ing “a police department’s policies, supervision, and
Statistics (LEMAS) survey, the Uniform Crime Report management practices” or even “training, hiring and recruit-
(UCR) program, and the Fatal Encounters (FE) database ment standards” (De Angelis et al., 2016).1
are examined using propensity score matching. Although oversight agencies’ authority and practice in
This research has both theoretical and practical merit. overseeing police agencies vary considerably, independence
How civilian oversight operates as an accountability mecha- from the police organization remains a significant feature
nism within the organizational context can be difficult to common across oversight agencies. By definition, civilian
observe from outside. Nevertheless, by focusing on the oversight should remain external to police organizations as
changes in organizational performance indicators, we it is supposed to ensure nonpolice civilian actors provide
would be able to see if the police change how they perform public input to the police. However, as much as civilian over-
law enforcement functions in response to civilian oversight sight remains outside the chain of command of police organi-
and understand if civilian oversight has a positive impact zations, oversight agencies generally do not have the legal
on police organizational performance as it is theorized authority to direct police agencies (Bovens, 2007, 2010;
without any other undesirable consequences. Furthermore, Walker, 2001). The inputs of civilian oversight agencies to
the empirical evidence of this study would show whether the police are mostly taken as recommendations, not neces-
civilian oversight of the police is effective or not and sarily leading to further disciplinary actions in many cases.
provide a link to bridge the findings of accountability
studies and management studies in public administration.
Lastly, understanding the causal impact of civilian oversight
Civilian Oversight as an Accountability Mechanism
of the police will also help us design better accountability Bovens (2010) has specified the accountability concept and
systems to improve police organizational performance in suggested that “accountability as a virtue” and “accountabil-
alignment with citizens’ interests. ity as a mechanism” should be distinguished. While account-
In the following sections, this study reviews previous lit- ability as a virtue is a normative concept which entails a
erature on civilian oversight and accountability and proposes broad set of desirable qualities of public officials or
384 American Review of Public Administration 52(5)

government agencies, accountability as a mechanism is an appoint and dismiss the chief of police and allocate budget
instrumental arrangement in which such public officials or for the police department (Bovens, 2007, 2010; Walker,
government agencies (i.e., accountability actors) can be 2001).
held accountable by their principals or principals’ other Furthermore, political accountability relationships con-
agents (i.e., accountability forums) (Bovens, 2007, 2010; tinue inside the organization, along the chain of command,
Day & Klein, 1987; Mulgan, 2003; Philp, 2009; Scott, 2000). following the “opposite direction of delegation,” ultimately
Generally, public officials or government agencies are the toward the citizen (Bovens, 2007). Even when the chief of
accountability actors “obliged to explain and to justify his or police appears to be the only actor giving accounts before
her conduct,” and they “may face consequences” when they the forums, accountability runs down along the “chain of
fail to do so (Bovens, 2007, 2010). Highly formalized mea- command” to the street-level officers through the hierarchy
sures or official sanctions are common consequences that of police bureaucracy (Bovens, 2007).
public actors may face, but the full slate of repercussions Thus, by relying on the political accountability established
also includes informal and even implicit dispositions unfa- between the police and the citizen or elected officials, civilian
vorable to these actors (Bovens, 2010). The possibility of oversight agencies can hold the police—police executives
facing such consequences is one of the central aspects of directing departments’ policies, supervisors managing offi-
the mechanism in accountability relationships (Bovens, cers, and officers enforcing laws at the street level—account-
2010). able. As police render accounts to civilian oversight agencies
Nevertheless, as Walker (2001) has pointed out, chiefs of in response, civilian oversight will be able to impact the
police commonly view and treat decisions of civilian over- police and induce changes in how they perform the police
sight agencies only as a recommendation, with very few work even without direct disciplinary authority.
exceptions. Even for the oversight agencies conducting inde- For instance, Syracuse is one of the cities facing the cons-
pendent investigations, it is rare for them to have the author- tant demand for police reform, which could show how
ity to direct the police to initiate disciplinary actions (De administrative accountability works to hold the police
Angelis et al., 2016). For this reason, civilian oversight accountable by accessing the political accountability relation-
may seem to lack such a crucial component of accountabil- ships around a police agency. Backed by public support,
ity—a means to pressure the police to make them accountable Syracuse’s Citizen Review Board (CRB) has been playing
for their conduct—and to fall short of functioning as a robust a central role in prompting changes in how Syracuse Police
accountability mechanism. Department performs. Although Syracuse’s CRB has not
Although commonly described as a “dyadic relationship” always been successful in its attempts of holding the police
between principal and agent, accountability, in reality, is gen- accountable, it has been pressuring the mayor and common
erally constituted upon the multiplicity and complexity of council to accept the citizens’ demand for police reform
relationships among various accountability actors and which includes the revision of the use-of-force policy and
forums (Bovens, 2007, 2010). By accessing adjacent body camera policy (Lohmann & Libonati, 2020; Syracuse
accountability relationships, civilian oversight agencies can Police Accountability and Reform Coalition, 2021). While
oblige actors to render accounts to forums even without the demand for police reform is facing a level of resistance,
authority to enforce compliance directly. Most oversight there have been meaningful changes in some police policies,
agencies such as ombudsmen and audit agencies serve in such as a release of police data to the public by legislature
an advisory capacity, as they are not direct superordinates mandate (Libonati, 2021).
of the organizations they oversee and have no direct discipli-
nary authority (Bovens, 2007). However, through “adminis-
Civilian Oversight Improving Organizational
trative accountability” (i.e., diagonal relationship), oversight
agencies can hold actors accountable indirectly via elected Performance
officials who sit higher in the hierarchy with authority to Civilian oversight and administrative procedures. Civilian over-
reward and sanction them (Bovens, 2007, 2010). sight holds police organizations accountable not only to over-
Based on Bovens (2007, 2010), Figure 1 shows account- sight agencies but also to citizens. McCubbins et al. (1987)
ability relationships surrounding a civilian oversight agency have provided a theoretical basis for the proposition that
and a police organization. It briefly presents how civilian civilian oversight and its influence lead police agencies to
oversight can hold the police accountable and, in turn, influ- yield better organizational performance more desirable to
ence how the police perform law enforcement functions. As the community they serve.
stated above, even oversight agencies lacking a direct disci- McCubbins and colleagues (1987) have theorized that
plinary authority may derive power from the political “administrative procedures” can shape the decisions of
accountability relationship established between the police public agencies and make them responsive to citizens’ inter-
and the citizen or elected officials. Specifically, the elected ests and preferences based on principal–agent theory. For
officials, such as mayors and city council members, can instance, the U.S. Environmental Protection Agency (EPA)
oblige the police and hold them accountable, as they can establishes environmental standards that function as
Kim 385

Figure 1. Accountability relationships surrounding civilian oversight agency and police.

administrative procedures impacting and shaping the state information in response to administrative accountability ren-
agencies’ decision-making on environmental policy. By dered by civilian oversight and make their policies and oper-
focusing on the EPA’s air quality standards, Potoski and ations more responsive to citizens’ perspectives to avoid
Woods (2001) have shown that administrative procedures negative consequences. Thus, the administrative policies
influence and limit the bureaucratic autonomy of state agen- and operational guidelines introduced by civilian oversight
cies, thus enhancing the agencies’ democratic responsiveness will improve police performance by altering how the police
in air pollution policy. operate to be more accountable to citizens.
Although McCubbins and colleagues (1987) have referred
to “oversight” as a political control mechanism of elected Civilian oversight and citizen participation. Democratic theory
officials directly monitoring, rewarding, and sanctioning of citizen participation also provides the basis for the propo-
their agents—bureaucrats, the idea of civilian oversight of sition that civilian oversight enhances the organizational per-
the police fits the definition of administrative procedure formance of public agencies (e.g., Nabatchi, 2010; Neshkova
they have proposed. Administrative procedures induce & Guo, 2012). Although focusing more on the relationship
public agencies’ compliance by establishing procedural between public agencies and citizens itself, not civilian over-
requirements impacting the institutional environment where sight, scholars suggest that public input can enhance the effi-
agencies make decisions, and in turn, constrain the range of ciency and effectiveness of public service by providing
agencies’ viable options for decision-making (McCubbins invaluable information to administrators (Moynihan, 2003;
et al., 1987). Public agencies are expected to perform Roberts, 1997; Skolnick & Fyfe, 1993; Walker, 2001).
better, yielding better organizational performance, more Particularly, citizens can provide specific knowledge only
desirable to citizens, as procedural arrangements require available from their experience and perspective (Moynihan,
them to disseminate the information which elected officials 2003). Such knowledge helps public officials understand
and citizens are interested in and increase the implementation the priority of citizens, and public officials can make better
cost of policies not aligned with the interests or preferences of decisions in allocating their administrative resources based
elected officials or citizens (McCubbins et al., 1987). on such knowledge (Moynihan, 2003; Skolnick & Fyfe,
Civilian oversight of the police as an administrative proce- 1993). For example, Community-Based Environmental
dure will require the police agencies to take in the relevant Protection (CBEP) efforts initiated by the EPA show that
386 American Review of Public Administration 52(5)

citizen participation can inform public agencies to yield As an administrative procedure obliging public agencies
better organizational performance. Specifically, CBEP to align their operation with citizens’ interests, civilian over-
brings community stakeholders into the environmental pro- sight could make police agencies more responsive to the
tection policy process of the EPA “to identify environmental community’s perspective and alter how police agencies
concerns, set priorities, and implement comprehensive solu- perform their function based on the community’s interests.
tions” (U.S. EPA, 1999). The evaluation of CBEP efforts In carrying out their law enforcement mission, what police
has provided evidence that citizen participation may agencies view as priority issues could drastically differ
improve the EPA’s performance in environmental manage- from the issues that concern citizens the most (Sparrow,
ment by yielding “new forms of integration and coordina- 2015). By exposing police agencies to the ideas and views
tion” and promoting “capacity building and sustainability” external to the organizations and instilling them in police
(U.S. EPA, 2003). agencies with the complaints and feedback through the estab-
In the policing context, the citizen complaints addressed lished administrative procedures, civilian oversight could
by civilian oversight can provide critical management infor- make police agencies operate based on the community’s pri-
mation (Moynihan, 2003; Skolnick & Fyfe, 1993). When we ority in dealing with crime and make them deal with crime
consider the work of police officers, mainly taking place on more effectively and efficiently.
the street, out of direct oversight of their supervisors Moreover, even with their heightened presence, police
(Lipsky, 1980; Skolnick & Fyfe, 1993), the significance of agencies alone cannot successfully deal with all crimes in
such complaints becomes even more critical since they help the community. The police could hardly know about the
police administrators understand how police officers criminal incidents never reported to them, only with a few
perform their duty in the field. Thus, civilian oversight can exceptions of high-profile cases. Clearly, police agencies
be understood as a channel of public input helping police need public input only available from citizens in the commu-
agencies improve their administrative decisions. The police nity to deal with crime effectively and efficiently. In this
would be able to yield better organizational performance, regard, civilian oversight could function as a channel for
more desirable to citizens, especially when such public delivering invaluable information regarding ineffective or
input can provide information that is not easily available insufficient law enforcement operations in the community
and attainable from the perspective of the police. where issues are overlooked or underenforced by police
agencies.
To understand police agencies’ organizational performance
in controlling crime, the current study relies on the following
Hypotheses
three indicators: arrest count, crime reduction, and clearance
Based on the theoretical basis above, this study expects that rate. Although not without flaws, these performance indicators
civilian oversight, as an accountability mechanism, would represent how effective the police agencies are in dealing with
improve police organizational performance. Conventionally, crime. As one of the primary police organizational perfor-
the focus of police organizational performance has been on mance indicators, arrest count indicates agencies’ enforcement
the crime control function of the police and the related perfor- productivity (Maslov, 2015; Sparrow, 2015). It shows how
mance outputs. Nevertheless, as police work comprises multi- hard the police are working to control crime, reflecting the
ple dimensions, a multifaceted approach is required in police presence and the organizational efforts in arresting
understanding police organizational performance (Moore & offenders. Crime reduction also has been widely adopted as
Braga, 2003a, 2003b; Sparrow, 2015). Police organizational an indicator of success in crime control (Maslov, 2015;
performance single-mindedly focusing on what police agen- Sparrow, 2015). While focusing on aspects more external to
cies produce, such as lower crime rate or faster response police organizations, crime reduction reflects how effective
time, clearly cannot reflect other critical dimensions that the police are in deterring crime. Clearance rate is another
could matter as much or even more to a service recipient, widely adopted performance measure indicating police agen-
the community. Especially in the policing context, as it cies’ accomplishment in countering crime (Maslov, 2015;
involves continued transactions between the police and the Nicholson-Crotty & O’Toole, 2004). It reflects police agen-
community, how the police perform and deliver law enforce- cies’ investigative and administrative efforts dedicated to
ment services in the community is also critical as much as solving criminal cases and bringing offenders before the
what they achieve in the community (Moore & Braga, justice system (Mas, 2006; Weisburd et al., 2010; Worrall,
2003b; Sparrow, 2015). Thus, when understanding police 2019).
organizational performance, this study focuses on both (1) With civilian oversight functioning as an accountability
how police agencies control crime and (2) how police agencies mechanism, police agencies will perform better in dealing
interact with citizens in the community as they deliver law with crime—apprehending more criminal offenders, deter-
enforcement services. For further analysis, each aspect of ring more crimes in the community, and closing more crim-
police organizational performance is measured through inal cases by solving them. Thus, this study proposes the
various quantifiable indicators. following hypotheses.
Kim 387

H1: Civilian oversight leads to higher arrest counts of citizens’ perception, such as perceived legitimacy or satisfac-
police agencies. tion with the police, these quantitative indicators still offer a
H2: Civilian oversight leads to higher crime reductions of possibility of gauging police organizational performance as
police agencies. proxies. Use of deadly force can measure one of the critical
H3: Civilian oversight leads to higher clearance rates of dimensions of police organizational performance that indi-
police agencies. cates how police agencies use force and authority in law
enforcement missions. Even when the use of force by the
Nevertheless, what is represented by crime statistics police is justified, police agencies’ organizational perfor-
should be only considered as the bottom line of police orga- mance should take the “proper use” of force into account
nizational performance at most (Maslov, 2015; Sparrow, (Moore & Braga, 2003a). Police agencies accomplishing
2015). As policing involves police agencies’ continued inter- law enforcement objects without resorting to excessive
action with citizens that directly impacts the community they force—causing death—will not only be considered efficient
serve, the police organizational performance should also but also deemed legitimate and preferred by the community.
reflect the community’s perspective on how police agencies Likewise, although it does not directly measure police agen-
perform and carry out their law enforcement mission in the cies’ performance in relationships with the community, as an
community. For instance, when police agencies are deemed extreme form of citizen aggression toward the police,
illegitimate, and their law enforcement is considered unjust homicide of police officers could reflect citizens’ per-
by the community, these agencies would not be appraised ceived legitimacy of the police. As the recent research
as performing well, even when all crime statistics indicate (Ali & Nicholson-Crotty, 2021) has alluded, enhanced
they are successfully controlling crime. police legitimacy could lead to fewer citizen aggression
In this regard, civilian oversight could play a crucial role toward the police. In the community where police agencies
in improving police agencies’ organizational performance are appraised as legitimate, the police are less likely to
—how police agencies maintain constructive relationships encounter such extreme hostility leading to the death of
with the community as they interact with citizens while car- police officers.
rying out the law enforcement mission. As an administra- With civilian oversight as an accountability mechanism,
tive procedure, civilian oversight can oblige police police agencies will perform better in establishing and main-
agencies to perform their functions in a way the community taining relationships with the community as they carry out the
accepts as legitimate through procedural requirements or mission—resorting to less use of deadly force and encounter-
administrative policies within the civilian oversight proce- ing less citizen aggression from the community. Thus, this
dure. By making particular practices taxing and costly for study proposes the following hypotheses.
police agencies, civilian oversight can limit the range of
viable options for police agencies’ practices to be aligned H4: Civilian oversight decreases the use of deadly force by
with citizens’ perspectives and make police agencies the police.
refrain from relying on illegitimate and unfair measures H5: Civilian oversight decreases the homicide of police
in practice. officers.
By enabling citizen participation, civilian oversight pro-
vides an opportunity for the community to be heard. It delivers
the community’s direct input that informs police agencies what Data and Variables
citizens think of the police and their law enforcement practice.
Such public input delivering external feedback is essential for Data
the police to establish and maintain constructive relationships This study uses the data retrieved from the LEMAS survey, the
with the community they serve. With the employees sharing a UCR program, and the FE database. Since 1987, the Bureau of
strong homogeneous occupational identity, unjust or unfair Justice Statistics (BJS) has periodically conducted the LEMAS
practices are not likely to be acknowledged or corrected by survey every 3–4 years to collect information from a represen-
the police alone (Hong, 2016). In this regard, citizens’ com- tative sample of state and local law enforcement agencies in the
plaints and feedback from the community will provide invalu- United States (BJS, 2016). The information obtained from the
able information regarding law enforcement practices that survey includes responsibilities of the agency, operating expen-
police agencies might have never considered problematic to ditures, education and training requirements, demographic
improve police organizational performance. characteristics of employees, policing initiatives, and written
Thus, to measure police agencies’ performance regarding policies or procedures regarding policing operations (BJS,
their relationships with the community, this study turns to 2016). The UCR program has been managed by the Federal
two additional organizational performance indicators: use Bureau of Investigation (FBI) since 1930 to establish the
of deadly force and homicide of police officers. Although basis for the periodic nationwide assessment of reported
police agencies’ organizational performance in relationships crimes(FBI, 2016a, 2016b, 2016c). As a part of summary
with the community would be more accurately reflected in data of the UCR program, Arrests by Age, Sex, and Race
388 American Review of Public Administration 52(5)

(ASR), Offenses Known and Clearances by Arrest, and Police analyzed. Specifically, 79 agencies reported as not serving
Employee Data provide the number of arrests reported by a specific population, including state primary law enforce-
police agencies (FBI, 2016a), “the number of reported offenses ment agencies, are excluded from the analysis since the per-
that have been cleared by arrests” (FBI, 2016b), and the formance of these agencies cannot be directly compared with
number of felonious killings upon law enforcement officers other local agencies with a specific population they serve.
in the line of duty (FBI, 2016c). The FE is a nonprofit organi- Similarly, 472 agencies classified to serve “less than 2,500”
zation collecting information about police-involved deaths in populations in the 2016 LEMAS survey are excluded from
the United States supported by paid and volunteer researchers the analysis. Police agencies serving a small population in
through crowdsourcing. The database is sourced from pub- their jurisdiction are prone to measurement errors as the
lished media reports, public record requests, and agency report- current indicators relying on crime statistics do not measure
ing. The information available in this database goes back to their performance accurately (Maslov, 2015). As police agen-
January 1, 2000, and the information of more than 30,000 inci- cies with a small population are not likely to have many
dents has been collected and verified until September 2021. As crimes, they do not have as many opportunities to deal
there is no complete nationwide police-involved deaths data with crime as other agencies with a larger population, no
administered by the government in the United States, the FE matter how well they perform (Maslov, 2015). Finally, 70
database can be considered the most complete data source pub- agencies with missing organizational or regional information
licly available. are also removed in a data cleaning process. The descriptive
Among 2,784 law enforcement agencies that responded to statistics of 1,951 police agencies analyzed in the study are
the 2016 LEMAS survey, a total of 1,951 agencies are provided in Table 1.

Table 1. Descriptive Statistics of Variables.

Variables Mean SD Median Min. Max.

Organizational performance
Number of arrests made by agency 2692.19 6605.22 588 0 86,600
Crime reduction (amount of reduced crime rates) −55.77 1113.40 2 −32.597 6,021
Clearance rate (%) 31.37 17.63 30.26 0 180
Number of use of deadly force cases by police 0.26 0.98 0 0 19
Number of officers killed by felonious acts 11.84 36.60 1 0 652
Treatment
Civilian oversight (No = 0, Yes = 1) 0.11 0.31 0 0 1
Organizational resources
Total amount of operational budget (logged) 15.73 1.71 15.71 11.41 22.44
Total number of employees (logged) 4.18 1.57 4.09 0 10.85
Race and gender
Proportion of non-White employees 0.35 0.24 0.33 0 1
Proportion of female employees 0.23 0.11 0.23 0 0.88
Organization type
Agency type (sheriff = 1, local police = 2) 1.26 0.44 1 1 2
Administrative policy (No = 0, Yes = 1)
Policy on civilian complaints 0.96 0.20 1 0 1
Policy on reporting use of force 0.99 0.10 1 0 1
Policy on body-worn camera 0.49 0.50 0 0 1
Policy on cultural awareness training 0.62 0.49 1 0 1
Required investigation of use of force resulting in bodily injury 0.47 0.50 0 0 1
Required investigation of use of force resulting in death 0.78 0.42 1 0 1
Required investigation of in-custody death not due to use of force 0.64 0.48 1 0 1
Required investigation of discharge of firearm 0.46 0.50 0 0 1
Regional characteristics
Number of officers per 1,000 population 21.94 30.26 18 2 765
Population (logged) 10.18 1.47 10.12 6.84 15.96
Core city (No = 0, Yes = 1) 0.20 0.40 0 0 1
MSA classifications (City = 1, Non-MSA county = 2, MSA county = 3) 1.44 0.75 1 1 3
Violent crime rates 11.50 10.74 8.62 0 174.47
Property crime rates 22.81 17.80 18.31 0 231.43

n = 1,951. MSA = metropolitan statistical area.

Kim 389

Variables 2016 UCR program data that are potentially associated with
police organizational performance and civilian oversight.
Dependent variables. The dependent variables of this study
Organizational characteristics of the police agencies are the
are the organizational performance indicators of police agen-
organizational aspects, including organizational resources,
cies: arrest count, crime reduction, clearance rate, use of
racial and gender compositions, organization type, and admin-
deadly force, and homicide of police officers. All dependent
istrative policies. More specifically, organizational resources
variables except for homicide of police officers are measured
include police agencies’ budgets and employees, each mea-
using the data from the 2016 UCR program, and homicide of
sured by logging the total amount of operational budget and
police officers is measured using the data from the FE
the total number of employees. Racial and gender composi-
database.
tions are measured by the proportion of non-White employees
Arrest count, crime reduction, and clearance rate are used
and female employees in police agencies. Furthermore, organi-
to indicate police agencies’ performance in dealing with
zation type is defined by an agency’s status as being either a
crime. Arrest count is measured by the number of arrests
sheriff’s office or a local police department.4 Lastly, to
made by agencies. The arrest counts of agencies range
measure agencies’ administrative policy guiding the police
from 0 to 86,600, with a mean of 2692.19. Crime reduction
operations, this study uses the 2016 LEMAS survey
is the reduction in reported crime rates in police agencies’
items asking police agencies if they have “written policy
jurisdictions. Its values range from − 32,597 to 6,021, and
or procedural directives” on “civilian complaints,”
the mean of these values is −55.77, meaning the crime
“reporting use of force,” “body-worn cameras,” and “cul-
rates increased by 55.77 on average in 2016 compared to
tural awareness training,” and if an external investigation
the previous year. Clearance rate is the portion of crime
is required in the cases of “use of force resulting in a
solved by agencies.2 The clearance rates of the police agen-
subject sustaining serious bodily injury,” “use of force
cies range from 0 to 180, with a mean of 31.37.3
resulting in a subject’s death,” “in-custody death not due
Use of deadly force and homicide of police officers are
to use of force,” or “discharge of a firearm at or in the
indicators of police agencies’ performance in managing the
direction of a person” (BJS, 2016).
relationships with the communities they serve. Use of
Regional characteristics of the police agencies’ jurisdic-
deadly force is a count of police use of deadly force, with
tions entail demographic and socioeconomic aspects that
values ranging from 0 to 19 and a mean of 0.26. Homicide
could also be related to organizational performance and civil-
of police officers is the number of police officers killed in
ian oversight of police agencies, measured by the number of
felonious acts. Its values range from 0 to 642, with a mean
officers per population, the logged population in jurisdic-
of 11.84.
tions, being a core city or not, and metropolitan statistical
area (MSA) classifications determining a region being a
Treatment. Civilian oversight is the treatment variable of the city, MSA county, or non-MSA county. Lastly, crime rates
current quasi-experimental analysis. Civilian oversight is include both violent and property crime rates, measured by
coded as a binary variable where “1” indicates “Yes” and the number of each type of crime per 100,000 population.
“0” indicates “No” based on the questionnaire item of the
2016 LEMAS survey, asking “Is there a civilian complaint
Methods
review board or agency in your jurisdiction that reviews
the complaints about use of force against officers in your A Quasi-Experimental Study: Propensity Score
agency?” (BJS, 2016). Among the 1,951 police agencies Matching
included in the analysis, there are 217 agencies with civilian
oversight. The agencies with civilian oversight are consid- To examine the causal effect of civilian oversight on police
ered as the treatment group (Civilian oversight = 1), and the organizational performance, this study employs a
agencies with no such civilian oversight are considered as quasi-experimental method: propensity score matching. As
the control group (Civilian oversight = 0). one of the methods to overcome the limitation of nonexperi-
mental data analysis, propensity score matching provides a
way to estimate average treatment effects (ATEs) while
Covariates. To control for the confounding factors simultane- reducing the systematic difference between two groups
ously influencing the dependent variables (police organiza- (Caliendo & Kopeinig, 2008; Imbens, 2004). The underlying
tional performance) and the treatment condition (civilian idea of this method is to control confounding factors by iden-
oversight), the study includes an extensive list of baseline char- tifying counterfactual non-treated observations identical to
acteristics as the covariates in the propensity matching for treated observations so the ATEs can be directly estimated
achieving the balance between the treatment group and the by comparing the outcome variables of treatment and
control group. These covariates are organizational characteris- control groups (Austin, 2011).
tics of the police agencies and regional characteristics of their Since the police agencies in this study are not randomly
jurisdictions retrieved from the 2016 LEMAS survey and the assigned to have civilian oversight, and only a little more
390 American Review of Public Administration 52(5)

Table 2. Mean Difference in Baseline Covariates of Treatment and Control Groups.

Controlled (No civilian oversight) Treated (Civilian oversight)

n = 1,734 n = 217

Organizational resources
Total amount of operational budget (logged) 15.61 16.69***
Total number of employees (logged) 4.07 5.10***
Race and gender
Proportion of non-White employees 0.35 0.37
Proportion of female employees 0.22 0.24*
Organization type
Agency type (sheriff = 1, local police = 2) 1.27 1.15***
Administrative policy (No = 0, Yes = 1)
Policy on civilian complaints 0.96 0.99**
Policy on reporting use of force 0.99 1.00
Policy on body-worn camera 0.48 0.59***
Policy on cultural awareness training 0.61 0.72***
Required investigation of use of force resulting in bodily injury 0.47 0.43
Required investigation of use of force resulting in death 0.78 0.71***
Required investigation of in-custody death not due to use of force 0.65 0.58**
Required investigation of discharge of firearm 0.46 0.47
Regional characteristics
Number of officers per 1,000 population 21.33 26.88***
Population (logged) 10.07 11.01***
Core city (No = 0, Yes = 1) 0.17 0.47***
MSA classifications (City = 1, Non-MSA county = 2, MSA county = 3) 1.46 1.28***
Violent crime rates 10.92 16.16***
Property crime rates 21.68 31.91***

MSA = metropolitan statistical area.

*p < .05, **p < .01, ***p < .001 on t-test of mean difference.

than 10% of them have civilian oversight, a propensity score Empirical Model and Analytic Strategy
matching can be useful to address the systematic difference
Propensity scores are estimated based on the probability of
between the agencies with and without civilian oversight.
police agencies having civilian oversight using a logistic
Not only the difference in the size of groups but also the dif-
regression model. The following is the mathematical model
ference in the baseline characteristics can be problematic, as
used for estimating propensity scores.
shown in Table 2. The organizational and regional character-
istics potentially associated with both treatment and depen-
dent variables have significantly different distributions Pr (Civilian Oversighti = 1)
between two groups of police agencies. By conducting a pro- = F(β0 + β1 Organizational Characteristicsi
pensity score matching analysis, this study controls for these + β2 Regional Characteristicsi )
confounding factors and addresses the systematic difference
when estimating the ATE of civilian oversight on police Using Stata software version 16, this study utilizes the “tef-
organizational performance. fects psmatch” command to conduct a series of propensity
The advantage of propensity score matching also lies in its score matching analyses (StataCorp, 2019). The ATEs of
less strict conditions required for analysis when compared to civilian oversight is estimated with the 1 Nearest-Neighbor
that of ordinary least squares (OLS) regression. For instance, (NN) matching option without a caliper restriction, then
a violation of normality assumption does not influence the increased the number of NN matching and added caliper
estimation of a propensity score matching (Li, 2013), and restrictions to examine if the results remain consistent over
no remedy is required for a multicollinearity issue since it different conditions of matching.
does not bias estimated propensity scores (McMurry et al., This study further checks the balance between police
2015). Thus, a propensity score matching is not only a way agencies with and without civilian oversight after the match-
to examine the causal effect even when the experimental ing analysis to ensure the robustness of its result. It is impor-
study is not feasible, but also a way to overcome the obstacles tant to have balanced control and treatment groups in
from conducting an OLS regression analysis while using matching analysis since ATEs are directly estimated from
observational data. the differences in outcome variables deriving from the only
Kim 391

Table 3. Comparison of Covariate Balance Before and After Matching Process.

Standardized differences Variance ratio

Raw Matched Raw Matched

Organizational resources
Total amount of operational budget (logged) 0.576 −0.06 1.665 1.271
Total number of employees (logged) 0.601 −0.066 1.682 1.245
Race and gender
Proportion of non-White employees 0.102 −0.085 0.831 0.956
Proportion of female employees 0.123 0.057 0.856 1.192
Organization type
Agency type (sheriff = 1, local police = 2) −0.302 −0.024 0.65 0.974
Administrative policy (No = 0, Yes = 1)
Policy on civilian complaints 0.185 −0.008 0.319 1.036
Policy on reporting use of force 0.072 −0.066 0.425 1.776
Policy on body-worn camera 0.221 0.088 0.973 0.994
Policy on cultural awareness training 0.242 0.036 0.844 0.98
Required investigation of use of force resulting in bodily injury −0.077 0.07 0.989 1.003
Required investigation of use of force resulting in death −0.181 0.087 1.232 0.881
Required investigation of in-custody death not due to use of force −0.152 0.114 1.078 0.922
Required investigation of discharge of firearm 0.008 0.136 1.005 1.004
Regional characteristics
Number of officers per 1,000 population 0.144 0.057 2.964 1.592
Population (logged) 0.583 −0.082 1.655 1.131
Core city (No = 0, Yes = 1) 0.686 −0.027 1.796 0.961
MSA classifications (2: Non-MSA county) −0.183 −0.043 0.6 0.898
MSA classifications (3: MSA county) −0.179 0.032 0.683 1.059
Violent crime rates 0.441 −0.048 1.707 1.024
Property crime rates 0.5 0.009 2.049 1.073
Interaction terms
Operational budget × agency type −0.287 −0.016 0.689 1.002
Number of officers × agency type −0.23 −0.001 0.828 1.072
Violent crime × core city 0.657 −0.068 3.015 0.837
Property crime × core city 0.702 −0.021 2.501 0.908
Number of obs. 1,951 3,902 1,951 3,902
Treated obs. 217 1,951 217 1,951
Controlled obs. 1,734 1,951 1,734 1,951

MSA = metropolitan statistical area.

difference between the two groups: the treatment condition. performance indicators of police agencies—arrest count,
Table 3 presents the comparison of covariate balance crime reduction, clearance rate, use of deadly force, and
before and after the matching process. When the covariates homicide of police officers—civilian oversight turns out to
are perfectly balanced, the standardized differences should have a significant impact only on clearance rate. In other
be 0, and the variance ratios should be 1. As shown in words, civilian oversight leads to lower clearance rates,
Table 3, although the baseline covariates are not balanced while it is not causing any significant difference in arrest
perfectly, their balance is substantially improved. Overall, count, crime reduction, use of deadly force, and homicide
the standardized differences get closer to 0, and the variance of police officers between police agencies with and without
ratios get closer to 1, after the matching process. civilian oversight.
In Figure 2, the propensity score overlap distribution The ATE of civilian oversight on clearance rate is esti-
between two groups is provided. It visually presents how mated to be −2.71, and it is statistically significant at the
well the propensity scores are balanced after the matching. level of α = 0.05. This result indicates that the police agen-
cies with civilian oversight have lower clearance rates by
2.71% points on average compared to the agencies with no
Results civilian oversight. Civilian oversight’s ATE on clearance
The result of the matching analysis is provided in Table 4. rate remains consistent in their direction and magnitude as
Among the dependent variables included in the analysis as the number of NN increases from 1 to 10. The estimated
392 American Review of Public Administration 52(5)

Figure 2. Propensity score overlap distribution before and after matching.

Table 4. Average Treatment Effect (ATE) of Civilian Oversight on Organizational Performance.

(1) (2) (3) (4) (5)

Arrest count Crime reduction Clearance rate Use of deadly force Homicide of police officers

Civilian oversight −31.59 33.91 −2.706* 0.0318 −0.00410

(432.0) (59.83) (1.282) (0.102) (0.0105)
N 1,951 1,951 1,951 1,951 1,951

Abadie-Imbens robust SE is given in parentheses.

*p < .05, **p < .01, ***p < .001.

ATE ranges from −2.40 to −2.80, and all ATEs remain stat- no significant influence on police agencies’ performance in
istically significant at least at the level of α = 0.05. The esti- dealing with crime, except for their clearance rates. At the
mation results with different NN specifications are shown in same time, civilian oversight neither shows any significant
Table 5. Also, when the caliper restriction is applied, the ATE impact on police agencies’ performance in managing the rela-
of civilian oversight on clearance rate remains constant until tionships with the community they serve. Even the only signifi-
the caliper size decreases to 0.1. With a caliper size smaller cant impact civilian oversight has on police agencies’ clearance
than 0.1, the ATE cannot be estimated in the current propen- rate is negative, lowering the clearance rates of police agencies
sity score matching analysis. with civilian oversight. In this sense, civilian oversight seems to
worsen rather than improve police agencies’ organizational per-
formance at best. Nevertheless, instead of jumping to a conclu-
Discussion and Conclusion sion generalizing that civilian oversight is ineffective and
The results of the current quasi-experimental analysis do not undermines police organizational performance, this study
seem to support the hypotheses that civilian oversight improves further reflects on its findings of civilian oversights’ impact
police organizational performance along the dimensions consid- on each organizational performance indicator of police agencies
ered in the analysis. Overall, civilian oversight turns out to have and attempts to draw implications of the findings.
Kim 393

Table 5. Average Treatment Effects (ATEs) of Civilian Oversight With Number of Nearest-Neighbors (NN) Adjustment.

(1) (2) (3) (4) (5) (6) (7) (8) (9)

2-NN 3-NN 4-NN 5-NN 6-NN 7-NN 8-NN 9-NN 10-NN

Civilian oversight −2.610** −2.558* −2.798** −2.401** −2.692*** −2.433** −2.531** −2.740** −2.504*
(0.894) (1.128) (0.938) (0.862) (0.809) (0.847) (0.877) (1.034) (1.098)
N 1,951 1,951 1,951 1,951 1,951 1,951 1,951 1,951 1,951

Abadie-Imbens robust SE is given in parentheses.

*p < .05, **p < .01, ***p < .001.

At a glance, police agencies clearing fewer criminal arise when principals provide strong incentives only on
offenses may seem to resemble a recent trend of some of the observable outcomes (Holmstrom &
“de-policing”—the police retreating and under-enforcing in Milgrom, 1991; Koppl & Sacks, 2013). Such strong incen-
response to the citizens’ hostility after severe confrontations tives leverage agents’ self-interest to focus only on those
between the police and community (Oliver, 2015; Shjarback observable outcomes, which are the basis of rewards or
et al., 2017). Some would suggest civilian oversight possibly sanctions, while neglecting others.
leads to de-policing of police agencies by introducing or In the policing context, civilian oversight primarily focus-
aggravating the conflict between the police and the commu- ing on the citizen complaints process may cause a multitask
nity while focusing on the confrontational nature of civilian problem, encouraging police agencies to exclusively focus on
oversight.5 As Hendrickson and Olson (2016) have sug- citizen complaints and incentivizing them to reduce the
gested, if civilian oversight is built upon confrontation, number of complaints at the expense of other organizational
reflecting the adversarial nature of the police-community performance. Instead of improving other less observable
relationship, the consequences of having civilian oversight organizational performance that could matter more to the
would inevitably be destructive. Civilian oversight would community, such as citizens’ satisfaction, police agencies
intensify tensions between the police and the community may tackle the number of complaints that is a more visible
instead of establishing constructive accountability relation- target for them, following an incentive structure set by civil-
ships and making the police more accountable to citizens. ian oversight. From this “multitask” perspective, the current
However, even when civilian oversight lowers police findings of civilian oversight leading to lower clearance
agencies’ clearance rates, it does not decrease arrest rates could be seen as a consequence of setting an undesir-
counts or undermine crime reduction. The police agencies able incentive structure. If the current study were to con-
with civilian oversight are still making as many arrests clude its findings as a case of a multitask problem, there
and reducing as much crime as the agencies with no civilian should be additional empirical evidence showing that
oversight. In other words, the police presence in the com- civilian oversight leads to an improvement in easily
munity—arresting offenders and controlling crime— observable and quantifiable performance indicators
seems to remain the same regardless of civilian oversight. related to citizen complaints, accompanied by worsening
More importantly, to support the proposition that civilian clearance rates. However, unlike the previous LEMAS
oversight damages police agencies’ organizational perfor- surveys in 2003 and 2007, the LEMAS survey in 2016
mance by aggravating the police-community relationship does not collect and provide the information on citizen
with its confrontational nature, there should be evidence complaints anymore, and thus, without any further analy-
showing that civilian oversight worsens police agencies’ sis, including citizen complaints information, the current
performance in managing the relationships with the com- study alone cannot come to a conclusion in favor of the
munity. Yet, as the study’s findings indicate, civilian over- multitask problem proposition.
sight has no significant negative influence on such To understand civilian oversight’s negative influence on
performance. police organizational performance, future research could
Alternatively, civilian oversight’s negative impact on turn to the job demands-resources (JD-R) model. The focus
police organizational performance could be understood of the current study is on the organizational level, and the
as an example of the multitask problem. Based on JD-R model focuses more on individuals, looking into their
principal-agent theory, Holmstrom and Milgrom (1991) motivational aspects. Yet, the JD-R model offers some mean-
have identified the multitask problem, a tendency of ingful insights for understanding what could have happened
agents to focus on the tasks rewarded by principals while in police agencies with civilian oversight to make them have
neglecting other tasks not relevant to rewards. To align a worsened organizational performance.
agents’ incentives with principals’ goals, principals gener- While theorizing job demands as “aspects of the job that
ally link rewards and sanctions to the outcomes of interest require sustained physical or mental effort and are therefore
(Koppl & Sacks, 2013). However, multitask problems associated with certain physiological and psychological
394 American Review of Public Administration 52(5)

costs” and job resources as “aspects of the job that may do Limitations
any of the following: (a) be functional in achieving work
goals; (b) reduce job demands and the associated physiolog- One major reason this study uses propensity score matching
ical and psychological costs; and (c) stimulate personal analysis is to address the potential selection bias inherent in
growth and development,” the JD-R model suggests that the introduction of civilian oversight among police agencies.
when job demands are constantly high, and job resources However, propensity score matching as an analytic method
are not sufficiently compensating such demands, employees’ itself has a weakness in that it is still susceptible to unobserv-
energy will be drained (Demerouti et al., 2001). In turn, this able covariates like OLS regression does (Dehejia & Wahba,
will eventually lead to adverse outcomes both at the individ- 2002; Li, 2013). To ensure the robustness of its findings, this
ual level (e.g., low work engagement) and organizational study checks the balance between the treatment and control
level (e.g., low performance) as well. groups. Yet, the improved balance of covariates only sug-
In the policing context, civilian oversight could increase gests that these groups are balanced with respect to the base-
job demands in police agencies without providing addi- line covariates included in the analysis.
tional job resources to compensate for the increased Thus, if unobservable variables not accounted for in the
amount of job demands caused by civilian oversight. analysis exist, these variables could still bias the estimation.
Civilian oversight is likely to put more job demands on For instance, major regional characteristics of police agen-
the administrative work as it will require additional cies’ jurisdictions are included as the covariates for the
resources for the works involved in the oversight matching process, but not all of the attributes are addressed in
process. As clearance rates reflect police agencies’ admin- the current analysis. There are other sociodemographic charac-
istrative efforts in dealing with crime, civilian oversight teristics of jurisdictions that could not be included in the analysis
could directly impact their clearance rates more severely due to the limited availability of the data. Although as extensive
than other performance indicators reflecting the efforts in as possible list of covariates is included in the analysis to min-
arresting criminal offenders out on the street, such as imize the concern of omitting such information, the findings of
arrest count or crime reduction. However, this explanation this study are not entirely free from the potential bias of the
also cannot be conclusive as the current study does not regional characteristics of jurisdictions.
provide direct evidence to support it. Moreover, it should also be noted that this study is aware
To more clearly understand if civilian oversight could of recent criticisms on using propensity scores for matching.
impede police organizational performance by increasing job Although widely accepted and applied, matching based on
demands, the future study may investigate individual police propensity scores has been doubted by some scholars
officers’ perceptions. Individual officers’ job demands because it may increase the imbalance between treatment
could be measured by asking about their perceived workload, and control groups rather than balancing them (King &
emotional demands, and organizational changes. Using the Nielsen, 2019), and may also “degrade inference to not
questionnaire items broadly employed in the research apply- matching at all” (King et al., 2011). Thus, the findings
ing the JD-R model (e.g., Xanthopoulou et al., 2007), the need to be carefully understood in consideration of such lim-
future study may ask individual officers if they have to itations of this study.
work too much, confront emotionally upsetting situations, Also, the police organizational performance indicators
adjust to changes in the work, and as such. By understanding in the current study capture a portion of the police organi-
the association between police officers’ perceived job zational performance as they only reflect some of the
demands and civilian oversight, the future study may dimensions of police work. Thus, this study only sheds
unfold the psychological aspects behind civilian oversight light on a partial picture at best. There are other important
as an accountability mechanism and provide further practical dimensions of police organizational performance that
implications for devising more effective and efficient civilian could not be addressed in this study, such as reducing cit-
oversight. izens’ fear of crime, enhancing citizens’ sense of safety,
Overall, the current findings indicate that civilian over- providing quality service to citizens, or maintaining citi-
sight does not significantly improve police agencies’ organi- zens’ satisfaction, not measured in the current study’s
zational performance, not as initially hypothesized in this data. Based on the current findings, this study can say
study. Instead, civilian oversight turns out to lower police nothing about civilian oversights’ influence on any of
agencies’ clearance rates while not significantly influencing police organizational performance related to these dimen-
any other organizational performance. Potential explanations sions. Such a limitation of this study calls for future
for such unexpected and rather undesirable consequences of research focusing more on police organizational perfor-
civilian oversight are discussed. Nevertheless, further empir- mance based on citizens’ perceptions. By analyzing citi-
ical evidence is necessary to support the above propositions if zens’ perceptions, it would be possible to understand
this study were to conclude any of them to be the case and how civilian oversight influences the organizational per-
make suggestions for mending civilian oversight if it is not formance of the police that impacts citizens’ daily lives
functioning as supposedly. and their well-being more directly.
Kim 395

Furthermore, although the current study exclusively reporting purposes (FBI, 2017). Within the UCR program, an
focuses on whether police agencies have civilian oversight agency can only report an offense as cleared when all three fol-
due to the limitation of the data utilized in the analysis, civil- lowing conditions are met: (1) “Arrested;” (2) “Charged” for
ian oversight agencies play multiple roles in practice, and the committing the offense; and (3) “Turned over to the court for
scope of civilian oversight varies across the agencies (Ali & prosecution” (FBI, 2017).
3. According to the UCR program, law enforcement agencies can
Nicholson-Crotty, 2021; Ali & Pirog, 2019; De Angelis et al.,
have a clearance rate higher than 100% as they can clear
2016; Walker, 2001). For instance, modern civilian oversight
offenses that occurred in previous years (FBI, 2017). In this
agencies in the United States are known to perform as inves-
study, a total of three agencies are identified to have a clearance
tigative agencies, auditing or monitoring agencies, review rate higher than 100%. However, these values are not sup-
boards, or commissions (Attard & Olson, 2013). pressed to 100% since they reflect the level of effort the
Considering the scope of civilian oversight or its varied police agencies put in clearing criminal offenses.
roles would possibly offer an opportunity to grasp a more 4. Although a sheriff’s office and a local police department both
detailed understanding of civilian oversight mechanisms— function as law enforcement agencies, they are structurally dif-
how civilian oversight would impact one aspect of police ferent since sheriffs are usually elected officials while chiefs of
agencies’ organizational performance while not influencing police departments are appointed by a government entity.
others at all, for instance. In turn, it will contribute to the bet- Thus, how a sheriff’s office and a local police department
terment of civilian oversight of the police, improving civilian perform as a law enforcement agency and maintain relation-
oversight’s efficiency and effectiveness in making police ships with citizens can be different from one another.
agencies more accountable to the citizens they serve. 5. Research on “police culture” (e.g., Paoline & Gau, 2017;
Lastly, the generalizability of the findings could be limited Workman-Stark, 2017) and “social identity theory” (e.g.,
Ashforth & Mael, 1989; Horwitz & Rabble, 1982; Turner,
since this study looks into the causality between civilian
1975) may provide further explanations on how intergroup
oversight and police organizational performance within a
conflicts arise between the police and citizens. However, it
single year. This study does not capture trends such as possi- will not be empirically analyzed in this study since it is
ble changes in dynamics between civilian oversight and the beyond the scope of analysis.
police or the relationship between the police and the commu-
nity they serve over time. Future research needs to address References
this limitation by examining whether the findings of this
Ali, M. U., & Nicholson-Crotty, S. (2021). Examining the
study remain constant even in a study analyzing the data accountability-performance link: The case of citizen oversight
with a broader time frame. of police. Public Performance & Management Review, 44(3),
523–559. https://doi.org/10.1080/15309576.2020.1806086
Declaration of Conflicting Interests Ali, M. U., & Pirog, M. (2019). Social accountability and institu-
The author(s) declared no potential conflicts of interest with respect tional change: The case of citizen oversight of police. Public
to the research, authorship, and/or publication of this article. Administration Review, 79(3), 411–426. https://doi.org/10.
1111/puar.13055
Alpert, G. P., Cawthray, T., Rojek, J., & Ferdik, F. V. (2016).
Funding
Citizen oversight in the United States and Canada: Applying
The author(s) received no financial support for the research, author- outcome measures and evidence-based concepts. In
ship, and/or publication of this article. T. Prenzler & G. Heyer (Eds.), Civilian oversight of police:
Advancing accountability in law enforcement (pp. 179–204).
ORCID iD CRC Press.
Jisang Kim https://orcid.org/0000-0002-8038-984X Ashforth, B. E., & Mael, F. (1989). Social identity theory and the
organization. Academy of Management Review, 14(1), 20–39.
https://doi.org/10.5465/amr.1989.4278999
Notes
Attard, B., & Olson, K. (2013). Overview of civilian oversight of law
1. Civilian oversight agencies can play varied roles at the same enforcement in the United States. http://nacole.org/wp-content/
time. Previous studies (Walker, 2001; De Angelis et al., uploads/Oversight-in-the-United-States-Attard-and-Olson-2013.
2016) have recognized more of “hybrid” or “holistic” agencies pdf
performing such multiple roles in practice. These multiple roles Austin, P. C. (2011). An introduction to propensity score methods
of civilian oversight agencies may have differential impacts on for reducing the effects of confounding in observational
police organizations. However, due to a lack of information studies. Multivariate Behavioral Research, 46(3), 399–424.
from the current data, this study limits its focus to the presence https://doi.org/10.1080/00273171.2011.568786
of civilian oversight. Bovens, M. (2007). Analysing and assessing accountability: A con-
2. As police agencies report the closed cases themselves, possible ceptual framework. European Law Journal, 13(4), 447–468.
arbitrary decisions made by police agencies may raise concerns https://doi.org/10.1111/j.1468-0386.2007.00378.x
regarding their clearance rates. However, even when law Bovens, M. (2010). Two concepts of accountability: Accountability
enforcement agencies close offenses administratively, the as a virtue and as a mechanism. West European Politics, 33(5),
UCR program does not allow agencies to clear cases for 946–967. https://doi.org/10.1080/01402382.2010.486119
396 American Review of Public Administration 52(5)

Bureau of Justice Statistics. (2016). Law Enforcement Management Hong, S. (2016). Does increasing ethnic representativeness reduce
and Administrative Statistics (LEMAS), 2016. Inter-university police misconduct? Public Administration Review, 77(2),
Consortium for Political and Social Research. 195–205. https://doi.org/10.1111/puar.12629
Caliendo, M., & Kopeinig, S. (2008). Some practical guidance for Horwitz, M., & Rabbie, J. M. (1982). Individuality and membership in
the implementation of propensity score matching. Journal of the intergroup system. In H. Tajfel (Ed.), Social identity and inter-
Economic Surveys, 22(1), 31–72. https://doi.org/10.1111/j. group relations (pp. 241–274). Cambridge University Press.
1467-6419.2007.00527.x Imbens, G. W. (2004). Nonparametric estimation of average treat-
Clarke, S. (2009). Arrested oversight: A comparative analysis and ment effects under exogeneity: A review. The Review of
case study of how civilian oversight of the police should func- Economics and Statistics, 86(1), 4–29. https://doi.org/10.
tion and how it fails. Columbia Journal of Law and Social 1162/003465304323023651
Problems, 43(1), 1–49. https://www.proquest.com/ Kerstetter, W. A. (1985). Who disciplines the police? Who should?
scholarlyjournals/ arrested-oversight-comparative-analysis- In W. A. Geller (Ed.), Police leadership in America: Crisis and
case/docview/2003018148/se-2?accountid=14166 opportunity (pp. 149–182). Praeger.
Day, P., & Klein, R. (1987). Accountabilities: Five public services. Kerstetter, W. A., & Rasinski, K. A. (1994). Opening a window into
Tavistock. police internal affairs: Impact of procedural justice reform on
De Angelis, J. (2009). Assessing the impact of oversight and proce- third-party attitudes. Social Justice Research, 7(2), 107–127.
dural justice on the attitudes of individuals who file police com- https://doi.org/10.1007/BF02337294
plaints. Police Quarterly, 12(2), 214–236. https://doi.org/10. King, G., & Nielsen, R. (2019). Why propensity scores should not
1177/1098611109332425 be used for matching. Political Analysis, 27(4), 435–454.
De Angelis, J., Rosenthal, R., & Buchner, B. (2016). Civilian oversight https://doi.org/10.1017/pan.2019.11
of law enforcement: Assessing the evidence. U.S. Department of King, G., Nielsen, R., Coberley, C., Pope, J. E., & Wells, A. (2011).
Justice, Office of Justice Programs Diagnostic Center. Comparative effectiveness of matching methods for causal
Dehejia, R. H., & Wahba, S. (2002). Propensity score-matching inference. http://j.mp/2nydGlv
methods for nonexperimental causal studies. Review of Koppl, R., & Sacks, M. (2013). The criminal justice system creates
Economics and Statistics, 84(1), 151–161. https://doi.org/10. incentives for false convictions. Criminal Justice Ethics, 32(2),
1162/003465302317331982 126–162. https://doi.org/10.1080/0731129X.2013.817070
Demerouti, E., Bakker, A. B., Nachreiner, F., & Schaufeli, W. B. Li, M. (2013). Using the propensity score method to estimate causal effects:
(2001). The job demands-resources model of burnout. A review and practical guide. Organizational Research Methods,
Journal of Applied Psychology, 86(3), 499–512. https://doi. 16(2), 188–226. https://doi.org/10.1177/1094428112447816
org/10.1037/0021-9010.86.3.499 Libonati, C. (2021, September 7). Syracuse police release first batch of
EPA. (1999). EPA’s Framework for Community-Based traffic stop data mandated by reform legislation. The Post-Standard.
Environmental Protection. https://www.syracuse.com/crime/2021/09/syracuse-police-release-
EPA. (2003). Evaluation of Community-Based Environmental first-batch-of-traffic-stop-data-mandated-by-reform-legislation.html
Protection Projects: Accomplishments and Lessons Learned. Lipsky, M. (1980). Street-level bureaucracy: The critical role of
FBI. (2016a). Uniform crime reporting program data: Arrests by street-level bureaucrats. In J. M. Shafritz & A. C. Hyde
age, sex, and race, summarized yearly, 2016. Inter-university (Eds.), Classics of public administration (8th ed., pp. 402–
Consortium for Political and Social Research. 409). Cengage Learning.
FBI. (2016b). Uniform crime reporting program data: Offenses Lohmann, P., & Libonati, C. (2020, July 13). Syracuse police took
known and clearances by arrest, 2016. Inter-university so long on misconduct investigations that some officers can’t be
Consortium for Political and Social Research. disciplined. The Post-Standard. https://www.syracuse.com/
FBI. (2016c). Uniform crime reporting program data: Police news/2020/07/syracuse-police-took-so-long-on-misconduct-
employee (LEOKA) data, 2016. Inter-university Consortium investigations-that-some-officers-cant-be-disciplined.html
for Political and Social Research. Mas, A. (2006). Pay, reference points, and police performance. The
FBI. (2017). Crime in the United States, 2016. https://ucr.fbi.gov/ Quarterly Journal of Economics, 121(3), 783–821. https://doi.
crime-in-the-u.s/2017 org/10.1162/qjec.121.3.783
Ferdik, F. V., Rojek, J., & Alpert, G. P. (2013). Citizen oversight in the Maslov, A. (2015). Measuring the performance of the police: The
United States and Canada: An overview. Police Practice and perspective of the public. Public Safety Canada.
Research, 14(2), 104–116. https://doi.org/10.1080/15614263. McCubbins, M. D., Noll, R. G., & Weingast, B. R. (1987).
2013.767089 Administrative procedures as instruments of political control.
Finn, P. (2001). Citizen review of the police: Approaches and imple- Journal of Law, Economics, & Organization, 3(2), 243–277.
mentation. Office of Justice Programs, National Institute of Justice. https://doi.org/10.1093/oxfordjournals.jleo.a036930
Hendrickson, K., & Olson, K. (2016). Introduction, NACOLE/CJPR McMurry, T. L., Hu, Y., Blackstone, E. H., & Kozower, B. D.
special issue: Moving beyond discipline—the role of civilians in (2015). Propensity scores: Methods, considerations, and appli-
police accountability. Criminal Justice Policy Review, 27(5), cations in the journal of thoracic and cardiovascular surgery.
447–454. https://doi.org/10.1177/0887403415594844 The Journal of Thoracic and Cardiovascular Surgery, 150(1),
Holmstrom, B., & Milgrom, P. (1991). Multitask principal-agent 14–19. https://doi.org/10.1016/j.jtcvs.2015.03.057
analyses: Incentive contracts, asset ownership and job design. Moore, M. H., & Braga, A. A. (2003a). The “bottom line” of polic-
Journal of Law, Economics and Organization, 7(Special ing: What citizens should value (and measure!) in police perfor-
Issue), 24–52. https://doi.org/10.1093/jleo/7.special_issue.24 mance. Police Executive Research Forum.
Kim 397

Moore, M. H., & Braga, A. A. (2003b). Measuring and improv- Sparrow, M. K. (2015). Measuring performance in a modern police
ing police performance: The lessons of compstat and its organization. Psychosociological Issues in Human Resource
progeny. Policing: An International Journal of Police Management, 3(2), 17–52. https://link.gale.com/apps/doc/
Strategies & Management, 26(3), 439–453. https://doi.org/ A436544001/AONE?u=albanyu&sid=bookmark-AONE&xid=
10.1108/13639510310489485 2bbea7cb
Moynihan, D. P. (2003). Normative and instrumental perspectives StataCorp. (2019). Stata statistical software: Release 16. StataCorp
on public participation: Citizens summits in Washington, LLC.
D.C. American Review of Public Administration, 33(2), 164– Syracuse Police Accountability and Reform Coalition. (2021,
188. https://doi.org/10.1177/0275074003251379 September). The people’s agenda for policing. SPAARC.
Mulgan, R. (2003). Holding power to account: Accountability in https://spaarc315.org/?page_id=39
modern democracies. Palgrave. Turner, J. C. (1975). Social comparison and social identity: Some
Nabatchi, T. (2010). Addressing the citizenship and democratic def- prospects for intergroup behaviour. European Journal of
icits: The potential of deliberative democracy for public admin- Social Psychology, 5(1), 5–34. https://doi.org/10.1002/ejsp.
istration. American Review of Public Administration, 40(4), 2420050102
376–399. https://doi.org/10.1177/0275074009356467 Walker, S. (2001). Police accountability: The role of citizen over-
Neshkova, M. I., & Guo, H. (2012). Public participation and orga- sight. Wadsworth/Thomson Learning.
nizational performance. Journal of Public Administration Walker, S. (2006). The history of the citizen oversight. In J.
Research and Theory, 22(2), 267–288. https://doi.org/10. C. Perino (Ed.), Citizen oversight of law enforcement agencies
1093/jopart/mur038 (pp. 1–10). ABA Book Publishing.
Nicholson-Crotty, S., & O’Toole, L. J. Jr . (2004). Public manage- Weisburd, D., Hasisi, B., Jonathan, T., & Aviv, G. (2010). Terrorist
ment and organizational performance: The case of law enforce- threats and police performance: A study of Israeli communities.
ment agencies. Journal of Public Administration Research and British Journal of Criminology, 50(4), 725–747. https://doi.org/
Theory, 14(1), 1–18. https://doi.org/10.1093/jopart/muh001 10.1093/bjc/azp064
Oliver, W. M. (2015). Depolicing: Rhetoric or reality? Criminal Wells, W., & Schafer, J. A. (2007). Police skepticism of citizen
Justice Policy Review, 28(5), 437–461. https://doi.org/10. oversight: Officers’ attitudes towards specific functions, pro-
1177/0887403415586790 cesses, and outcomes. Journal of Crime and Justice, 30(2),
Paoline, E. A.III, & Gau, J. M. (2017). Police occupational culture: 1–25. https://doi.org/10.1080/0735648X.2007.9721233
Testing the monolithic model. Justice Quarterly, 35(4), 670– Wilson, S., & Buckler, S. (2010). The debate over police reform:
698. https://doi.org/10.1080/07418825.2017.1335764 Examining minority support for citizen oversight and resistance
Philp, M. (2009). Delimiting democratic accountability. Political by police unions. American Journal of Criminal Justice, 35(4),
Studies, 57(1), 28–53. https://doi.org/10.1111/j.1467-9248. 184–197. https://doi.org/10.1007/s12103-010-9079-x
2008.00720.x Workman-Stark, A. L. (2017). Understanding police culture. In
Potoski, M., & Woods, N. D. (2001). Designing state clean air agencies: Inclusive policing from the inside out (pp. 19–35). Springer.
Administrative procedures and bureaucratic autonomy. Journal of Worrall, J. L. (2002). If you build it, they will come:
Public Administration Research and Theory, 11(2), 203–222. Consequences of improved citizen complaint review proce-
https://doi.org/10.1093/oxfordjournals.jpart.a003499 dures. Crime & Delinquency, 48(3), 355–379. https://doi.
Prenzler, T. (2011). The evolution of police oversight in Australia. org/10.1177/0011128702048003001
Policing and Society, 21(3), 284–303. https://doi.org/10.1080/ Worrall, J. L. (2019). Investigative resources and crime clearances: A
10439463.2011.570866 group-based trajectory approach. Criminal Justice Policy Review,
President’s Task Force on 21st Century Policing. (2020). Final 30(2), 155–175. https://doi.org/10.1177/0887403416650251
report of the President’s task force on 21st century policing. Xanthopoulou, D., Bakker, A. B., Demerouti, E., & Schaufeli, W. B.
Office of Community Oriented Policing Services. (2007). The role of personal resources in the job demands-
Roberts, N. C. (1997). Public deliberation: An alternative approach resources model. International Journal of Stress Management,
to crafting policy and setting direction. Public Administration 14(2), 121–141. https://doi.org/10.1037/1072-5245.14.2.121
Review, 57(2), 124–132. https://doi.org/10.2307/977060
Scott, C. (2000). Accountability in the regulatory state. Journal of
Law and Society, 27(1), 38–60. https://doi.org/10.1111/1467- Author Biography
6478.00146 Jisang Kim is a Ph.D. candidate at the Department of Public
Shjarback, J. A., Pyrooz, D. C., Wolfe, S. E., & Decker, S. H. (2017). Administration and Policy and a full-time research associate with
De-policing and crime in the wake of ferguson: Racialized the Center for Women in Government and Civil Society at the
changes in the quantity and quality of policing among Missouri Rockefeller College of Public Affairs and Policy, University at
police departments. Journal of Criminal Justice, 50, 42–52. Albany – SUNY. His research interests lie in furthering the under-
https://doi.org/10.1016/j.jcrimjus.2017.04.003 standing of accountability in relation to performance management
Skolnick, J. H., & Fyfe, J. J. (1993). Police accountability ii: The and human resource management in the public sector. Other
public. In J. H. Skolnick & J. J. Fyfe (Eds.), Above the law: research interests include gender and diversity issues in public
Police and the excessive use of force (pp. 217–236). Free Press. administration.