629180

2016
EDMXXX10.1177/1555343416629180Month XXXXPrioritization: A Double-Edged Sword?

Prioritization: A Double-Edged Sword?

Catherine M. Burns, University of Waterloo, and
Neelam Naikar, Defence Science and Technology Group

In analyzing and designing new decision the constraints of implementation and the need
­support systems, some key tensions arise. How for an effective result, broad and deep may be
do you keep your analysis broad enough to at odds with an effective final design. In their
cover a wide range of needs, some of which article, Bodin and Krupenia have proposed one
may not have been fully formulated? How do approach for making CWA efficient, specifi-
you maintain enough focus so that key tasks cally an approach based on prioritization.
can be executed quickly, efficiently, and with a In general, the authors’ approach involves
good user experience? Against these fore- conducting a work domain analysis and a con-
ground questions there is a continual back- trol task analysis and then applying a prioritiza-
ground context of limited budget, resources, tion technique to the latter to determine where
and time to develop a solution that will make a the subsequent analysis and design effort should
difference. Bodin and Krupenia’s paper is moti- be focused. Their prioritization technique, which
vated by such pragmatic concerns. Specifically, is demonstrated in the context of long haulage
the authors propose an approach for prioritizing trucking, essentially involves establishing which
the analytical effort in utilizing the first two of the object-related processes and purpose-
dimensions of cognitive work analysis (CWA), related functions in their contextual activity
work domain analysis and control task analysis, template of the system are the most frequently
for design. In this commentary, we discuss the occurring ones. These object-related processes
possible risks of prioritization and present some and purpose-related functions are obtained from
solutions to aid prioritization, in ways that are their work domain model of the system. Our
consistent with a CWA approach. commentary sheds light on the benefits, and
drawbacks, to this prioritization approach.
The Risk of Prioritization in CWA
First, we acknowledge that there are good
CWA is an analysis approach that is often reasons to prioritize your approach by the fre-
used in large complex systems where the space quency of situations. These are the use cases that
of potential actions and events can be very system users will encounter most often, and an
large. CWA describes cognitive work in a efficient and effective interaction in these cases
broad sense, over a series of models that take has many benefits. It is a principle of good
different looks at the work, considering the design to ensure that these tasks are efficient and
sociotechnical environment, human information provide a good experience for users.
processing, strategies, social organizational con- However, it is important to remember that
straints, and worker competencies. The analysis CWA evolved from the need to design for com-
is simultaneously both broad and deep, as suits plex safety critical systems. In these kinds of sys-
the approach’s initially intended domains of tems, events cannot be fully anticipated. Some of
complex and safety critical systems. But given the largest and most costly accidents have
resulted from these unanticipated events (e.g.,
Three Mile Island, Bhopal, and Chernobyl;
Address correspondence to Catherine M. Burns, University Meshkati, 1991). In evaluations of ecological
of Waterloo, Waterloo, ON, Canada, [email protected].
interface design (Burns & Hajdukiewicz, 2004),
Journal of Cognitive Engineering and Decision Making
the design approach linked to CWA, ecological
2016, Volume 10, Number 1, March 2016, pp. 105­–108 designs are often found to support improved
DOI: 10.1177/1555343416629180 diagnosis and response to more complex
Copyright © 2016, Human Factors and Ergonomics Society. events—and typically less predictable or f­ requent

Downloaded from edm.sagepub.com at Auburn University on March 1, 2016

106 March 2016 - Journal of Cognitive Engineering and Decision Making

events—with perhaps equal or indifferent of situations. These constraints are typically mod-
response to detection tasks or predictable events eled with a contextual activity template or set of
(Burns et al., 2008; Vicente, 2002). Although we decision ladders, or both. Within the contextual
acknowledge the pressures and benefits that activity template, the constraints are modeled as a
could be realized by prioritizing cognitive work, combination of work situations and work func-
we are left with a critical question: Where can tions (Naikar, Moylan, & Pearce, 2006). Subse-
that prioritization be done safely without risking quently, decision ladders may be created for these
the response to critical and unanticipated work situations or work functions to depict activity-
events—the particular space where CWA was related constraints at a more detailed level, spe-
designed to aid decision making? Prioritization cifically in terms of control tasks relating to
must occur in such ways that the key benefits of observation of information, situation analysis,
an analysis like CWA are not lost. In the rest of goal evaluation, and planning and execution.
this section, we explore why prioritization by Arguably, it may be more appropriate to pri-
frequency could be dangerous. oritize the constraints associated with recurring
The challenge, in particular, is that unantici- classes of situations in terms of frequency than it
pated events are by their very nature often infre- is to prioritize work domain constraints in this
quent (in fact, they have probably never occurred manner. Providing users with support to deal
by the time of the analysis) as well as being with the recurring events they will encounter
unpredicted. They may have never happened most frequently out of the full set of anticipated
before, and they may never happen in the future, situations seems, on the face of it, reasonable.
although the latent possibility of such an event However, the problem with this approach is that
continues to exist. CWA provides support for it may fail to provide users with support for less
these situations by providing a constraint-based frequent events that are in fact more demanding
view of the workspace, primarily through the than the most frequently occurring ones. Indeed,
abstraction–decomposition space and abstrac- it is a commonly reported problem, particularly
tion hierarchy to connect the components of the within the automation literature, that users find
system through their processes to their intended they are provided with support when they do not
purpose. The intention of this space is to create a really need it and yet are left to their own devices
map of connections that will allow a user to nav- on the very occasions when they do need help,
igate through the work domain and develop rea- specifically when the situations are most demand-
soned solutions by understanding how compo- ing (e.g., Bainbridge, 1983; Kirlik, 1993; ­Wiener,
nents work and connect. In this way, designs 1989; Woods & Hollnagel, 2006).
developed on a work domain analysis support
diagnostic decision-making in unanticipated sit- Solutions
uations. Regardless of the situation, the func-
tion, purpose, and capabilities of the resources in Although prioritization by the frequency of
the work domain remain, and effective ecologi- use of work domain and other elements has risks,
cal design shows these connections. As a result, there are other approaches that can be used to
prioritization of analysis, or design, based on provide the benefits of prioritization, in a way
frequency of use of work domain elements is that is consistent with the intention of CWA.
inherently risky. It can modify the view of the
work domain such that the focus is only on Prioritization by Function
nodes and connections accessed frequently and In the work domain analysis, there are two
could therefore endanger resilient behavior in levels that can be useful in setting priorities.
the case of infrequent (and therefore nonpriori- At the Functional Purpose level, there are
tized) but dangerous events. often multiple purposes and some prioritization
CWA, however, is not concerned solely with approaches may be apparent at this level. For
unanticipated events. Control task analysis, the example, a plant with the purpose of producing
second dimension of the framework, focuses on electricity while maintaining safety may quite
the constraints associated with recurring classes rationally prioritize safety over generation. This

Downloaded from edm.sagepub.com at Auburn University on March 1, 2016

Prioritization: A Double-Edged Sword? 107

tradeoff should appear at the Value and Priority Object-related processes and purpose-related
Measures level (also called Abstract Function, functions are not activities—they are affordances
as in Rasmussen, 1985), where values and prior- of the work domain. Hence, they rightfully
ities are most effectively modeled. Showing this belong in a work domain model. The authors
tradeoff point can be useful to decision makers may not have run into the need for prioritization
to understand when certain functions are at risk if they had represented work functions, instead of
(e.g., the system is advancing to an unsafe state) object-related processes and purpose-related
yet does not constrain or limit the scope of the functions, in their contextual activity template.
decision maker to respond to the situation. This Specifically, as the work functions close the
approach essentially moves the prioritization degrees of freedom, or space of possibilities, for
decision from the designer to the operator, while behavior afforded by the work domain, there
giving the operator the support needed to make should be considerably fewer work functions
that decision. Given that the designer cannot, by than the large number of object-related processes
definition, make prioritization decisions in rela- and purpose-related functions identified by the
tion to events that cannot be predicted, giving authors. Hence, the need for prioritization argued
users the discretion to make these choices when by the authors may in fact be an artifact of their
novel events do occur is essential. contextual activity template, which is inconsis-
tent with the principles of CWA.
Prioritization by Activity
Summary
Potentially, one way in which prioritization
could be conducted within control task analysis, In summary, we recognize and acknowledge
or for recurring classes of situations, is by focus- the benefits of being able to prioritize your
ing on demanding activities. Within the CWA CWA approach to have a more efficient design
framework, demanding activities may be viewed, process and successful design. We think there
for example, as those that are cognitively chal- are alternatives to prioritization by frequency
lenging or that have serious consequences for that may be more in keeping with the intentions
system safety and productivity. However, this of CWA and the need to retain support for unan-
approach is dependent on effective strategies for ticipated events. We would encourage further
pinpointing the demanding activities. This is not a discussion on this matter, as all efforts to gener-
trivial problem when one considers, for instance, ate better decision support and better designs are
that it may not be sufficient to examine each ele- worthwhile and challenge us to look at our ideas
ment of a control task analysis in isolation. As and practice more deeply.
observed by Naikar et al. (2006), numerous com-
binations of work situations, work functions, and Acknowledgments
control tasks are possible, which impose qualita- We thank Bodin and Krupenia for opening up
tively different cognitive demands on actors. their paper for commentary and Ann Bisantz and
Perhaps a more important issue to address, Amy Pritchett, associate editor and editor of the Cog-
then, is the need for prioritization within a con- nitive Engineering and Decision Making journal, for
trol task analysis in the first place. Bodin and giving us the opportunity to write this commentary.
Krupenia discuss that the reason they undertook
a prioritization process was because of the large References
number of “activities” in their contextual activity Bainbridge, L. (1983). Ironies of automation. Automatica, 19(6),
template. However, following the example set by 775–779.
Burns, C. M., & Hajdukiewicz, J. R. (2004). Ecological interface
some other publications (e.g., Stanton & McIl- design. Boca Raton, FL: CRC Press.
roy, 2010), their contextual activity template Burns, C. M., Skraaning, G., Jamieson, G., Lau, N., Kwok, J.,
actually included work domain elements. That is, Welch, R., & Andresen, G. (2008). Evaluation of ecological
instead of the work functions that rightfully interface design for nuclear process control: Situation aware-
ness effects. Human Factors, 50, 663–679.
belong in a contextual activity template, their Kirlik, A. (1993). Modeling strategic behavior in human-automa-
model focused initially on object-related pro- tion interaction: Why an “aid” can (and should) go unused.
cesses and then on purpose-related functions. Human Factors, 35(2), 221–242.

Downloaded from edm.sagepub.com at Auburn University on March 1, 2016

108 March 2016 - Journal of Cognitive Engineering and Decision Making

Meshkati, N. (1991). Human factors in large-scale technological Stanton, N. A., & McIlroy, R. C. (2010). Designing mission communi-
systems’ accidents: Three Mile Island, Bhopal, Chernobyl. cation planning: The role of rich pictures and cognitive work anal-
Organization and Environment, 5, 133–154. ysis. Theoretical Issues in Ergonomics Science, 13(2), 146–168.
Naikar, N., Moylan, A., & Pearce, B. (2006). Analysing activity Vicente, K. J. (2002). Ecological interface design: Progress and
in complex systems with cognitive work analysis: Concepts, challenges. Human Factors, 44, 62–78.
guidelines, and case study for control task analysis. Theoretical Wiener, E. L. (1989). Human factors of advanced technology (“glass
Issues in Ergonomics Science, 7(4), 371–394. cockpit”) transport aircraft (NASA Contractor Report No.
Rasmussen, J. (1985). The role of hierarchical knowledge rep- 177528). Moffett Field, CA: NASA-Ames Research Center.
resentation in decision making and system management. Woods, D. D., & Hollnagel, E. (2006). Joint cognitive systems:
IEEE Transactions on Systems, Man and Cybernetics, 15, Patterns in cognitive systems engineering. Boca Raton, FL:
234–243. Taylor & Francis.

Downloaded from edm.sagepub.com at Auburn University on March 1, 2016