Supporting Delivery of

Resilient Software, or
“The Jenga View of
Threat Modeling”
by Adam Shostack
Summary
This paper presents a new view of how security
can be a part of technology designs from the
start. We introduce a metaphor of Jenga blocks
supporting a secure product. In the game of
Jenga, there’s a tower of blocks. Play involves
removing them one at a time, and then piling
the removed blocks on top of the tower until
it collapses. In the business world, we aim
to construct a tower that is both stable and
lightweight, using only as many blocks as
needed, because each block comes at a cost.

This paper is intended for leadership in engineering and application security.

Security can and should be a part of the design of new products, services,
features, system architecture and other technological systems. Including
security at design time results in faster and more predictable launches of
better designs with fewer security problems.

1122 E Pike St #1299, Seattle WA 98122 2 +1 917 391 2168 ■ [email protected] ■ associates.shostack.org
Context
Security, in the sense of resilience to attack, is increasingly acknowledged by
executives as a requirement. This requirement often leads to the creation of a
secure development lifecycle program (SDL or SSDL). These programs often
have early wins from penetration testing, fuzz testing or static analysis.
As these programs start, they often focus on tools and outsource-friendly
techniques, which may limit their impact. They may reduce bugs without
dramatically improving resilience.

Threat modeling is a key to achieving resilience. Without we need more than just technical skills in threat modeling.
threat modeling, without considering what could go wrong, We need soft skills such as active listening, respect, and
it is hard to have confidence that software or a service will assumption of good intentions. We need organizational
be free of unpleasant and hard to fix surprises. Tools and support, including processes and policies. We need
automation are important, but as long as software devel- standards for how the work is to be done, including gates
opment requires judgement and human decisions, some and nets to prevent or catch mistakes. We might want
of those decisions will involve security or have security software to help, and training that covers each of these.
effects that tools don’t see. To support the goal of resilient software, we have
Structure is key to any practice growing from something started using a metaphor of Jenga blocks, where each
that a few people do to something an organization does. block is a skill, technique, or tool that helps ensure appro-
Without structure, it’s random activity. It’s hard to measure priate threat modeling. The blocks come in three flavors:
or justify. It’s hard to judge if the work has been productive. technical, interpersonal, and organizational. With more
Threat modeling has been an artisanal activity that some blocks in place, the structure is more stable (the real game
people perform to great effect. As the discipline matures, it of Jenga is more complex, but our metaphor does not
has become clear that to reach the goal of resilient software include piling blocks on top.)

1122 E Pike St #1299, Seattle WA 98122 3 +1 917 391 2168 ■ [email protected] ■ associates.shostack.org
 Types of
Building Blocks
A successful threat modeling
program includes:
technical

interpersonal

organizational

building blocks. The mix of blocks

is different from those needed to
support other software security
activities. For example, static
analysis is very much centered on
technical blocks, and interpersonal
ones are less important.

1122 E Pike St #1299, Seattle WA 98122 4 +1 917 391 2168 ■ [email protected] ■ associates.shostack.org
Technical Building Blocks for Threat Modeling
The technical skills of threat modeling are the ones that in explaining “what are we working on?” Various forms
help us answer the first three questions in the four-ques- of STRIDE and Kill Chains can help address “what can
tion framework: go wrong?” STRIDE is a mnemonic of various threats
(Spoofing, Tampering, Repudiation, Information disclo-
1. What are we working on? sure, Denial of service and Elevation of privilege), while
Kill Chains are a model of how attackers chain activities
■ M odel systems using approaches like white- together to reach (kill) their objectives.
boards, data flow diagrams, state machines, Security professionals are often advised to start from
swim lanes and other diagramming techniques risk assessment. My experience is there are often improve-
■ Specific modeling techniques such as C4, DFD3 ments that can be made faster than assessing their risk.
or UML Examples include moving files from /tmp or changing
permissions on an S3 bucket. When the analysis is harder,
various prioritization analyses can help.
2. What can go wrong?
Additional technical skills and knowledge-style building
blocks are broadly helpful, and many require no threat
■ D iscover threats using STRIDE or Kill Chain
modeling specific adaptation. These include:
■ Analogize from existing threats using CAPEC or
ATT&CK
■ Educate peers about technical threats ■ Critical thinking

■ Knowledge of a repertoire of attacks

3. What are we going to do about it? 1
■ G
 eneral technical knowledge of the systems
■ A dvise on remediation approaches being used
■ Analyze prioritization approaches
■ Write tests for TDD, acceptance testing, or ■ U
 nderstanding of software delivery models
anywhere in between including DevOps, agile, Scrum and the local
customizations to these models
■ File actionable, compelling bugs or tickets
■ Write user stories, epics, or reports to capture ■ Teaching (especially around security)
findings
■ A
 gile approaches to work, including small
For example, being able to draw a data flow diagram incremental delivery, testing, and improvement.
(DFD) is a technical skill that helps people be precise

The fourth question, “Did we do a good job” does double duty, giving a place to assess work at in both technical and organizational senses.
1

1122 E Pike St #1299, Seattle WA 98122 5 +1 917 391 2168 ■ [email protected] ■ associates.shostack.org
Interpersonal Building Blocks for Threat Modeling
The best technical skills in the world can be overshadowed Of course, any of these can be taken to a counter-pro-
by interpersonal issues. Even with the best technical skills, ductive extreme. Moreover, we prefer to label these as
interpersonal issues can not only ruin a threat modeling “interpersonal skills” because “soft skills” get a bad rap,
session, but can also shake trust, destroy rapport, and often for being imprecise. Each of these is specific enough
reduce collaboration between security and other teams. to be taught or assessed.
Critical analysis is harder to accept when delivered conde-
scendingly. Even good suggestions from “know-it-alls” are
left on the cutting room floor.
In contrast, effective communication has many
elements that we see consistently across very different work
cultures. Those elements include:

■ Active listening

■ Focus on solutions

■ Patience

■ Humility

■ Respect

■ Assumption of good intent

■ Moderation and facilitation

■ Understanding the working culture

■ Working the organization2

■ Developing a support network

■ Commitments and predictability

1122 E Pike St #1299, Seattle WA 98122 6 +1 917 391 2168 ■ [email protected] ■ associates.shostack.org
Organizational Building Blocks for Threat Modeling
We split organizational blocks into rough groups of gover- Operations
nance and operations. Governance means that the execu-
tive leadership has accepted the importance of the work, 1. Roles and responsibilities
possibly incorporated into a vision such as secure by design
or product security, and assigned someone to govern it. ■ W ho is responsible for what?
That person is accountable for the operational decisions to ■ Are new roles created, or are existing roles
implement the goal. altered? For example, are there security architects
To this point, we have been considering threat modeling or champions?
not only as a set of activities, but also the skills that allow ■ How does threat modeling relate to expectations
people to execute on those activities. Some of those activities at different levels of seniority?
have outputs (deliverables), and those can be specified by an ■ Who is empowered to sign off on exceptions, and
organization to great effect. Other activities simply inform how are those tracked?
the work, but the steps are not written down. For example, if ■ Are meetings expected, and who participates in
a back of an envelope analysis quickly shows that a possible what capacity?
design choice has many side effects, recording that the
analysis was done may not be worthwhile. 2. Enablement

Governance ■ H ow are people made aware of the change?

■ What training do people need to enable them to
■ Clear goals meet the new requirements?
■ What support is offered (Office hours, Slack
■ Executive support channel)? What SLA is committed?

■ Defined stakeholders and accountability

3. Deliverables (new and changed)
■ Definitions, monitoring, and optimization
■ H ow and where are the deliverables stored?
■ What form do they take, or what existing forms
are altered?

Working the organization essentially means before a formal meeting, informally talking to those who will make a decision and those who influence
2

them. These conversations provide an opportunity to gather feedback and address criticisms before a formal decision.

1122 E Pike St #1299, Seattle WA 98122 7 +1 917 391 2168 ■ [email protected] ■ associates.shostack.org
 4. Software (threat modeling software
can help with or enforce) Software
People often want to threat model in a specific
■ P rocess: Software can help model systems, tool. This as a common approach; the other
discover threats and/or mitigations, and connect common approach is to integrate threat models
to gates and nets. into other software artifacts. In that case, the
■ Documentation: software can help produce software, such as Jira or Word, is not threat
documentation of the threat model (system modeling specific. And threat modeling is more
models, analysis and mitigations.) It can also dependent on human thought than other aspects
produce documentation of what’s being done, of secure software development, which are more
and integrate that into systems for process amenable to automation.
visibility.
■ Automation: Discover threat types, file bugs, track
changes, and the like.

Roles and Responsibilities

5. Standards, policies, and procedures
Software and operations engineers must be deeply
■ G ates and nets 3 involved in considering what can go wrong and
■ What deliverables become gated on threat what are they going to do about it. When they are
modeling deliverables? not, the results will be less impactful. They can and
should be supported by people in other roles.
Of course, as statistician George Box teaches us, “all
models are wrong and some models are useful.” Some
blocks may fit into several of the above categories without
detracting from the usefulness. For example, software that
helps technical execution of threat modeling is a technical
building block as it helps answer the question “what can
go wrong?”, and an organizational block when it helps
measure the program.

Gates and nets is a way of thinking about how projects proceed. Hollywood famously “green lights” projects: it’s a known gate that projects must
3

pass through. Other projects proceed on their own, but with safety nets to catch their failures, possibly including penetration testing or bug
bounties. Smart organizations will often use and instrument nets, and convert the common failures to mandated gates of various types.

1122 E Pike St #1299, Seattle WA 98122 8 +1 917 391 2168 ■ [email protected] ■ associates.shostack.org
Conclusion:
What Structure Do You Need?
As anyone who’s played Jenga knows, a tower is stable when the game begins,
and as blocks are removed, stability decreases until the whole thing comes
crashing down.

W e remove blocks in the game because that’s the

game. But in business, we may not start from a
stable tower. We are probably missing blocks when we start,
If you have a security or threat modeling program that’s
not standing up on its own, perhaps studying the building
blocks will help you see that you’ve been focused on
and need to insert some. Also, we remove blocks because technical skills, or organizational matters such as software,
each block has a cost. There’s the cost of executing on the at the expense of interpersonal skills. If your program
block, training, measuring, dealing with exceptions and has collapsed under its own weight or been pushed to a
escalations, and so on. From an organizational viewpoint, a separate “safety” team, perhaps questioning the value of
bit of a risky tower may be an acceptable steady state. These each block can help. The combination of blocks to make a
questions are separate from those of maturity. successful program is as varied as the companies deploying
A key question to consider is how much structure your threat modeling.
organization wants or needs. Your organization may
want more structure because you make a high-assurance
product, or you may need it because you work in a highly Get In Touch
regulated field. Either way, you likely have a framework like
If threat modeling isn’t delivering what you
FMEA or HAZID, possibly embedded in an approach like
hope for, then it’s our hope that this paper will
Safety-II. You may be working in a software startup that
help. If we can help further, please don’t hesitate
applies a YAGNI test to everything. (YAGNI stands for “You
to reach out for a confidential consultation, at
Ain’t Gonna Need It.”).
[email protected].

1122 E Pike St #1299, Seattle WA 98122 9 +1 917 391 2168 ■ [email protected] ■ associates.shostack.org
FAQ
Q 
Does the positioning of the Q 
Is the set of blocks complete? Q Can I buy a set?
blocks matter?

A 
No. A 
No. It’s intended to be A 
Not from us, and not anywhere
representative and evocative. we’re aware of. Sorry, it’s a
metaphor, not a product.

Acknowledgements License
Cédric Lévy-Bencheton, John Benninghoff, Edward This document is copyright ©2020 Shostack &
Bonver, Izar Tarandach and others gave early Associates. The content is licensed under a Creative
helpful feedback. Jenga is an awesome party game Commons Attribution-NonCommercial License
from Pokonobe Associates. Adrian Lane’s blog 4.0 https://creativecommons.org/licenses/by-nc/4.0.
post “Enterprise DevSecOps: Security’s Role In
DevOps” https://securosis.com/blog/15000 inspired
expansion of the building block list.

1122 E Pike St #1299, Seattle WA 98122 10 +1 917 391 2168 ■ [email protected] ■ associates.shostack.org
About Shostack & Associates
shostack & associates is a specialized security consultancy, focused on meeting
the unique needs of each client through a variety of services including threat
modeling, security engineering, and security process analysis. We have proudly
served the very largest companies in technology, finance and other sectors. For
more, please visit https://associates.shostack.org.

Principal Consultant
adam shostack is a leading expert on threat modeling, and consultant,
entrepreneur, technologist, author and game designer. He helped found the
CVE and a variety of startups. During his years at Microsoft, he was Principal
Program Manager for the Trustworthy Computing SDL team, created the Micro-
soft SDL Threat Modeling Tool (v3), the Elevation of Privilege threat modeling
game, and fixed autorun. He has taught threat modeling at a wide range of
commercial, non-profit and government organizations. He’s a member of the
Black Hat Review Board, is the author of Threat Modeling: Designing for Security,
and the co-author of The New School of Information Security.

adam shostack 1122 E Pike St #1299 [email protected]

+1 917 391 2168 Seattle WA 98122 associates.shostack.org