Penetration Testing Management

Maturity Assessment Tool

Introduction

Overview
Many organisations are extremely concerned about potential and actual cyber security attacks, both on their own
organisations and in ones similar to them. Many of these attacks exploit weaknesses in an organisations applications and
underlying infrastructure. To help identify these vulnerabilities effectively - and address them effectively - many
organisations carry out penetration testing. However, establishing and managing a penetration testing programme can be a
very difficult task, even for the most advanced organisations. Each organisation should therefore develop an appropriate
penetration testing programme which will enable them to adopt a systematic, structured approach to undertaking
penetration testing.

Your penetration testing programme should consist of appropriately skilled people guided by well-designed, repeatable
programmes and effective use of relevant technologies that will enable you to conduct thorough penetration tests,
successfully identifying and addressing vulnerabilities - and to prevent new ones from occurring.

Many organisations do not know how effective their penetration testing programme is in practice. One of the best ways to
help determine the effectiveness of your programme is to measure the level of maturity of your penetration testing
programme in terms of:

• People, process, technology and information

• Requirements, testing and follow up

This assessment tool (which does not use macros) provides a mechanism for carrying out an assessment of the level of
maturity an organisation has for their penetration testing programme at an intermediate, fairly detailed, level. It can be
used to assess the effectiveness of your penetration testing programme.

Note: There two other Penetration Testing Maturity Assessment Tools available, which are the:

• Summary assessment tool (no macros), which allows an assessment to be made to determine the level of maturity of your penetration testin

• Consolidated assessment tool (requires macros to be enabled), which allows more sophisticated assessments to be made to determine the le

Penetration testing process

The assessment tool has been developed in conjunction with representatives from a broad range of organisations, including
industry bodies, consumer organisations, the UK government and suppliers of expert technical security services. It provides
you with an assessment against a maturity model that is based on the 15 steps within the 3 phase penetration testing
management programme presented in the CREST Penetration Testing Management Guide, as shown in the diagram below.
Instructions on how the tool works and how it can be used can be found on the Guidelines worksheet.

Supplier selection process

Instructions on how the tool works and how it can be used can be found on the Guidelines worksheet.

Note: The penetration testing maturity assessment tool is one of a series of assessment tools developed by CREST, which include high level and detailed C

Acknowledgements
CREST would like to extend its special thanks to those CREST member organisations and third parties who took part in
interviews, participated in the workshop and completed questionnaires.

Warning
This Guide has been produced with care and to the best of our ability. However, CREST accepts no responsibility for any
problems or incidents arising from its use.

Credits
This tool has been developed for CREST by

© CREST 2016
 Penetration Testing Management
Maturity Assessment Tool

Guidelines

Maturity model
To carry out penetration testing effectively you will need to build an appropriate penetration testing programme the
maturity of which can be assessed against an appropriate maturity model by using this assessment tool.

The maturity model used in this tool is based on a traditional, proven model shown below. This model can be used to
determine the level of maturity of your penetration testing programme at an intermediate, fairly detailed level, ranging
from 1 (least effective) to 5 (most effective).

Different types of organisation will require different levels of maturity for their penetration testing programme. For
example, a small company operating in the retail business will not have the same requirement – or ability – to carry out
penetration tests in the same way as a major corporate organisation in the finance sector – or a government department.

Consequently, the level of maturity your organisation has in penetration testing should be reviewed in context and
compared to your actual requirements for such a capability. The maturity of your organisation can then be compared with
other similar organisation to help determine if the level of maturity is appropriate.

Note: The maturity of the penetration testing programme can play a significant role in determining the level of third-party
involvement required to conduct independent penetration testing. Organisations with a mature penetration testing
programme may manage most of their operations in-house, while those who are less mature may depend entirely on third
parties.
How to use the tool
This tool allows an assessment to be made to determine the level of maturity of an organisations’ penetration testing
programme at a high level. It is based on a simple selection of the level of maturity for each of the 15 steps.

A weighting factor can be set to give the results for particular steps more importance than others. The selected levels of
maturity are then displayed graphically for each of the three phases and overall. Calculations are based on a carefully
designed algorithm that takes account of both the level of maturity selected for each step and the step's given weighting.

Step 1 - Complete the details for the environment being assessed in the Profile and Scope worksheet using the text boxes
and drop-down lists provided. The name entered for Target of Assessment will automatically appear on the Results
worksheets.

Step 2 - On the Targets Worksheet, select the target level required for this assessment by pressing the radar button next to
any of the six options available, which are:

Introductory - which sets a target of 2 out of 5 across the board

Standard - which sets a target of 2 out of 5 across the board
Important - which sets a target of 3 out of 5 across the board
Very important - which sets a target of 4 out of 5 across the board
Critical - which sets a target of 5 out of 5 across the board
Custom - which allows you to overwrite any of the individual settings in the Custom column on the far right.

These target ratings will show up in all the Results worksheets allowing you to compare actual performance against targets.

Step 3 - A standard set of default weightings have been set by the project Team at CREST and should typically be used to
enable effective and consistent comparisons to be made. However, on the Weightings worksheet, you can overwrite the
weighting values for any particular question. If you are configuring the tool for a respondent and do not want these
weightings to be changed, use the Lock weighting button if appropriate, and then hide the Weightings worksheet by right-
clicking on the relevant tab at the bottom of this spreadsheet and choosing Hide.

Step 4 - Carry out the assessment by selecting the appropriate level of maturity within the assessed environment for each
step using the drop-down lists on the 3 Assessment worksheets, together with any supporting evidence. Any additional
comments can be entered in the Comments column.

Step 5 - Review a summary of the results using the Aggregated Results worksheet to gain a high level picture of the overall
level of maturity for the environment assessed.
 Penetration Testing Management
Maturity Assessment Tool

Organisation / business unit

All fields marked * MUST be completed

A1 Name of organisation *

A2 Name of internal penetration testing coordinator *

A3 Role or position *

A4 Business unit (or equivalent) *

A5 Sector *

A6 Size of business *

A7 Type of business *

A8 Date of assessment *
 Target
level
configura
tion
Penetration testing process Target maturity (1 to 5) Choose a set of targets by
clicking on the radio buttons Very
Stage A - Preparation below. Introductory Standard Important
important
Critical Custom

Step 1 - Maintain a technical security assurance framework 4 Introdu 2 2 3 4 5 1

Step 2 - Establish a penetration testing governance structure 4 ctory

Standa 2 2 3 4 5 1

Step 3 - Evaluate drivers for conducting penetration tests 4 rd

Import 2 2 3 4 5 1

Step 4 - Identify target environments 4

Very
ant 2 2 3 4 5 1
● import
Step 5 - Define the purpose of the penetration tests 4 Critic
ant 2 2 3 4 5 1

Step 6 - Produce requirements specifications 4 al

Custo 2 2 3 4 5 1

Step 7 - Select suitable suppliers 4 m 2 2 3 4 5 1

Stage B - Testing

Step 1 - Agree testing style and type 4 2 2 3 4 5 1

Step 2 - Identify testing constraints 4 2 2 3 4 5 1

Step 3 - Produce scope statements 4 2 2 3 4 5 1

Step 4 - Establish a management assurance framework 4 2 2 3 4 5 1

Step 5 - Implement management control processes 4 2 2 3 4 5 1

Step 6 - Use an effective testing methodology 4 2 2 3 4 5 1

Step 7 - Conduct sufficient research and planning 4 2 2 3 4 5 1

Step 8 - Identify and exploit vulnerabilities 4 2 2 3 4 5 1

Step 9 - Report key findings 4 2 2 3 4 5 1

Stage C - Follow up

Step 1 - Remediate weaknesses 4 2 2 3 4 5 1

Step 2 - Address root causes of weaknesses 4 2 2 3 4 5 1

Step 3 - Initiate improvement programme 4 2 2 3 4 5 1

Step 4 - Evaluate penetration testing effectiveness 4 2 2 3 4 5 1

Step 5 - Build on lessons learned 4 2 2 3 4 5 1

Step 6 - Create and monitor action plans 4 2 2 3 4 5 1
 Level 1 (%) Level 2 (%) Level 3 (%) Level 4 (%) Level 5 (%)
Weighting configuration 0-7 9-30 31-70 71-92 93-100
Level 1 Level 2 Level 3 Level 4 Level 5

Weighting
Stage A Preparation
Step 1 Maintain a technical security assurance framework
A.1.01 Have you identified and recorded all main internal systems that support your
organisation?
Documentation about these systems would typically include: their level of
criticality to the business; the sensitivity of any information they handle; any
key dependencies; network diagrams, data flow and trust boundaries; details
about important third party suppliers; IT infrastructure; and points of
contact, roles and responsibilities.

A.1.02 Do you apply different levels of security assurance for different systems based on
their criticality or the sensitivity of the information they handle?
A.1.03 Have you identified and categorised all main third party systems, processes and
functions that support your organisation?
A.1.04 Do you maintain an underlying technical security assurance framework?

A technical security assurance framework would typically include: multiple

environments for testing; a security architecture; an ongoing security
monitoring services (e.g. in a SOC); an adequate range of technical security
services; a balanced selection of preventative, detective and reactive security
controls; and a road map or similar to provide a short, medium and long term
outlook for security posture.

A.1.05 Does your technical security assurance framework include testing: incident
response processes; backups, to ensure that critical information and systems can
be restored within critical timescales; incident response processes; and disaster
recovery / fail-over processes?

A.1.06 Is your technical security assurance framework supported by sufficient budget,

skilled resources, processes, tools and technology; backed up by adequate
management support and an IT or Cyber security risk management programme?
 An IT or Cyber security risk management programme would typically include:
a documented risk management architecture and framework; a risk
management strategy (including risk appetite); details of relevant legal,
regulatory and contractual compliance requirements; a list of all main
threats, a risk register showing exposure of key assets; and a method of
assessing the effectiveness of technical security arrangements.

Step 2 Establish a penetration testing governance structure

A.2.01 Have you established a suitable governance structure to oversee and coordinate a
regular penetration testing programme?
An effective governance structure for penetration testing would typically
cover all main systems enterprise-wide, while focusing on the most critical,
allowing for the protection of any sensitive information.

A.2.02 Have you established a joint management and technical team to agree the
programme and scope of regular penetration testing?
An effective management and technical team would typically have direct
access to senior management to raise significant concerns, supported by the
ability and authority to contribute to a wider security improvement,
providing adequate control over the penetration testing programme.

A.2.03 Does your penetration testing programme include an approved: set of penetration
testing processes and methodologies that apply enterprise-wide, supplier selection
criteria, a penetration testing assurance management framework and a range of
follow up activities to ensure that remediation activities are carried out in an
effective manner, reducing the risk of vulnerabilities being exploited in the future?

A.2.04 Is your penetration testing programme reviewed and approved by appropriate

business and IT management, supported by stated objectives and timelines, and
integrated in to your underlying technical security assurance framework?

A.2.05 Does your penetration testing programme align with a wider security review
framework, technical security infrastructure and system development processes
(particularly for Web applications)?

A.2.06 Do you have a change management process that enables the secure introduction
of or changes to: business initiatives, business processes, web applications and IT
infrastructure; legal and regulatory requirements; your threat landscape, security
governance approach and security controls framework?

A.2.07 To support your penetration testing programme, do you maintain key performance
indicators for the results of the penetration tests, subscribe to information sharing
platforms or services and use them to feed into the penetration testing
programme?
A.2.08 Are a series of actions taken to provide assurance about the suitability and
effectiveness of your penetration testing programme?
Appropriate assurance actions would typically include traceability and
monitoring of the programme, a continuous improvement process, and
independent audits (or similar).

Step 3 Evaluate drivers for conducting penetration tests

A.3.01 Have you identified drivers for carrying out penetration tests as part of a technical
assurance programme?
A.3.02 Are your drivers for carrying out penetration tests based on an evaluation of
relevant criteria?
Criteria for determining the drivers for penetration testing should include any
growing requirement for compliance, the impact of serious cyber security
attacks, any outsourcing services used, the introduction of new systems and
services, significant changes to IT or the business and changes in the type or
level of perceived threat.

A.3.03 Do your drivers for carrying out penetration tests take account of how a
penetration test fits into your organisation’s overall security arrangements; the
nature and direction of your business – and your risk appetite?

A.3.04 Are your drivers for carrying out penetration tests informed by findings from risk
assessments, audits or reviews; analysis of security incidents; and lessons learnt
from any previous penetration tests?

A.3.05 Have you placed your penetration tests within a wider context of security
assessment and strategy to help contextualise the findings and recommendations?

A.3.06 Do your drivers for penetration testing help to: support the adoption of a strategic
view of security management; ensure that major system vulnerabilities are
identified and addressed; and reduce the risk of discovering that the same
problems still exits the next time a penetration test is carried out?

Step 4 Identify target environments

A.4.01 Have you identified target environments that need to be subject to penetration
testing, such as critical web applications and important IT infrastructure?
A.4.02 Does your identification of the target environment take into account business
processes; web applications; key parts of IT infrastructure and specialised
equipment (e.g. mobile devices and process control systems)?

A.4.03 Does your identification of the target environment consider system criticality;
compliance requirements; major business or it services; critical systems under
development; and outsourced services (e.g. cloud computing)?
A.4.04 Does your identification of the target environment include a risk assessment of
your organisation’s critical information and systems – and ensure that the testing
planned will focus on the assets which pose the highest risk to your organisation?

A.4.05 Does your identification of the target environment take into account significant
changes to critical business processes, business applications, IT infrastructure and
business environments (e.g. in particular business units or jurisdictions)?

A.4.06 Have penetration testing requirements been built into relevant stages of systems
development lifecycles (SDLC) in use by your organisation?
Consideration should be given to conducting penetration tests during the
testing stage; implementation stage; and during live operation.
A.4.07 Have you gained permission to test important systems / environments controlled
by third parties?
If you are not permitted to test important systems / environments controlled
by third parties, you should gain assurances that appropriate penetration
tests are regularly carried out; tests are conducted by suitably qualified staff
working for a certified organisation; and recommendations from the tests are
acted upon.

Step 5 Define the purpose of the penetration tests

A.5.01 Do you define the purpose of penetration tests, and assess how these tests can
help your organisation?
A well-defined penetration tests should help your organisation to identify
weaknesses in your security controls; reduce the frequency and impact of
security incidents; comply with legal and regulatory requirements; provide
assurance to third parties that business applications can be trusted and that
customer data is adequately protected; and limit liabilities if things go wrong,
or if there is a court case (i.e. take ‘reasonable’ precautions).

A.5.02 Do you determine what penetration testing will help you to achieve (i.e. the
benefits)?
Benefits of penetration tests can include IT cost reductions; technical and
business improvements; as well as greater awareness of security risks and
controls.

A.5.03 Do you consider the scope limitations of penetration testing?

When evaluating the scoping limitations of penetration testing you should

take into account that a test covers just the target environment that has
been selected; is only a snapshot of a system at a point in time; and plays
only a small part in an organisation’s defence system.

A.5.04 Do you consider the technical limitations of penetration testing?

 When evaluating the testing limitations of penetration testing you should
take into account that a test focuses on the exposures in technical
infrastructure; can be limited by legal or commercial considerations (limiting
the breadth or depth of a test); and may not uncover all security weaknesses.

A.5.05 Do you evaluate the potential difficulties involved with scoping penetration tests?

The potential scoping difficulties involved with carrying out penetration

testing can include: determining the depth and breadth of coverage of the
test; identifying the style, type, targets and frequency of tests; managing
risks associated with potential system failure and exposure of sensitive data;
and remediating system vulnerabilities effectively.

A.5.06 Do you evaluate the potential difficulties involved with resourcing penetration
tests?
The potential resourcing difficulties in carrying out penetration testing can
include: understanding the costs of external services (and in determining the
true overall cost of testing); and finding a suitable penetration testing expert
when required (e.g. at short notice).

Step 6 Produce requirements specifications

A.6.01 Do you define formal requirements for penetration testing carried out in your
organisation?
A.6.02 Do your requirements for penetration testing include consideration of any impact
on important business applications, key systems and networks (IT infrastructure)
and confidential data?

A.6.03 Do your requirements for penetration testing specify that testers must validate
that the test will be legal and that it will not compromise data protection
requirements?

A.6.04 Are your requirements for a penetration test formally recorded in a requirements
specification, formulated by competent technical experts, reviewed and signed-off,
monitored to ensure they are met and then reviewed / revised on a regular basis?

Step 7 Select suitable suppliers

A.7.01 Do you appoint suitable third party suppliers to undertake independent
penetration testing, based on defined requirements?
Effective requirements for penetration testing suppliers are typically based on
a cost / benefit analysis, driven by clear objectives, recorded in a
requirements specification and integrated into an organisation’s
procurement process.

A.7.02 Do you evaluate the benefits of using external suppliers?

 When evaluating the benefits of using external suppliers, you should consider
their ability to help you: deploy a structured process and plan, developed by
experts; increase the scope and frequency of tests; conduct short term
engagements, eliminating the need to employ your own specialised (and
often expensive) staff; and take advantage of automation (e.g. by using
penetration testing workflows and importing vulnerability management
reports).

A.7.03 Do you define selection criteria to help you choose suitable suppliers?

Effective supplier selection criteria typically specify that potential suppliers

should be able to: provide a reliable, effective and proven penetration testing
service; meet compliance standards; protect your information and systems
both during and after testing; perform rigorous and effective penetration
tests; adhere to a proven testing methodology; carry out a full range of
testing, discover all major vulnerabilities, identifying associated ‘root causes’;
produce insightful, practical and easy to read reports; provide on-going
advice on how to manage systems effectively over time as part of a trusted
relationship.

A.7.04 Does your selection criteria consider if potential suppliers can provide: solid
reputation, history and ethics; high quality, value-for-money services; research and
development capability; highly competent, technical testers; and security and risk
management, supported by a strong professional accreditation and complaint
process?

A.7.05 Do you ensure that your chosen suppliers are able to effectively meet – or exceed -
your supplier selection criteria and provide tangible value for money?
A.7.06 Do you validate the ability of potential suppliers to meet your specific
requirements (not just one who can offer a variety of often impressive products
and services, some of which may not necessarily be relevant)?

A.7.07 Do you go through a formal, approved appointment process for selected

penetration testing suppliers?

Stage B Testing
Step 1 Agree testing style and type
B.1.01 Do you identify what style of penetration testing is required?

B.1.02 Does your identification of testing style evaluate the need for ‘Black box’, ‘Grey
box’ and / or ‘White box’ testing?
B.1.03 Does your identification of testing style consider the use of an ‘external’
penetration test (the most common type of test), which is aimed at IT systems
from ‘outside the building’?
B.1.04 Does your identification of testing types consider the use of an internal security
test; end-to-end testing (i.e. for people, through data, devices, applications and
infrastructure); emerging technologies (e.g. mobile applications); and social
engineering?

B.1.05 Does your identification of testing types consider Web application testing,
Infrastructure testing and Specialised penetration testing, such as for mobile, client
server or cloud-based applications

B.1.06 Is your test environment as similar to the live environment as possible?

Step 2 Identify testing constraints

B.2.01 Do you identify any testing constraints associated with the planned penetration
testing?
B.2.02 When identifying testing constraints, do you allow for aspects of the business that
cannot be tested due to operational limitations, considering that attackers often
do whatever it takes to penetrate an organisation or system (If they are not able to
penetrate a particular system, they may simply try another route.)?

Methods of dealing with operational testing constraints can include

simulating live tests as closely as possible and conducting tests outside of
normal hours (and locations).

B.2.03 When identifying testing constraints, do you allow for testing having to be
conducted within the confines of the law (considering that attackers often do
whatever it takes to penetrate an organisation or system)?

Methods of dealing with legal testing constraints can include tailoring the
way tests are structured and run to simulate most forms of attack) and
taking back-ups of critical systems and files before testing.

B.2.04 When identifying testing constraints, do you allow for testers being limited to the
scope of the testing and a finite time to conduct tests, considering that attackers
will utilise the weakest point of security in any part of connected systems or
networks to mount an attack, regardless of ownership, location or jurisdiction –
and will often have unlimited time to mount a concerted attack against a system if
they have the motivation, capability and resources to do so?

Methods of dealing with scope-related testing constraints can include placing

perimeter controls within the scope of the test and applying more rigorous
testing to applications that are accessible from outside traditional
organisational boundaries.

B.2.05 When identifying testing constraints, do you allow for testers having limited time
to conduct tests, considering that attackers have unlimited time to mount a
concerted attack against a system if they have the motivation, capability and
resources to do so?
 Methods of dealing with time constraints can include Investing more time in
testing critical systems; providing testers with as much background
information as possible; and conducting penetration testing on a regular
basis, rather than as a one-off exercise.

B.2.06 When identifying testing constraints, do you allow for the likelihood that most
penetration testing will not find all vulnerabilities of a given environment (the law
of diminishing returns often applies in that the most obvious vulnerabilities will be
discovered first, with further time yielding more and more obscure issues)?

Methods of dealing with this type of testing constraint can include adopting
a ‘risk to cost balance’ when performing tests and doing more than simply
fixing vulnerabilities uncovered during testing as this could leave a number of
other vulnerabilities present for an attacker to find.

B.2.07 Have you identified technical issues that can affect the scope of the test or the
security countermeasures in place to detect and deter attacks?
Methods of dealing with technical testing constraints can include
Implementing policy exceptions; allowing for vulnerabilities that will not be
discovered if the testing is undertaken from outside your network; adopting a
practical scope that will meet your requirements; and ensuring that the test
simulation comes very close to replicating a real malicious attack.

B.2.08 Have you determined how you will make sure that all parties adhere to testing
constraints?

Step 3 Produce scope statements

B.3.01 Do you formally define the scope of penetration tests prior to tests commencing?

B.3.02 Is the scope of penetration tests recorded in a formal document, such as a scope
statement, that is signed-off by all relevant parties?
Relevant parties (i.e. named individuals or groups) required to sign-off the
scope statement should include authorised and suitably qualified individuals
from all relevant parties; plus relevant, qualified individuals dependent on
the value of the system being tested (or similar).

B.3.03 Does your scope statement include a definition of the target environment?

The definition of the target environment should include: which systems are in
and out of scope; the testing approach being adopted (e.g. black, white or
grey box); types of test that are prohibited (e.g. ‘denial of service’ type
testing); where the testing team will need to be in order to conduct the
testing (e.g. on the customer’s site or at the test service provider’s premises);
and approvals required for various elements of the testing to go ahead.
B.3.04 Does your scope statement specify resourcing requirements?

Resourcing requirements should specify who will be leading the testing

engagement, the names of testers that will be used for the testing
engagement (with details about their roles, skills, experience, qualifications
and backgrounds) and the number of days required (including the days on
which testing will take place) – and require a disclaimer stating that they are
legally authorised to carry out specified activity on your property and
systems.

B.3.05 Does your scope statement define liabilities?

The definition of liabilities in the scope statement should specify the steps
required by both parties should problems (e.g. slippage) arise and the details
of liability (indemnity) insurance to be held by the testing service provider.

B.3.06 Does your scope statement include follow-up activities?

Follow-up activities should include presentation of key findings and

recommendations to senior management and any re-testing needed once
mitigations have been made for the discovered vulnerabilities’ required by
both parties should problems (e.g. slippage) arise.

B.3.07 Do you formally define reporting requirements for your penetration testing prior
to tests commencing?
B.3.08 Do your reporting requirements specify the format and type of content to be used
in the test report (template often used; when the test report will be delivered (not
later than a few days after completion of the test); and how the test report will be
delivered (electronic and / or physical)?

Step 4 Establish a management assurance framework

B.4.01 Are you aware that responsibility for the actual systems and data during
penetration testing – and any assurance about them - rests with your
organisation?

B.4.02 Have you created a documented management assurance framework to help

manage all aspects of penetration tests?
B.4.03 Does your management assurance framework provide assurance to stakeholders
that the objectives of penetration tests are achieved (i.e. business requirements
are met)?

The management assurance framework should provide assurance to

stakeholders that contracts with service providers are defined, agreed,
signed off and monitored and risks to your organisation (e.g. degradation or
loss of services; disclosure of sensitive information) are kept to a minimum.
B.4.04 Does your management assurance framework provide assurance to stakeholders
that changes to the scope of tests – and any problems arising - are well managed?

The management assurance framework should provide assurance to

stakeholders that any changes to the scope of penetration tests (e.g.
additional testing requested, such as to include wireless or device testing) or
to organisational controls (e.g. to address a critical weakness uncovered
during testing) are managed quickly and efficiently; and that any problems
(or complaints) arising during tests (e.g. due to resources not being made
available, tests not working as planned or an ethical breach) are
satisfactorily resolved.

B.4.05 Have you established an assurance process to ensure that the penetration testing
process meets requirements?
B.4.06 Does your assurance process define control processes over all important
management aspects of testing, including test administration; test execution, and
data security?

Test administration typically includes scope of tests, legal constraints,

disclosure and reporting; test execution typically includes testing approach,
separation of systems and duties, tool heritage, traceability and repeatability
of tests; whilst data security typically includes secure storage, transmission,
processing and destruction of critical or sensitive information provided or
accessed during the test.

B.4.07 Is the scope of your penetration tests documented in an agreement, defined in a

legally binding contact and signed off by all relevant parties before testing starts
Penetration testing contracts should specify explicit exclusions (e.g. systems
that are out of scope); any technical and operational constraints; roles and
responsibilities for all parties’ concerned; and specific legal, regulatory and
operational requirements (e.g., timings and checkpoints; a problem
escalation process and post-test corrective action strategy).

Step 5 Implement management control processes

B.5.01 Is your organisation aware that performing any sort of penetration test carries
with it some risk to the target system and the business information associated with
it (e.g. degradation or loss of services; disclosure of sensitive information)?

B.5.02 Have you developed methods of keeping risks to your organisation to a minimum?

You can help to reduce risk associated with penetration testing by carrying
out planning in advance; having a clear definition of scope; using predefined
escalation procedures; supported by qualified testing individuals and certified
organisations.
B.5.03 When conducting penetration tests, do you ensure that those individuals
responsible for the running of the target systems have full knowledge of the tests
to help protect against unexpected business consequences, such an inadvertent
trigger of internal controls; and are aware of – and adhere to - any escalation
procedures?

B.5.04 Are individuals responsible for the running of the target systems available, as
required, during the test period to help: ensure that testing takes place as agreed;
keep risks within acceptable boundaries; deal with any problems arising; and
manage issues that have been escalated?

B.5.05 Is your penetration testing supported by a change management process?

An effective change management process should cover changes to the scope

of the penetration test, organisational controls and the individuals on the
testing team; as well as ensuring that all parties involved adhere to the
process and that changes to penetration testing are made quickly and
efficiently.

B.5.06 Is your penetration testing supported by an effective problem resolution process?

An effective problem resolution process should cover tests not working as

planned and resources not being made available; as well as problems caused
as a result of the penetration testing, which can include: interruptions to or
B.5.07 Are problems arising
degradation of during penetration
live systems; testing resolved
unauthorised disclosureinofanconfidential
effective, timely
manner in accordance
information; and with your problem
compromise of themanagement process? (e.g. affecting
integrity of information
the accuracy or timeliness of information).
Step 6 Use an effective testing methodology
The problem resolution process should also include breaches of: contract;
B.6.01 Whenspecifications in the scopetests
conducting penetration statement; andaasystematic,
do you use relevant code of conduct.
structured testing
methodology?
B.6.02 Is your penetration testing methodology based on proven approaches designed by
authoritative publicly available sources?
Authoritative publicly available sources include the Open Source Security
Testing Methodology Manual (OSSTM) or the penetration testing in SP800-
115.[3] and the Open Web Application Security Project (OWASP).

B.6.03 Does your penetration testing methodology detail specific evaluation or testing
criteria and adhere to a standard common language and scope for performing
penetration testing (i.e. security evaluations)?

Specific evaluation and testing criteria include the Information Systems

Security Assessment Framework (ISSAF) and a standard common language
and scope for performing penetration testing is defined in the Penetration
Testing Execution Standard (PTES).

B.6.04 Does your penetration testing methodology specify a required approach (or
approaches) for carrying out all stages of a comprehensive end-to-end penetration
test?
 An effective penetration testing methodology should specify a required
approach (or approaches) for carrying out planning; conducting research;
identifying vulnerabilities; exploiting weaknesses; reporting findings; and
remediating issues.

B.6.05 Do your service providers demonstrate compliance to ‘standard’ methodologies, if

required, and develop or augment testing methodologies that each scenario
demands?

Step 7 Conduct sufficient research and planning

B.7.01 Are detailed, agreed test plans produced to provide guidelines for the penetration
testing to be undertaken?
Test plans should be produced by your testing service provider and agreed
with your organisation prior to any testing commencing.
B.7.02 Do test plans specify what will actually be done during the test itself and help to
assure the process for a proper security test without creating misunderstandings,
misconceptions, or false expectations?

B.7.03 Do penetration tests include carrying out sufficient research to imitate the
research activities that a potential attacker could undertake to find out as much
about the target environment and how it works as possible?

B.7.04 Does the research undertaken include gathering, collating and analysing all
relevant information about the target?
Relevant information should include data: obtained from public sources of
information, such as the Internet; through information sharing networks (e.g.
CERTs); and via authorised social engineering sources.

B.7.05 Is information about the target based on threat intelligence?

B.7.06 Does the research undertaken include carrying out reconnaissance?

Reconnaissance should include collating and analysing information about the

target obtaining positive confirmation of information about the target (e.g.
to confirm that system configuration and security controls are as expected).

B.7.07 Does the research undertaken include network enumeration / scanning?

Network enumeration / scanning should include identifying the potential

points of access being offered by a target by scanning for open services on
targets and establishing the existence of possible user identification
credentials.

B.7.08 Does the research undertaken include discovery and assessment?

Step 8 Identify and exploit vulnerabilities

B.8.01 Do penetration tests identify a range of potential vulnerabilities in target systems?

B.8.02 Does vulnerability identification include testers examining: attack avenues, vectors
and threat agents (e.g. using attack trees); results from threat analysis; technical
system / network / application vulnerabilities; and control weaknesses

B.8.03 Do tests include reviewing vulnerabilities identified by third parties, such as the
‘OWASP Top Ten’, which presents a list of common security vulnerabilities found in
web applications (i.e. injection attacks, cross-site scripting and failure to restrict
URL access)?

B.8.04 Do tests include identifying the cause of any vulnerabilities discovered, for
example resulting from a lack of understanding of IT security issues (e.g. by web
developers and users of mobile devices)?

B.8.05 Do penetration testers try to exploit the vulnerabilities identified and actually
penetrate the target system?
B.8.06 Do testers use a range of techniques (e.g. exploitation frameworks, stand-alone
exploits, and other tactics) to try and take advantage of specific weaknesses?
Exploitation techniques include: specific Exploit techniques (e.g. for web
applications); Escalation techniques, gaining further access within a target,
once an initial level of access has been obtained; advancement techniques,
attempting to move on from the compromised target to find other vulnerable
systems; and analysis techniques, verifying the raw data to ensure that the
test has been thorough and comprehensive.

Step 9 Report key findings

B.9.01 Are findings identified during the penetration test reported to your organisation in
both technical terms that can be acted upon and non-technical, business context,
so that the justifications for the corrective actions are understood; as well as in a
formal report?

B.9.02 Do penetration testing reports describe the vulnerabilities found?

Penetration testing reports should describe the vulnerabilities found by

including: test narrative – describing the process that the tester used to
achieve particular results; test evidence – results of automated testing tools
and screen shots of successful exploits; and details about the associated
technical risks (and how to address them).

B.9.03 Are penetration testing reports used to present remediation activities undertaken
and the root causes of issues identified?
B.9.04 Are penetration testing reports disseminated to relevant staff - and acted upon?

B.9.05 Does test reporting include a comprehensive presentation from your service
provider about the key findings identified?
 The presentation about test findings identified should provide details about:
how testers found the vulnerabilities; what could be the outcome of each
vulnerability; the level of risk to the business for each vulnerability; and
advice on how to remediate each vulnerability.

Stage C Follow up
Step 1 Remediate weaknesses
C.1.01 Do follow-up activities include remediating weaknesses identified in penetration
testing?
C.1.02 Are weaknesses remediated in line with a comprehensive, approved remediation
process?
An effective remediation process should include addressing all issues;
applying immediate or short terms solutions (e.g. patching systems, closing
ports and preventing traffic from particular web sites or IP addresses),
replicating results of penetration tests, determining which weaknesses to
address first (e.g. based on risk ratings for critical assets), and reporting
weaknesses to relevant third party organisations.

Step 2 Address root causes of weaknesses

C.2.01 Do follow-up activities include analysing and addressing the root causes of
weaknesses identified in penetration testing?
C.2.02 Does root cause analysis include the full range of required actions?

Root cause analysis should include: identifying the real root causes of
exposures; evaluating potential business impact; identifying more endemic or
fundamental root causes; qualified, experienced security professionals to
help define corrective action strategy and plans.

Step 3 Initiate improvement programme

C.3.01 On completion of penetration tests, is an improvement programme initiated?

C.3.02 Is the improvement programme carried out in a structured / systematic manner?

C.3.03 Does your improvement programme include all key elements?

The improvement programme should address root causes of weakness;

evaluate penetration testing effectiveness; identify lessons learned; apply
good practice enterprise-wide; create and monitor action plans; and agree
approaches for future testing.

Step 4 Evaluate penetration testing effectiveness

C.4.01 Is the effectiveness of your penetration testing evaluated?

C.4.02 Does evaluation of test effectiveness cover the full range of required actions?

Evaluation of the effectiveness of penetration testing should include:

determining if objectives were met; assessing if sufficient weaknesses were
identified; reviewing exploitations undertaken; comparing test results to
external benchmarks; and determining if value for money was obtained from
your service provider.

Step 5 Build on lessons learned

C.5.01 Does your penetration testing approach include identifying lessons learned,
disseminating them to relevant stakeholders and acting on them?
C.5.02 Are lessons learned used to help in planning future tests, and provide feedback to
service providers to help them improve processes?
C.5.03 When addressing the weaknesses identified in an environment, are good practices
identified (including fixes) and then applied to a wide range of other
environments?

C.5.04 Are good practices rolled out by: performing trend analysis across multiple
systems; applying lessons learnt during a penetration test of one application to
similar applications; and fixing root causes endemically?

Step 6 Create and monitor action plans

C.6.01 Are action plans created to help act upon follow-up activities undertaken?

C.6.02 Are action plans formally documented, formulated by competent technical

experts, reviewed by business management and signed-off by senior
management?

C.6.03 Do action plans outline all the relevant actions to be taken to prevent
vulnerabilities identified through testing from recurring and help improve the
overall information security programme

C.6.04 Do action plans include a brief description of each action, including their priority
and category; individuals responsible and accountable for each action; and target
dates for completion

C.6.05 Are action plans implemented effectively and on a timely basis?

C.6.06 Are action plans monitored on a regular basis to: ensure progress is being made;
highlight any delays or difficulties being experienced; and reassess the level of risk?
 Aggregated maturity levels A.1 - Maintain a technical security assurance framework
C.6 - Create andA.2monitor
- Establish
action
a penetration
plans testing governance structure

C.5 - Build on lessons learned 5.00 A.3 - Evaluate drivers for conducting pe

Benchmark C.4 - Evaluate penetration testing effectiveness 4.00

A.4 - Identify target environm
Maturity level (1 to 5) Target maturity (1 to 5) Rating
Stage A - Preparation Maturity level: Level 1 3.00
C.3 - Initiate improvement programme A.5 - Define the purpos
Step 1 - Maintain a technical security assurance framework 1 4 3
2.00
Step 2 - Establish a penetration testing governance structure 1 4 3
1.00
Step 3 - Evaluate drivers for conducting penetration tests 1 4 3 C.2 - Address root causes of weaknesses A.6 - Produce requir

Step 4 - Identify target environments 1 4 3 0.00

Step 5 - Define the purpose of the penetration tests 1 4 3 C.1 - Remediate weaknesses A.7 - Select suitable

Step 6 - Produce requirements specifications 1 4 3

Step 7 - Select suitable suppliers 1 4 3 B.9 - Report key findings B.1 - Agree testing style

Stage B - Testing Maturity level: Level 1

Step 1 - Agree testing style and type 1 4 3 B.8 - Identify and exploit vulnerabilities B.2 - Identify testing constrain

Step 2 - Identify testing constraints 1 4 3

B.7 - Conduct sufficient research and planning B.3 - Produce scope statements
Step 3 - Produce scope statements 1 4 3 B.6 - Use an effectiveB.4
testing
- Establish
methodology
a management assurance framework
B.5 - Implement management control processes
Step 4 - Establish a management assurance framework 1 4 3

Step 5 - Implement management control processes 1 4 3

Step 6 - Use an effective testing methodology 1 4 3

Step 7 - Conduct sufficient research and planning 1 4 3

Step 8 - Identify and exploit vulnerabilities 1 4 3

Step 9 - Report key findings 1 4 3

Stage C - Follow up Maturity level: Level 1

Step 1 - Remediate weaknesses 1 4 3

Step 2 - Address root causes of weaknesses 1 4 3

Step 3 - Initiate improvement programme 1 4 3

Step 4 - Evaluate penetration testing effectiveness 1 4 3

Step 5 - Build on lessons learned 1 4 3

Step 6 - Create and monitor action plans 1 4 3

 Maturity model for Stage A - Preparation

Weighted
Response Weighting score Evidence supplied Comments
Step 1 Maintain a technical security assurance framework
A.1.01 Have you identified and recorded all main internal systems that support your
organisation? x2
Documentation about these systems would typically include: their level of
criticality to the business; the sensitivity of any information they handle; any
key dependencies; network diagrams, data flow and trust boundaries; details
about important third party suppliers; IT infrastructure; and points of
contact, roles and responsibilities.

A.1.02 Do you apply different levels of security assurance for different systems based on
their criticality or the sensitivity of the information they handle? x4
A.1.03 Have you identified and categorised all main third party systems, processes and
functions that support your organisation? x3
A.1.04 Do you maintain an underlying technical security assurance framework?
x4
A technical security assurance framework would typically include: multiple
environments for testing; a security architecture; an ongoing security
monitoring services (e.g. in a SOC); an adequate range of technical security
services; a balanced selection of preventative, detective and reactive security
controls; and a road map or similar to provide a short, medium and long term
outlook for security posture.

A.1.05 Does your technical security assurance framework include testing: incident
response processes; backups, to ensure that critical information and systems can
be restored within critical timescales; incident response processes; and disaster x3
recovery / fail-over processes?

A.1.06 Is your technical security assurance framework supported by sufficient budget,

skilled resources, processes, tools and technology; backed up by adequate
management support and an IT or Cyber security risk management programme? x5

An IT or Cyber security risk management programme would typically include:

a documented risk management architecture and framework; a risk
management strategy (including risk appetite); details of relevant legal,
regulatory and contractual compliance requirements; a list of all main
threats, a risk register showing exposure of key assets; and a method of
assessing the effectiveness of technical security arrangements.

Step 2 Establish a penetration testing governance structure

A.2.01 Have you established a suitable governance structure to oversee and coordinate a
regular penetration testing programme? x1
An effective governance structure for penetration testing would typically
cover all main systems enterprise-wide, while focusing on the most critical,
allowing for the protection of any sensitive information.
A.2.02 Have you established a joint management and technical team to agree the
programme and scope of regular penetration testing? x4
An effective management and technical team would typically have direct
access to senior management to raise significant concerns, supported by the
ability and authority to contribute to a wider security improvement,
providing adequate control over the penetration testing programme.

A.2.03 Does your penetration testing programme include an approved: set of penetration
testing processes and methodologies that apply enterprise-wide, supplier selection
criteria, a penetration testing assurance management framework and a range of
follow up activities to ensure that remediation activities are carried out in an x3
effective manner, reducing the risk of vulnerabilities being exploited in the future?

A.2.04 Is your penetration testing programme reviewed and approved by appropriate

business and IT management, supported by stated objectives and timelines, and
integrated in to your underlying technical security assurance framework? x3

A.2.05 Does your penetration testing programme align with a wider security review
framework, technical security infrastructure and system development processes x3
(particularly for Web applications)?

A.2.06 Do you have a change management process that enables the secure introduction
of or changes to: business initiatives, business processes, web applications and IT
infrastructure; legal and regulatory requirements; your threat landscape, security
governance approach and security controls framework? x4

A.2.07 To support your penetration testing programme, do you maintain key performance
indicators for the results of the penetration tests, subscribe to information sharing
platforms or services and use them to feed into the penetration testing x5
programme?

A.2.08 Are a series of actions taken to provide assurance about the suitability and
effectiveness of your penetration testing programme? x5
Appropriate assurance actions would typically include traceability and
monitoring of the programme, a continuous improvement process, and
independent audits (or similar).

Step 3 Evaluate drivers for conducting penetration tests

A.3.01 Have you identified drivers for carrying out penetration tests as part of a technical
assurance programme? x1
A.3.02 Are your drivers for carrying out penetration tests based on an evaluation of
relevant criteria? x3
Criteria for determining the drivers for penetration testing should include any
growing requirement for compliance, the impact of serious cyber security
attacks, any outsourcing services used, the introduction of new systems and
services, significant changes to IT or the business and changes in the type or
level of perceived threat.

A.3.03 Do your drivers for carrying out penetration tests take account of how a
penetration test fits into your organisation’s overall security arrangements; the
nature and direction of your business – and your risk appetite? x4
A.3.04 Are your drivers for carrying out penetration tests informed by findings from risk
assessments, audits or reviews; analysis of security incidents; and lessons learnt x3
from any previous penetration tests?

A.3.05 Have you placed your penetration tests within a wider context of security
assessment and strategy to help contextualise the findings and recommendations? x4

A.3.06 Do your drivers for penetration testing help to: support the adoption of a strategic
view of security management; ensure that major system vulnerabilities are
identified and addressed; and reduce the risk of discovering that the same
problems still exits the next time a penetration test is carried out? x5

Step 4 Identify target environments

A.4.01 Have you identified target environments that need to be subject to penetration
testing, such as critical web applications and important IT infrastructure? x1
A.4.02 Does your identification of the target environment take into account business
processes; web applications; key parts of IT infrastructure and specialised
equipment (e.g. mobile devices and process control systems)? x2

A.4.03 Does your identification of the target environment consider system criticality;
compliance requirements; major business or it services; critical systems under
development; and outsourced services (e.g. cloud computing)? x3

A.4.04 Does your identification of the target environment include a risk assessment of
your organisation’s critical information and systems – and ensure that the testing
planned will focus on the assets which pose the highest risk to your organisation? x4

A.4.05 Does your identification of the target environment take into account significant
changes to critical business processes, business applications, IT infrastructure and
business environments (e.g. in particular business units or jurisdictions)? x4

A.4.06 Have penetration testing requirements been built into relevant stages of systems
development lifecycles (SDLC) in use by your organisation? x4
Consideration should be given to conducting penetration tests during the
testing stage; implementation stage; and during live operation.
A.4.07 Have you gained permission to test important systems / environments controlled
by third parties? x5
If you are not permitted to test important systems / environments controlled
by third parties, you should gain assurances that appropriate penetration
tests are regularly carried out; tests are conducted by suitably qualified staff
working for a certified organisation; and recommendations from the tests are
acted upon.

Step 5 Define the purpose of the penetration tests

A.5.01 Do you define the purpose of penetration tests, and assess how these tests can
help your organisation? x3
 A well-defined penetration tests should help your organisation to identify
weaknesses in your security controls; reduce the frequency and impact of
security incidents; comply with legal and regulatory requirements; provide
assurance to third parties that business applications can be trusted and that
customer data is adequately protected; and limit liabilities if things go wrong,
or if there is a court case (i.e. take ‘reasonable’ precautions).

A.5.02 Do you determine what penetration testing will help you to achieve (i.e. the
benefits)? x5
Benefits of penetration tests can include IT cost reductions; technical and
business improvements; as well as greater awareness of security risks and
controls.

A.5.03 Do you consider the scope limitations of penetration testing?

x3
When evaluating the scoping limitations of penetration testing you should
take into account that a test covers just the target environment that has
been selected; is only a snapshot of a system at a point in time; and plays
only a small part in an organisation’s defence system.

A.5.04 Do you consider the technical limitations of penetration testing?

x3
When evaluating the testing limitations of penetration testing you should
take into account that a test focuses on the exposures in technical
infrastructure; can be limited by legal or commercial considerations (limiting
the breadth or depth of a test); and may not uncover all security weaknesses.

A.5.05 Do you evaluate the potential difficulties involved with scoping penetration tests?
x3
The potential scoping difficulties involved with carrying out penetration
testing can include: determining the depth and breadth of coverage of the
test; identifying the style, type, targets and frequency of tests; managing
risks associated with potential system failure and exposure of sensitive data;
and remediating system vulnerabilities effectively.

A.5.06 Do you evaluate the potential difficulties involved with resourcing penetration
tests? x4
The potential resourcing difficulties in carrying out penetration testing can
include: understanding the costs of external services (and in determining the
true overall cost of testing); and finding a suitable penetration testing expert
when required (e.g. at short notice).

Step 6 Produce requirements specifications

A.6.01 Do you define formal requirements for penetration testing carried out in your
organisation? x1
A.6.02 Do your requirements for penetration testing include consideration of any impact
on important business applications, key systems and networks (IT infrastructure) x3
and confidential data?

A.6.03 Do your requirements for penetration testing specify that testers must validate
that the test will be legal and that it will not compromise data protection x4
requirements?
A.6.04 Are your requirements for a penetration test formally recorded in a requirements
specification, formulated by competent technical experts, reviewed and signed-off,
monitored to ensure they are met and then reviewed / revised on a regular basis? x5

Step 7 Select suitable suppliers

A.7.01 Do you appoint suitable third party suppliers to undertake independent
penetration testing, based on defined requirements? x2
Effective requirements for penetration testing suppliers are typically based on
a cost / benefit analysis, driven by clear objectives, recorded in a
requirements specification and integrated into an organisation’s
procurement process.

A.7.02 Do you evaluate the benefits of using external suppliers?

x3
When evaluating the benefits of using external suppliers, you should consider
their ability to help you: deploy a structured process and plan, developed by
experts; increase the scope and frequency of tests; conduct short term
engagements, eliminating the need to employ your own specialised (and
often expensive) staff; and take advantage of automation (e.g. by using
penetration testing workflows and importing vulnerability management
reports).

A.7.03 Do you define selection criteria to help you choose suitable suppliers?
x4
Effective supplier selection criteria typically specify that potential suppliers
should be able to: provide a reliable, effective and proven penetration testing
service; meet compliance standards; protect your information and systems
both during and after testing; perform rigorous and effective penetration
tests; adhere to a proven testing methodology; carry out a full range of
testing, discover all major vulnerabilities, identifying associated ‘root causes’;
produce insightful, practical and easy to read reports; provide on-going
advice on how to manage systems effectively over time as part of a trusted
relationship.

A.7.04 Does your selection criteria consider if potential suppliers can provide: solid
reputation, history and ethics; high quality, value-for-money services; research and
development capability; highly competent, technical testers; and security and risk
management, supported by a strong professional accreditation and complaint x5
process?

A.7.05 Do you ensure that your chosen suppliers are able to effectively meet – or exceed -
your supplier selection criteria and provide tangible value for money? x3

A.7.06 Do you validate the ability of potential suppliers to meet your specific
requirements (not just one who can offer a variety of often impressive products x4
and services, some of which may not necessarily be relevant)?

A.7.07 Do you go through a formal, approved appointment process for selected

penetration testing suppliers? x3
 Maturity model for Stage B - Testing

Weighted
Response Weighting score Evidence supplied Comments
Step 1 Agree testing style and type
B.1.01 Do you identify what style of penetration testing is required?
x1
B.1.02 Does your identification of testing style evaluate the need for ‘Black box’, ‘Grey
box’ and / or ‘White box’ testing? x2
B.1.03 Does your identification of testing style consider the use of an ‘external’
penetration test (the most common type of test), which is aimed at IT systems x4
from ‘outside the building’?

B.1.04 Does your identification of testing types consider the use of an internal security
test; end-to-end testing (i.e. for people, through data, devices, applications and
infrastructure); emerging technologies (e.g. mobile applications); and social x5
engineering?

B.1.05 Does your identification of testing types consider Web application testing,
Infrastructure testing and Specialised penetration testing, such as for mobile, client x5
server or cloud-based applications

B.1.06 Is your test environment as similar to the live environment as possible?

x5

Step 2 Identify testing constraints

B.2.01 Do you identify any testing constraints associated with the planned penetration
testing? x1
B.2.02 When identifying testing constraints, do you allow for aspects of the business that
cannot be tested due to operational limitations, considering that attackers often
do whatever it takes to penetrate an organisation or system (If they are not able to
penetrate a particular system, they may simply try another route.)? x5

Methods of dealing with operational testing constraints can include

simulating live tests as closely as possible and conducting tests outside of
normal hours (and locations).

B.2.03 When identifying testing constraints, do you allow for testing having to be
conducted within the confines of the law (considering that attackers often do x4
whatever it takes to penetrate an organisation or system)?

Methods of dealing with legal testing constraints can include tailoring the
way tests are structured and run to simulate most forms of attack) and
taking back-ups of critical systems and files before testing.

B.2.04 When identifying testing constraints, do you allow for testers being limited to the
scope of the testing and a finite time to conduct tests, considering that attackers
will utilise the weakest point of security in any part of connected systems or
networks to mount an attack, regardless of ownership, location or jurisdiction –
and will often have unlimited time to mount a concerted attack against a system if x4
they have the motivation, capability and resources to do so?
 Methods of dealing with scope-related testing constraints can include placing
perimeter controls within the scope of the test and applying more rigorous
testing to applications that are accessible from outside traditional
organisational boundaries.

B.2.05 When identifying testing constraints, do you allow for testers having limited time
to conduct tests, considering that attackers have unlimited time to mount a
concerted attack against a system if they have the motivation, capability and x4
resources to do so?

Methods of dealing with time constraints can include Investing more time in
testing critical systems; providing testers with as much background
information as possible; and conducting penetration testing on a regular
basis, rather than as a one-off exercise.

B.2.06 When identifying testing constraints, do you allow for the likelihood that most
penetration testing will not find all vulnerabilities of a given environment (the law
of diminishing returns often applies in that the most obvious vulnerabilities will be
discovered first, with further time yielding more and more obscure issues)? x4

Methods of dealing with this type of testing constraint can include adopting
a ‘risk to cost balance’ when performing tests and doing more than simply
fixing vulnerabilities uncovered during testing as this could leave a number of
other vulnerabilities present for an attacker to find.

B.2.07 Have you identified technical issues that can affect the scope of the test or the
security countermeasures in place to detect and deter attacks? x4
Methods of dealing with technical testing constraints can include
Implementing policy exceptions; allowing for vulnerabilities that will not be
discovered if the testing is undertaken from outside your network; adopting a
practical scope that will meet your requirements; and ensuring that the test
simulation comes very close to replicating a real malicious attack.

B.2.08 Have you determined how you will make sure that all parties adhere to testing
constraints? x5

Step 3 Produce scope statements

B.3.01 Do you formally define the scope of penetration tests prior to tests commencing?
x1
B.3.02 Is the scope of penetration tests recorded in a formal document, such as a scope
statement, that is signed-off by all relevant parties? x2
Relevant parties (i.e. named individuals or groups) required to sign-off the
scope statement should include authorised and suitably qualified individuals
from all relevant parties; plus relevant, qualified individuals dependent on
the value of the system being tested (or similar).

B.3.03 Does your scope statement include a definition of the target environment?
x3
The definition of the target environment should include: which systems are in
and out of scope; the testing approach being adopted (e.g. black, white or
grey box); types of test that are prohibited (e.g. ‘denial of service’ type
testing); where the testing team will need to be in order to conduct the
testing (e.g. on the customer’s site or at the test service provider’s premises);
and approvals required for various elements of the testing to go ahead.
B.3.04 Does your scope statement specify resourcing requirements?
x3
Resourcing requirements should specify who will be leading the testing
engagement, the names of testers that will be used for the testing
engagement (with details about their roles, skills, experience, qualifications
and backgrounds) and the number of days required (including the days on
which testing will take place) – and require a disclaimer stating that they are
legally authorised to carry out specified activity on your property and
systems.

B.3.05 Does your scope statement define liabilities?

x5
The definition of liabilities in the scope statement should specify the steps
required by both parties should problems (e.g. slippage) arise and the details
of liability (indemnity) insurance to be held by the testing service provider.

B.3.06 Does your scope statement include follow-up activities?

x3
Follow-up activities should include presentation of key findings and
recommendations to senior management and any re-testing needed once
mitigations have been made for the discovered vulnerabilities’ required by
both parties should problems (e.g. slippage) arise.

B.3.07 Do you formally define reporting requirements for your penetration testing prior
to tests commencing? x1
B.3.08 Do your reporting requirements specify the format and type of content to be used
in the test report (template often used; when the test report will be delivered (not
later than a few days after completion of the test); and how the test report will be x3
delivered (electronic and / or physical)?

Step 4 Establish a management assurance framework

B.4.01 Are you aware that responsibility for the actual systems and data during
penetration testing – and any assurance about them - rests with your x1
organisation?

B.4.02 Have you created a documented management assurance framework to help

manage all aspects of penetration tests? x2
B.4.03 Does your management assurance framework provide assurance to stakeholders
that the objectives of penetration tests are achieved (i.e. business requirements x5
are met)?

The management assurance framework should provide assurance to

stakeholders that contracts with service providers are defined, agreed,
signed off and monitored and risks to your organisation (e.g. degradation or
loss of services; disclosure of sensitive information) are kept to a minimum.

B.4.04 Does your management assurance framework provide assurance to stakeholders

that changes to the scope of tests – and any problems arising - are well managed? x4
 The management assurance framework should provide assurance to
stakeholders that any changes to the scope of penetration tests (e.g.
additional testing requested, such as to include wireless or device testing) or
to organisational controls (e.g. to address a critical weakness uncovered
during testing) are managed quickly and efficiently; and that any problems
(or complaints) arising during tests (e.g. due to resources not being made
available, tests not working as planned or an ethical breach) are
satisfactorily resolved.

B.4.05 Have you established an assurance process to ensure that the penetration testing
process meets requirements? x1
B.4.06 Does your assurance process define control processes over all important
management aspects of testing, including test administration; test execution, and x5
data security?

Test administration typically includes scope of tests, legal constraints,

disclosure and reporting; test execution typically includes testing approach,
separation of systems and duties, tool heritage, traceability and repeatability
of tests; whilst data security typically includes secure storage, transmission,
processing and destruction of critical or sensitive information provided or
accessed during the test.

B.4.07 Is the scope of your penetration tests documented in an agreement, defined in a

legally binding contact and signed off by all relevant parties before testing starts x1
Penetration testing contracts should specify explicit exclusions (e.g. systems
that are out of scope); any technical and operational constraints; roles and
responsibilities for all parties’ concerned; and specific legal, regulatory and
operational requirements (e.g., timings and checkpoints; a problem
escalation process and post-test corrective action strategy).

Step 5 Implement management control processes

B.5.01 Is your organisation aware that performing any sort of penetration test carries
with it some risk to the target system and the business information associated with
it (e.g. degradation or loss of services; disclosure of sensitive information)? x1

B.5.02 Have you developed methods of keeping risks to your organisation to a minimum?
x4
You can help to reduce risk associated with penetration testing by carrying
out planning in advance; having a clear definition of scope; using predefined
escalation procedures; supported by qualified testing individuals and certified
organisations.

B.5.03 When conducting penetration tests, do you ensure that those individuals
responsible for the running of the target systems have full knowledge of the tests
to help protect against unexpected business consequences, such an inadvertent
trigger of internal controls; and are aware of – and adhere to - any escalation x3
procedures?

B.5.04 Are individuals responsible for the running of the target systems available, as
required, during the test period to help: ensure that testing takes place as agreed;
keep risks within acceptable boundaries; deal with any problems arising; and x3
manage issues that have been escalated?

B.5.05 Is your penetration testing supported by a change management process?

x2
 An effective change management process should cover changes to the scope
of the penetration test, organisational controls and the individuals on the
testing team; as well as ensuring that all parties involved adhere to the
process and that changes to penetration testing are made quickly and
efficiently.

B.5.06 Is your penetration testing supported by an effective problem resolution process?

x3
An effective problem resolution process should cover tests not working as
planned and resources not being made available; as well as problems caused
as a result of the penetration testing, which can include: interruptions to or
degradation of live systems; unauthorised disclosure of confidential
information; and compromise of the integrity of information (e.g. affecting
the accuracy or timeliness of information).

The problem resolution process should also include breaches of: contract;
specifications in the scope statement; and a relevant code of conduct.

B.5.07 Are problems arising during penetration testing resolved in an effective, timely
manner in accordance with your problem management process? x4

Step 6 Use an effective testing methodology

B.6.01 When conducting penetration tests do you use a systematic, structured testing
methodology? x1
B.6.02 Is your penetration testing methodology based on proven approaches designed by
authoritative publicly available sources? x2
Authoritative publicly available sources include the Open Source Security
Testing Methodology Manual (OSSTM) or the penetration testing in SP800-
115.[3] and the Open Web Application Security Project (OWASP).

B.6.03 Does your penetration testing methodology detail specific evaluation or testing
criteria and adhere to a standard common language and scope for performing x3
penetration testing (i.e. security evaluations)?

Specific evaluation and testing criteria include the Information Systems

Security Assessment Framework (ISSAF) and a standard common language
and scope for performing penetration testing is defined in the Penetration
Testing Execution Standard (PTES).
 Maturity model for Stage C - Follow up

Weighted
Response Weighting score Evidence supplied Comments
Step 1 Remediate weaknesses
C.1.01 Do follow-up activities include remediating weaknesses identified in penetration
testing? x2
C.1.02 Are weaknesses remediated in line with a comprehensive, approved remediation
process? x4
An effective remediation process should include addressing all issues;
applying immediate or short terms solutions (e.g. patching systems, closing
ports and preventing traffic from particular web sites or IP addresses),
replicating results of penetration tests, determining which weaknesses to
address first (e.g. based on risk ratings for critical assets), and reporting
weaknesses to relevant third party organisations.

Step 2 Address root causes of weaknesses

C.2.01 Do follow-up activities include analysing and addressing the root causes of
weaknesses identified in penetration testing? x2
C.2.02 Does root cause analysis include the full range of required actions?
x4
Root cause analysis should include: identifying the real root causes of
exposures; evaluating potential business impact; identifying more endemic or
fundamental root causes; qualified, experienced security professionals to
help define corrective action strategy and plans.

Step 3 Initiate improvement programme

C.3.01 On completion of penetration tests, is an improvement programme initiated?
x1
C.3.02 Is the improvement programme carried out in a structured / systematic manner?
x3
C.3.03 Does your improvement programme include all key elements?
x5
The improvement programme should address root causes of weakness;
evaluate penetration testing effectiveness; identify lessons learned; apply
good practice enterprise-wide; create and monitor action plans; and agree
approaches for future testing.

Step 4 Evaluate penetration testing effectiveness

C.4.01 Is the effectiveness of your penetration testing evaluated?
x1
C.4.02 Does evaluation of test effectiveness cover the full range of required actions?
x5
 Evaluation of the effectiveness of penetration testing should include:
determining if objectives were met; assessing if sufficient weaknesses were
identified; reviewing exploitations undertaken; comparing test results to
external benchmarks; and determining if value for money was obtained from
your service provider.

Step 5 Build on lessons learned

C.5.01 Does your penetration testing approach include identifying lessons learned,
disseminating them to relevant stakeholders and acting on them? x1
C.5.02 Are lessons learned used to help in planning future tests, and provide feedback to
service providers to help them improve processes? x5
C.5.03 When addressing the weaknesses identified in an environment, are good practices
identified (including fixes) and then applied to a wide range of other x1
environments?

C.5.04 Are good practices rolled out by: performing trend analysis across multiple
systems; applying lessons learnt during a penetration test of one application to
similar applications; and fixing root causes endemically? x5

Step 6 Create and monitor action plans

C.6.01 Are action plans created to help act upon follow-up activities undertaken?
x1
C.6.02 Are action plans formally documented, formulated by competent technical
experts, reviewed by business management and signed-off by senior x3
management?

C.6.03 Do action plans outline all the relevant actions to be taken to prevent
vulnerabilities identified through testing from recurring and help improve the x3
overall information security programme

C.6.04 Do action plans include a brief description of each action, including their priority
and category; individuals responsible and accountable for each action; and target x3
dates for completion

C.6.05 Are action plans implemented effectively and on a timely basis?

x4
C.6.06 Are action plans monitored on a regular basis to: ensure progress is being made;
highlight any delays or difficulties being experienced; and reassess the level of risk? x5
 Results
Maturity model for Stage A - Preparation

Maturity score Target Evidence supplied

Step 1 Maintain a technical security assurance framework Maturity level: Level 1
A.1.01 Have you identified and recorded all main internal systems that support your
organisation?
Documentation about these systems would typically include: their level of
criticality to the business; the sensitivity of any information they handle; any
key dependencies; network diagrams, data flow and trust boundaries; details
about important third party suppliers; IT infrastructure; and points of
contact, roles and responsibilities.

A.1.02 Do you apply different levels of security assurance for different systems based on
their criticality or the sensitivity of the information they handle?
A.1.03 Have you identified and categorised all main third party systems, processes and
functions that support your organisation?
A.1.04 Do you maintain an underlying technical security assurance framework?

A technical security assurance framework would typically include: multiple

environments for testing; a security architecture; an ongoing security
monitoring services (e.g. in a SOC); an adequate range of technical security
services; a balanced selection of preventative, detective and reactive security
controls; and a road map or similar to provide a short, medium and long term
outlook for security posture.

A.1.05 Does your technical security assurance framework include testing: incident
response processes; backups, to ensure that critical information and systems can
be restored within critical timescales; incident response processes; and disaster
recovery / fail-over processes?

A.1.06 Is your technical security assurance framework supported by sufficient budget,

skilled resources, processes, tools and technology; backed up by adequate
management support and an IT or Cyber security risk management programme?

An IT or Cyber security risk management programme would typically include:

a documented risk management architecture and framework; a risk
management strategy (including risk appetite); details of relevant legal,
regulatory and contractual compliance requirements; a list of all main
threats, a risk register showing exposure of key assets; and a method of
assessing the effectiveness of technical security arrangements.
Step 2 Establish a penetration testing governance structure Maturity level: Level 1
A.2.01 Have you established a suitable governance structure to oversee and coordinate a
regular penetration testing programme?
An effective governance structure for penetration testing would typically
cover all main systems enterprise-wide, while focusing on the most critical,
allowing for the protection of any sensitive information.

A.2.02 Have you established a joint management and technical team to agree the
programme and scope of regular penetration testing?
An effective management and technical team would typically have direct
access to senior management to raise significant concerns, supported by the
ability and authority to contribute to a wider security improvement,
providing adequate control over the penetration testing programme.

A.2.03 Does your penetration testing programme include an approved: set of penetration
testing processes and methodologies that apply enterprise-wide, supplier selection
criteria, a penetration testing assurance management framework and a range of
follow up activities to ensure that remediation activities are carried out in an
effective manner, reducing the risk of vulnerabilities being exploited in the future?

A.2.04 Is your penetration testing programme reviewed and approved by appropriate

business and IT management, supported by stated objectives and timelines, and
integrated in to your underlying technical security assurance framework?

A.2.05 Does your penetration testing programme align with a wider security review
framework, technical security infrastructure and system development processes
(particularly for Web applications)?

A.2.06 Do you have a change management process that enables the secure introduction
of or changes to: business initiatives, business processes, web applications and IT
infrastructure; legal and regulatory requirements; your threat landscape, security
governance approach and security controls framework?

A.2.07 To support your penetration testing programme, do you maintain key performance
indicators for the results of the penetration tests, subscribe to information sharing
platforms or services and use them to feed into the penetration testing
programme?

A.2.08 Are a series of actions taken to provide assurance about the suitability and
effectiveness of your penetration testing programme?
Appropriate assurance actions would typically include traceability and
monitoring of the programme, a continuous improvement process, and
independent audits (or similar).

Step 3 Evaluate drivers for conducting penetration tests Maturity level: Level 1
A.3.01 Have you identified drivers for carrying out penetration tests as part of a technical
assurance programme?
A.3.02 Are your drivers for carrying out penetration tests based on an evaluation of
relevant criteria?
Criteria for determining the drivers for penetration testing should include any
growing requirement for compliance, the impact of serious cyber security
attacks, any outsourcing services used, the introduction of new systems and
services, significant changes to IT or the business and changes in the type or
level of perceived threat.

A.3.03 Do your drivers for carrying out penetration tests take account of how a
penetration test fits into your organisation’s overall security arrangements; the
nature and direction of your business – and your risk appetite?

A.3.04 Are your drivers for carrying out penetration tests informed by findings from risk
assessments, audits or reviews; analysis of security incidents; and lessons learnt
from any previous penetration tests?

A.3.05 Have you placed your penetration tests within a wider context of security
assessment and strategy to help contextualise the findings and recommendations?

A.3.06 Do your drivers for penetration testing help to: support the adoption of a strategic
view of security management; ensure that major system vulnerabilities are
identified and addressed; and reduce the risk of discovering that the same
problems still exits the next time a penetration test is carried out?

Step 4 Identify target environments Maturity level: Level 1

A.4.01 Have you identified target environments that need to be subject to penetration
testing, such as critical web applications and important IT infrastructure?
A.4.02 Does your identification of the target environment take into account business
processes; web applications; key parts of IT infrastructure and specialised
equipment (e.g. mobile devices and process control systems)?

A.4.03 Does your identification of the target environment consider system criticality;
compliance requirements; major business or it services; critical systems under
development; and outsourced services (e.g. cloud computing)?

A.4.04 Does your identification of the target environment include a risk assessment of
your organisation’s critical information and systems – and ensure that the testing
planned will focus on the assets which pose the highest risk to your organisation?
A.4.05 Does your identification of the target environment take into account significant
changes to critical business processes, business applications, IT infrastructure and
business environments (e.g. in particular business units or jurisdictions)?

A.4.06 Have penetration testing requirements been built into relevant stages of systems
development lifecycles (SDLC) in use by your organisation?
Consideration should be given to conducting penetration tests during the
testing stage; implementation stage; and during live operation.
A.4.07 Have you gained permission to test important systems / environments controlled
by third parties?
If you are not permitted to test important systems / environments controlled
by third parties, you should gain assurances that appropriate penetration
tests are regularly carried out; tests are conducted by suitably qualified staff
working for a certified organisation; and recommendations from the tests are
acted upon.

Step 5 Define the purpose of the penetration tests Maturity level: Level 1
A.5.01 Do you define the purpose of penetration tests, and assess how these tests can
help your organisation?
A well-defined penetration tests should help your organisation to identify
weaknesses in your security controls; reduce the frequency and impact of
security incidents; comply with legal and regulatory requirements; provide
assurance to third parties that business applications can be trusted and that
customer data is adequately protected; and limit liabilities if things go wrong,
or if there is a court case (i.e. take ‘reasonable’ precautions).

A.5.02 Do you determine what penetration testing will help you to achieve (i.e. the
benefits)?
Benefits of penetration tests can include IT cost reductions; technical and
business improvements; as well as greater awareness of security risks and
controls.

A.5.03 Do you consider the scope limitations of penetration testing?

When evaluating the scoping limitations of penetration testing you should

take into account that a test covers just the target environment that has
been selected; is only a snapshot of a system at a point in time; and plays
only a small part in an organisation’s defence system.

A.5.04 Do you consider the technical limitations of penetration testing?

 When evaluating the testing limitations of penetration testing you should
take into account that a test focuses on the exposures in technical
infrastructure; can be limited by legal or commercial considerations (limiting
the breadth or depth of a test); and may not uncover all security weaknesses.

A.5.05 Do you evaluate the potential difficulties involved with scoping penetration tests?

The potential scoping difficulties involved with carrying out penetration

testing can include: determining the depth and breadth of coverage of the
test; identifying the style, type, targets and frequency of tests; managing
risks associated with potential system failure and exposure of sensitive data;
and remediating system vulnerabilities effectively.

A.5.06 Do you evaluate the potential difficulties involved with resourcing penetration
tests?
The potential resourcing difficulties in carrying out penetration testing can
include: understanding the costs of external services (and in determining the
true overall cost of testing); and finding a suitable penetration testing expert
when required (e.g. at short notice).

Step 6 Produce requirements specifications Maturity level: Level 1

A.6.01 Do you define formal requirements for penetration testing carried out in your
organisation?
A.6.02 Do your requirements for penetration testing include consideration of any impact
on important business applications, key systems and networks (IT infrastructure)
and confidential data?

A.6.03 Do your requirements for penetration testing specify that testers must validate
that the test will be legal and that it will not compromise data protection
requirements?

A.6.04 Are your requirements for a penetration test formally recorded in a requirements
specification, formulated by competent technical experts, reviewed and signed-off,
monitored to ensure they are met and then reviewed / revised on a regular basis?

Step 7 Select suitable suppliers Maturity level: Level 1

A.7.01 Do you appoint suitable third party suppliers to undertake independent
penetration testing, based on defined requirements?
Effective requirements for penetration testing suppliers are typically based on
a cost / benefit analysis, driven by clear objectives, recorded in a
requirements specification and integrated into an organisation’s
procurement process.

A.7.02 Do you evaluate the benefits of using external suppliers?

 When evaluating the benefits of using external suppliers, you should consider
their ability to help you: deploy a structured process and plan, developed by
experts; increase the scope and frequency of tests; conduct short term
engagements, eliminating the need to employ your own specialised (and
often expensive) staff; and take advantage of automation (e.g. by using
penetration testing workflows and importing vulnerability management
reports).

A.7.03 Do you define selection criteria to help you choose suitable suppliers?

Effective supplier selection criteria typically specify that potential suppliers

should be able to: provide a reliable, effective and proven penetration testing
service; meet compliance standards; protect your information and systems
both during and after testing; perform rigorous and effective penetration
tests; adhere to a proven testing methodology; carry out a full range of
testing, discover all major vulnerabilities, identifying associated ‘root causes’;
produce insightful, practical and easy to read reports; provide on-going
advice on how to manage systems effectively over time as part of a trusted
relationship.

A.7.04 Does your selection criteria consider if potential suppliers can provide: solid
reputation, history and ethics; high quality, value-for-money services; research and
development capability; highly competent, technical testers; and security and risk
management, supported by a strong professional accreditation and complaint
process?

A.7.05 Do you ensure that your chosen suppliers are able to effectively meet – or exceed -
your supplier selection criteria and provide tangible value for money?

A.7.06 Do you validate the ability of potential suppliers to meet your specific
requirements (not just one who can offer a variety of often impressive products
and services, some of which may not necessarily be relevant)?

A.7.07 Do you go through a formal, approved appointment process for selected

penetration testing suppliers?
 Results
Maturity model for Stage B - Testing

Maturity score Target Evidence supplied

Step 1 Agree testing style and type Maturity level: Level 1
B.1.01 Do you identify what style of penetration testing is required?

B.1.02 Does your identification of testing style evaluate the need for ‘Black box’, ‘Grey
box’ and / or ‘White box’ testing?
B.1.03 Does your identification of testing style consider the use of an ‘external’
penetration test (the most common type of test), which is aimed at IT systems
from ‘outside the building’?

B.1.04 Does your identification of testing types consider the use of an internal security
test; end-to-end testing (i.e. for people, through data, devices, applications and
infrastructure); emerging technologies (e.g. mobile applications); and social
engineering?

B.1.05 Does your identification of testing types consider Web application testing,
Infrastructure testing and Specialised penetration testing, such as for mobile, client
server or cloud-based applications

B.1.06 Is your test environment as similar to the live environment as possible?

Step 2 Identify testing constraints Maturity level: Level 1

B.2.01 Do you identify any testing constraints associated with the planned penetration
testing?
B.2.02 When identifying testing constraints, do you allow for aspects of the business that
cannot be tested due to operational limitations, considering that attackers often
do whatever it takes to penetrate an organisation or system (If they are not able to
penetrate a particular system, they may simply try another route.)?

Methods of dealing with operational testing constraints can include

simulating live tests as closely as possible and conducting tests outside of
normal hours (and locations).

B.2.03 When identifying testing constraints, do you allow for testing having to be
conducted within the confines of the law (considering that attackers often do
whatever it takes to penetrate an organisation or system)?

Methods of dealing with legal testing constraints can include tailoring the
way tests are structured and run to simulate most forms of attack) and
taking back-ups of critical systems and files before testing.
B.2.04 When identifying testing constraints, do you allow for testers being limited to the
scope of the testing and a finite time to conduct tests, considering that attackers
will utilise the weakest point of security in any part of connected systems or
networks to mount an attack, regardless of ownership, location or jurisdiction –
and will often have unlimited time to mount a concerted attack against a system if
they have the motivation, capability and resources to do so?

Methods of dealing with scope-related testing constraints can include placing

perimeter controls within the scope of the test and applying more rigorous
testing to applications that are accessible from outside traditional
organisational boundaries.

B.2.05 When identifying testing constraints, do you allow for testers having limited time
to conduct tests, considering that attackers have unlimited time to mount a
concerted attack against a system if they have the motivation, capability and
resources to do so?

Methods of dealing with time constraints can include Investing more time in
testing critical systems; providing testers with as much background
information as possible; and conducting penetration testing on a regular
basis, rather than as a one-off exercise.

B.2.06 When identifying testing constraints, do you allow for the likelihood that most
penetration testing will not find all vulnerabilities of a given environment (the law
of diminishing returns often applies in that the most obvious vulnerabilities will be
discovered first, with further time yielding more and more obscure issues)?

Methods of dealing with this type of testing constraint can include adopting
a ‘risk to cost balance’ when performing tests and doing more than simply
fixing vulnerabilities uncovered during testing as this could leave a number of
other vulnerabilities present for an attacker to find.

B.2.07 Have you identified technical issues that can affect the scope of the test or the
security countermeasures in place to detect and deter attacks?
Methods of dealing with technical testing constraints can include
Implementing policy exceptions; allowing for vulnerabilities that will not be
discovered if the testing is undertaken from outside your network; adopting a
practical scope that will meet your requirements; and ensuring that the test
simulation comes very close to replicating a real malicious attack.

B.2.08 Have you determined how you will make sure that all parties adhere to testing
constraints?

Step 3 Produce scope statements Maturity level: Level 1

B.3.01 Do you formally define the scope of penetration tests prior to tests commencing?
B.3.02 Is the scope of penetration tests recorded in a formal document, such as a scope
statement, that is signed-off by all relevant parties?
Relevant parties (i.e. named individuals or groups) required to sign-off the
scope statement should include authorised and suitably qualified individuals
from all relevant parties; plus relevant, qualified individuals dependent on
the value of the system being tested (or similar).

B.3.03 Does your scope statement include a definition of the target environment?

The definition of the target environment should include: which systems are in
and out of scope; the testing approach being adopted (e.g. black, white or
grey box); types of test that are prohibited (e.g. ‘denial of service’ type
testing); where the testing team will need to be in order to conduct the
testing (e.g. on the customer’s site or at the test service provider’s premises);
and approvals required for various elements of the testing to go ahead.

B.3.04 Does your scope statement specify resourcing requirements?

Resourcing requirements should specify who will be leading the testing

engagement, the names of testers that will be used for the testing
engagement (with details about their roles, skills, experience, qualifications
and backgrounds) and the number of days required (including the days on
which testing will take place) – and require a disclaimer stating that they are
legally authorised to carry out specified activity on your property and
systems.

B.3.05 Does your scope statement define liabilities?

The definition of liabilities in the scope statement should specify the steps
required by both parties should problems (e.g. slippage) arise and the details
of liability (indemnity) insurance to be held by the testing service provider.

B.3.06 Does your scope statement include follow-up activities?

Follow-up activities should include presentation of key findings and

recommendations to senior management and any re-testing needed once
mitigations have been made for the discovered vulnerabilities’ required by
both parties should problems (e.g. slippage) arise.

B.3.07 Do you formally define reporting requirements for your penetration testing prior
to tests commencing?
B.3.08 Do your reporting requirements specify the format and type of content to be used
in the test report (template often used; when the test report will be delivered (not
later than a few days after completion of the test); and how the test report will be
delivered (electronic and / or physical)?
Step 4 Establish a management assurance framework Maturity level: Level 1
B.4.01 Are you aware that responsibility for the actual systems and data during
penetration testing – and any assurance about them - rests with your
organisation?

B.4.02 Have you created a documented management assurance framework to help

manage all aspects of penetration tests?
B.4.03 Does your management assurance framework provide assurance to stakeholders
that the objectives of penetration tests are achieved (i.e. business requirements
are met)?

The management assurance framework should provide assurance to

stakeholders that contracts with service providers are defined, agreed,
signed off and monitored and risks to your organisation (e.g. degradation or
loss of services; disclosure of sensitive information) are kept to a minimum.

B.4.04 Does your management assurance framework provide assurance to stakeholders

that changes to the scope of tests – and any problems arising - are well managed?

The management assurance framework should provide assurance to

stakeholders that any changes to the scope of penetration tests (e.g.
additional testing requested, such as to include wireless or device testing) or
to organisational controls (e.g. to address a critical weakness uncovered
during testing) are managed quickly and efficiently; and that any problems
(or complaints) arising during tests (e.g. due to resources not being made
available, tests not working as planned or an ethical breach) are
satisfactorily resolved.

B.4.05 Have you established an assurance process to ensure that the penetration testing
process meets requirements?
B.4.06 Does your assurance process define control processes over all important
management aspects of testing, including test administration; test execution, and
data security?

Test administration typically includes scope of tests, legal constraints,

disclosure and reporting; test execution typically includes testing approach,
separation of systems and duties, tool heritage, traceability and repeatability
of tests; whilst data security typically includes secure storage, transmission,
processing and destruction of critical or sensitive information provided or
accessed during the test.

B.4.07 Is the scope of your penetration tests documented in an agreement, defined in a

legally binding contact and signed off by all relevant parties before testing starts
 Penetration testing contracts should specify explicit exclusions (e.g. systems
that are out of scope); any technical and operational constraints; roles and
responsibilities for all parties’ concerned; and specific legal, regulatory and
operational requirements (e.g., timings and checkpoints; a problem
escalation process and post-test corrective action strategy).

Step 5 Implement management control processes Maturity level: Level 1

B.5.01 Is your organisation aware that performing any sort of penetration test carries
with it some risk to the target system and the business information associated with
it (e.g. degradation or loss of services; disclosure of sensitive information)?

B.5.02 Have you developed methods of keeping risks to your organisation to a minimum?

You can help to reduce risk associated with penetration testing by carrying
out planning in advance; having a clear definition of scope; using predefined
escalation procedures; supported by qualified testing individuals and certified
organisations.
B.5.03 When conducting penetration tests, do you ensure that those individuals
responsible for the running of the target systems have full knowledge of the tests
to help protect against unexpected business consequences, such an inadvertent
trigger of internal controls; and are aware of – and adhere to - any escalation
procedures?

B.5.04 Are individuals responsible for the running of the target systems available, as
required, during the test period to help: ensure that testing takes place as agreed;
keep risks within acceptable boundaries; deal with any problems arising; and
manage issues that have been escalated?

B.5.05 Is your penetration testing supported by a change management process?

An effective change management process should cover changes to the scope

of the penetration test, organisational controls and the individuals on the
testing team; as well as ensuring that all parties involved adhere to the
process and that changes to penetration testing are made quickly and
efficiently.

B.5.06 Is your penetration testing supported by an effective problem resolution process?

An effective problem resolution process should cover tests not working as

planned and resources not being made available; as well as problems caused
as a result of the penetration testing, which can include: interruptions to or
degradation of live systems; unauthorised disclosure of confidential
information; and compromise of the integrity of information (e.g. affecting
the accuracy or timeliness of information).

The problem resolution process should also include breaches of: contract;
specifications in the scope statement; and a relevant code of conduct.
B.5.07 Are problems arising during penetration testing resolved in an effective, timely
manner in accordance with your problem management process?

Step 6 Use an effective testing methodology Maturity level: Level 1

B.6.01 When conducting penetration tests do you use a systematic, structured testing
methodology?
B.6.02 Is your penetration testing methodology based on proven approaches designed by
authoritative publicly available sources?
Authoritative publicly available sources include the Open Source Security
Testing Methodology Manual (OSSTM) or the penetration testing in SP800-
115.[3] and the Open Web Application Security Project (OWASP).

B.6.03 Does your penetration testing methodology detail specific evaluation or testing
criteria and adhere to a standard common language and scope for performing
penetration testing (i.e. security evaluations)?

Specific evaluation and testing criteria include the Information Systems

Security Assessment Framework (ISSAF) and a standard common language
and scope for performing penetration testing is defined in the Penetration
Testing Execution Standard (PTES).
 Results
Maturity model for Stage C - Follow up

Maturity score Target Evidence supplied

Step 1 Remediate weaknesses Maturity level: Level 1
C.1.01 Do follow-up activities include remediating weaknesses identified in penetration
testing?
C.1.02 Are weaknesses remediated in line with a comprehensive, approved remediation
process?
An effective remediation process should include addressing all issues;
applying immediate or short terms solutions (e.g. patching systems, closing
ports and preventing traffic from particular web sites or IP addresses),
replicating results of penetration tests, determining which weaknesses to
address first (e.g. based on risk ratings for critical assets), and reporting
weaknesses to relevant third party organisations.

Step 2 Address root causes of weaknesses Maturity level: Level 1

C.2.01 Do follow-up activities include analysing and addressing the root causes of
weaknesses identified in penetration testing?
C.2.02 Does root cause analysis include the full range of required actions?

Root cause analysis should include: identifying the real root causes of
exposures; evaluating potential business impact; identifying more endemic or
fundamental root causes; qualified, experienced security professionals to
help define corrective action strategy and plans.

Step 3 Initiate improvement programme Maturity level: Level 1

C.3.01 On completion of penetration tests, is an improvement programme initiated?

C.3.02 Is the improvement programme carried out in a structured / systematic manner?

C.3.03 Does your improvement programme include all key elements?

The improvement programme should address root causes of weakness;

evaluate penetration testing effectiveness; identify lessons learned; apply
good practice enterprise-wide; create and monitor action plans; and agree
approaches for future testing.
Step 4 Evaluate penetration testing effectiveness Maturity level: Level 1
C.4.01 Is the effectiveness of your penetration testing evaluated?

C.4.02 Does evaluation of test effectiveness cover the full range of required actions?

Evaluation of the effectiveness of penetration testing should include:

determining if objectives were met; assessing if sufficient weaknesses were
identified; reviewing exploitations undertaken; comparing test results to
external benchmarks; and determining if value for money was obtained from
your service provider.

Step 5 Build on lessons learned Maturity level: Level 1

C.5.01 Does your penetration testing approach include identifying lessons learned,
disseminating them to relevant stakeholders and acting on them?
C.5.02 Are lessons learned used to help in planning future tests, and provide feedback to
service providers to help them improve processes?
C.5.03 When addressing the weaknesses identified in an environment, are good practices
identified (including fixes) and then applied to a wide range of other
environments?

C.5.04 Are good practices rolled out by: performing trend analysis across multiple
systems; applying lessons learnt during a penetration test of one application to
similar applications; and fixing root causes endemically?

Step 6 Create and monitor action plans Maturity level: Level 1

C.6.01 Are action plans created to help act upon follow-up activities undertaken?

C.6.02 Are action plans formally documented, formulated by competent technical

experts, reviewed by business management and signed-off by senior
management?

C.6.03 Do action plans outline all the relevant actions to be taken to prevent
vulnerabilities identified through testing from recurring and help improve the
overall information security programme

C.6.04 Do action plans include a brief description of each action, including their priority
and category; individuals responsible and accountable for each action; and target
dates for completion

C.6.05 Are action plans implemented effectively and on a timely basis?

C.6.06 Are action plans monitored on a regular basis to: ensure progress is being made;
highlight any delays or difficulties being experienced; and reassess the level of risk?