January 1993 ComputerAudit Update

IT S E C U R I T Y T E S T I N G , which protect a specific data type or applica-

tion function and which, if defective, could
A PRACTICAL GUIDE m allow a specific type of fraud to occur, e.g.
control on financial authorization limits.
PART 3
Examples of the types of components on
HARDWARE/SOFTWARE SECURITY which security testing may be performed include:
TESTING
• A PC security card used for access control
Bernard Robertson and David Pullen purposes.
PA Consulting Group

Introduction A terminal used for the entry of confidential

data where there may be a concern about the
This article, the third in a series on the subject level of electromagnetic emissions.
of security testing, describes how to go about the
security testing of IT hardware and software • An access control software application.
systems. Examples of the types of tests that may
be performed are given.
• A cryptographic device used to provide link
Hardware/software security testing involves encryption.
determining how well a component resists
attempts to subvert its security mechanisms. • Asoftware application handling classified, fin-
Both positive and negative testing (defined in the ancially sensitive or safety critical information.
first article of this series) can be applied in
hardware/software. Test process

Scope The general process to be followed when

performing hardware or software security testing
Only components (hardware devices or is detailed in a previous article: 'The Security
software modules) which could have an impact Testing Process'. In summary the key steps are:
on the overall system security should be subject
to security testing. As test resources are usually • System familiarization, which includes athor-
limited, testing effort may be focused by ough documentation review (see below).
categorizing the systems to be tested. A
specimen categorization is components with
security controls: • Business and security control identification.

which, if defective, could put the entire system • Risk identification.

at risk e.g. the access control system of a
hardware security module • Selection of appropriate testing techniques.

protecting sensitive security data which, if • Formal structured tests.

defective, could compromise confidential in-
formation within the system, e.g. tamper re- • Attack tests.
sistant mechanisms within an encryption
device
• Problem classification and reporting.

©1993 Elsevier Science Publishers Ltd 7

Computer Audit Update January 1993

The following sections describe stages in the Types and examples of tests
test process.
A wide variety of positive and negative testing
Documentation review techniques may be used and typical examples
are given for both hardware and software in the
A t h o r o u g h r e v i e w of the s y s t e m following section. This is described firstly for
documentation needs to be performed as early in hardware and then for software.
the test cycle as possible. The three primary
purposes of this review are: Hardware security tests

Checks on the accurate implementation of

• To assist in the system familiarization pro-
specified functions
cess.
For example if the module under test utilizes
To determine the quality of the documenta- a cryptographic processor, tests should be
tion. This will give the tester an indication of performed to check that the algorithm is
the overall quality of the system. Experience implemented correctly. This may be done by
indicates that a system with poor quality do- comparing the output of the processor with that
cumentation is likely to have serious security from the same algorithm implemented on a test
problems. tool. A key test to perform when a cryptographic
process is used is to determine what happens to
To compare the currency of the documenta- the data that is normally encrypted when the
tion with the actual system. If updates have processor is removed or bypassed. Is sensitive
been made to the system without amend- data now sent in clear form? Another test should
ments to the documentation, poor change be performed to determine how the absence or
control procedures may be in place for both failure of the cryptographic processor is reported
the system and the documentation. to the system control - - do diagnostic systems
work? Other tests to ensure that generated keys
Relevant documentation would include are suitably random may be performed by
(depending .on the nature of the system under statistical analysis.
test):
Test tamper resistant mechanisms
• functional requirements specification, The key areas to evaluate are:

• design specification, • The strength of the tamper resistance relative

to the risks the unit faces.
• circuit diagrams,
• The ability of the unit to actively destroy all the
• process flow charts, secret information in a timely fashion.

This type of testing can take considerable

• dataentity descriptions,
amounts of time and resource so for reasons of
efficiency the following process should be
• memory management structures. followed:

A section in the final test report should

describe the results of the documentation review. • Fully understand the function and identify the
key protective controls.

8 ©1993 Elsevier Science Publishers Ltd

January 1993 Computer Audit Update

weaknesses arise when the power inputs or clock

Examine all design drawings and documents
speed are varied outside their specified bands.
to evaluate the intended response to threats
and identify any vulnerabilities (such as
Change in the hardware configuration
chemical, X-ray or intrusion attacks).
Tests should be performed to examine the
Confirm the existence of the vulnerabilities- modules security state when hardware elements
this may be done by a paper analysis or by are modified or removed. For example removing
live tests. an access control board from the terminal it was
protecting or removing the battery used to power
Perform some unusual (negative) attacks on the key storage area in a cryptographic
the unit to determine how it reacts. For processor.
example dropping, heating or cooling the unit
with the objective of making the detection Device authentication security
mechanisms fail without deleting the secret
If it is easy to make unauthorized duplicates
information.
of the security hardware it may be possible to
Evaluate hardware reliability and resilience masquerade as a legitimate unit and perpetrate
a fraud. A safeguard against this line of attack is
This test area investigates the reaction of the to perform some form of secure device
system to a partial or total failure. The optimal authentication (normally a challenge response
system reaction would be to fail safe. For type protocol). Tests should be performed to
example a test could be to determine what check that the safeguards function as specified.
happens when the disks used for audit trail In addition design effort may be spent trying to
logging become full or when the system is make the unit hard to duplicate - - perhaps by the
brought down and then restored to service. If the use of embossed designs or holograms.
audit logging is lost in either case a major security E x a m p l e s of units t h a t are t a r g e t s for
breach could remain undetected. unauthorized duplication are:

Test environmental security Access control hardware units and cash dis-
pensing machines (where the objective is to
Tests should be performed to determine the
collect PIN and card data).
effect on the component of extremes of
temperature, h u m i d i t y and m a g n e t i c
interference. These tests should identify whether Terminals on a LAN where an attacker may
any security weaknesses are exposed when the read all sensitive information such as pass-
words on the LAN.
module fails due to any of these factors. The tests
should be based on the e n v i r o n m e n t a l
Failures in diagnostic function
performance specification of the device and test
the reaction of the module to conditions beyond Tests should be performed to test the
the specified boundaries. A full environmental response of the module to failures in the self-test
test chamber may be required for these tests. diagnostic functions. The tests should determine
whether the failures are reported and whether
Power and speed variation
any security weaknesses arise. For example,
confidential information may be transmitted in
In these tests the modules power inputs or
clear text if the diagnostics fail to report that the
clock speed are varied. Tests should be
cryptographic processor in a link encryptor is out
p e r f o r m e d to d e t e r m i n e if any s e c u r i t y
of service.

@1993 Elsevier Science Publishers Ltd 9

Computer Audit Update January 1993

Software security testing Use of terminate and stay resident (TSR)

programs.
Static analysis
A TSR program is normally active at all times
Static analysis tools provide the software and can allow authorized activities to take place
security tester with information on the functioning every time an application is run. Tests should be
of software modules while they are inactive. undertaken to determine if a TSR can be run
Static analysis tools include disassemblers or simultaneously with the application under test.
decompilers which may be used to access the For example, a banking application accessed by
object code and convert it to its source code form. PC-based terminals could be subverted by a
These tools would enable the tester to read TSR. The TSR could send its own commands
confidential information (such as encryption from an otherwise legitimate terminal address;
keys) stored in the source code. these commands would be invisible to the
terminal user and it could transfer funds
Dynamic analysis fraudulently. The best defence against this type
of attack is for the application program to inhibit
Dynamic analysis tools provide information the TSR by removing all unrequired applications
on the software whilst it is running. Examples of from primary storage as its first activity.
dynamic tools include debuggers and code
exercisers. A debugger can allow a tester to Attempting to access the operating system by
determine intermediate values (perhaps during a abnormal exits from the application
key generation cycle) while a code exerciser can
be used to identify areas of infrequently used Most security applications try and prevent
code that may contain unauthorized functionality. users accessing operating system functions.
Such areas of code should be examined in more However, in many applications, it is relatively
depth by code analysis. easy to cause the application to terminate
abnormally and return the user to the operating
Code analysis system. An abnormal exit can occur if fields are
caused to overflow or if particular keystroke
Code analysis involves an in-depth review of combinations are entered. An example is the
each line of code in the areas selected for testing. entry of a ControI-AIt-Del string in a DOS
The objective of code analysis is to detect any application. Once in the operating system,
code of dubious security or purpose. It should be system file, password files and system trace tools
undertaken by someone who is fluent in the
become available and may be used to corrupt
programming language used to write the
data or deny access to authorized applications.
application. In performing code analysis it is
important to focus attention on the key functions
Control of source code
to be analysed. If this is not done the scale of the
test will be too large and the level of detailed
The software security tester should ascertain
analysis required will not be achieved. Areas
how securely copies of the software source code
which the tester should focus on are: access
control modules; sensitive record processing; are stored. Insecurely stored software allows an
and financial limit sub-routines. attacker to:

When a doubtful area of code is found the Modify the software before it is introduced into
quickest way to check its intended purpose is to the operational environment. The unauth-
ask the programmer. However, the reply supplied orized introduction of software versions into
should not be accepted without question, it the production environment should be de-
should be verified to the complete satisfaction of
tected by change control procedures.
the tester. Comments in the code should be
viewed with scepticism as any fraudulent piece of
code would not broadcast its function.

1 (3 (~1 ~cl.q I = l ~ v i A r .qni~nnA P= = h l i ~ h A r ¢ I trl

January 1993 Computer Audit Update

This article has described hardware and

• Examine source code listings to find weak-
software testing and given some examples of the
nesses that could later be exploited.
types of tests that may be performed. The next
Invalid data article in this series describes the security testing
of communication channels.
Tests should be performed to exercise all of
Bernard Robertson is a Principal Consultant
the data validation controls within the application.
For example, attempting to enter illegal values, in the Security Consulting Practice of PA
perhaps alphabetic characters in a field for a Consulting Group. He has extensive experience
in performing a range of security testing
telephone number, or entering too many
programmes for the public and financial sector
characters in a field for a financial amount to
determine how the system reacts to an overflow clients. Bernard is a regular speaker on IT
in this field. security issues and holds degrees in Economics
and Business Administration. David Pullen is a
File manipulation Senior Consultant within the same Security
Consulting Practice. Over the last five years he
It may be possible to subvert the processing has conducted several security testing projects,
of an application if data or source files are including one lasting two years with a team of 15
manipulated. Tests should determine if it is security testers. David is a physics graduate and
possible to circumvent controls on failed a qualified teacher who has produced a wide
password attempts by manipulating the file range of educational material on security testing.
containing this count with a file edit utility.

Complex interaction tests

Interaction tests should be performed when AUDITING THE

a c o m p l e x c o m b i n a t i o n of a p p l i c a t i o n
e n v i r o n m e n t s , o p e r a t i n g s y s t e m s and
CONTINGENCY PLAN
c o m m u n i c a t i o n protocols exist around a
business application. These tests can take the David Firth
form of modifying avalue supplied by a base layer
to an application layer and monitoring the A contingency plan, by its very nature, must
response of the application. For example, if an work if ever it has to be implemented. If the plan
application receives encryption services from the fails to meet the recovery targets or provide the
operating system, the reaction of the application expected service levels, the results may be
to a key management failure should be tested. catastrophic.

Error conditions Ask yourself honestly whether your own plan

will be effective if your pager alerted you today
A valuable testing technique is to try and that there had been a fire or bomb at your data
recreate all possible error conditions. Tests which centre or key offices. Despite the time and effort
generate the error conditions should create all put into contingency or business continuity
foreseen problems and exercise all the error planning it is likely that only a minority could put
handling code. The tester should set about their hand on their heart and answer 'yes'. Others
creating error conditions so that all error may find cause for concern in that their plans
messages listed in the system documentation are have not been kept up-to-date or tested
obtained. Once this has been achieved testing effectively.
may continue by creating other unspecified error
conditions and monitoring the response. The development of a contingency plan
r e p r e s e n t s a major investment for most
organizations. The costs of the backup site, the

©1993 Elsevier Science Publishers Ltd 11