Harrine Freeman

ÓEYEWIRE

T
here are several types of software testing that are include courses they are registered for, academic status, gen-
widely used in today’s IT world. I will explain some of der, social security number, and address.
these types of tests and describe my experiences with There are several different types of static techniques.
them. When a tester knows what type of testing is ◗ Statement coverage testing is performed by executing ev-
needed, it greatly improves the test results and ultimately de- ery statement at least once.
creases the number of defects. ◗ Branch coverage testing is performed by running a series of
tests or test cases to ensure that all branches of a test re-
quirement or software component are tested at least once.
Glass Box Testing ◗ Path coverage testing is performed by testing all the
Glass box testing (also known as structural testing, white box, various paths of each test requirement or software
or clear box testing) focuses on the inside of the “box”; it relies component.
on the internal knowledge of the system as a method for test- ◗ All definition use path coverage testing is performed by test-
ing. Knowledge of the code is needed during glass box testing ing all the various paths or software components be-
(Fig. 1). Glass box testing has several variations of testing: tween the definition of a particular variable and the use
static and dynamic analysis, which facilitates the testing of the of that definition that are identified and tested.
software product being tested. There are several advantages of glass box testing. It forces
Dynamic analysis is the testing portion that involves run- the tester to use reason when testing the software. It approxi-
ning the system. This type of testing can be used for a software mates the partitioning performed by execution equivalence. It
product such as a database of college students, which might reveals all errors in the “hidden” code. It reveals optimizations.

48 IEEE Instrumentation & Measurement Magazine September 2002

1094-6969/02/$17.00©2002IEEE
 The disadvantages of glass box testing are that it is expen- uses previous test cases and scenarios developed by the
sive and a tester may miss cases omitted in the code. testers. The user should manipulate data through the
software product using various scenarios without rely-
Black Box Testing ing on the knowledge of the requirements but using their
Black box testing (also known as behavioral, functional, closed knowledge of the business logic. If acceptance testing is
box, or opaque box testing) focuses on testing the functionality not successful, the developers and testers should meet to
of the system using the functional requirements. Black box test- determine how to resolve the failures and to ensure a
ing treats the system as a “black box.” This type of testing is successful acceptance test.
based solely on system requirements (Fig. 2). This it can be used Regression testing is a form of system testing that verifies
when testing an order tracking system. An example is purchas- that previous test cases or test scenarios work properly. Re-
ing a product over the Internet. It involves creating a user ac-
count, selecting items to purchase, entering personal
Static
information (name, address, telephone number), and entering Analysis
payment information. All functionality stated in the require-
ments must be met before the software product is ready for pro-
duction, e.g., one requirement would be that a user has to be
Manual Automated
able to select an item to purchase and have the ability to select
additional items to purchase or delete the items currently in the
shopping cart. Knowledge of the code is not necessary during Static Symbolic
Inspection Walkthrough
black box testing, knowledge of the requirements is. Verification Execution

System Testing Fig. 1. Glass box testing.

System testing allows the tester to prove that the system meets
all objectives and system requirements. System testing is a sort
of verification process and provides an external view of the
Input Test Data
system (Fig. 3). This type of testing should be approached in a
manner that would be easy for the user to use, since they are
not concerned with how the system works, the system re-
sponses, and the interface. The user only wishes to know that
the system functions properly. System
System testing verifies the system when the testing occurs
in a nonlive test environment using nonlive test data and
could also be referred to as verification testing. An IT group
consisting of testers, developers, end users, and operations
Output Test Results
staff usually performs system testing. This type of testing is
based on system requirements, an architecture document, or
an operations document. This type of testing would ensure
Fig. 2. Black box testing.
that the software product operates
in various operating systems,
Internet browsers, Internet speeds,
System Requirements High-Level Business
and resolutions. Design Analysis Requirements Design
◗ Acceptance testing is a form of Document Document Document

system testing. It allows the

tester to prove that the system User
meets all the user require- Documentation
ments or business logic. It is a
form of validation testing and
is performed by end users. Ac-
Integration Functional Performance Acceptance
ceptance testing is the final Testing Testing Testing Testing
stage of the testing process and
uses customer-supplied data
rather than simulated data and
confirms that the system is Unit
ready for production. Testing

◗ Acceptance testing is also a

form of regression testing and Fig. 3. Software testing processes.

September 2002 IEEE Instrumentation & Measurement Magazine 49

gression testing ensures that no anomalies appear in the sys-
tem due to changes made. This involves testing any
modifications made to the system to ensure no new anomalies Experienced
Tester
are found. An example of this type of testing would be to test a
shopping cart feature of an order tracking system by using an
existing user and purchasing additional products or purchas-
System
ing products as a new user ensuring that all the key function- Analyst
Test Team Designer
ality of the system is tested.
◗ Load testing identifies the volume of traffic accessing a
particular application. It measures the number of simul-
taneous users that can successfully access the applica-
tion. Load testing determines an optimum number of Configuration
User Management
simultaneous users. The simplest form of load testing al- Analyst
lows users to log on simultaneously and navigate
through the application and monitor the results. A more
Fig. 4. Software test team.
complex form of load testing can be performed by using
an automated test tool such as Load Runner by Mercury
ample of this type of testing would be to ensure that sensitive
Interactive or QALoad by Compuware.
information appears encrypted during various stages of the
◗ Stress testing usually coincides with load testing. Stress
purchase process, i.e., the user’s social security number should
testing steadily increases the load on the system beyond
appear encrypted on the user information page and the user’s
the maximum design load until the system fails or
credit card number should appear encrypted on a receipt page.
crashes. It benefits the testing by revealing the behavior
of the failures of the system.
▲ It determines if system overload results in loss of data
Test Team
A testing team should be carefully selected. Fig. 4 illustrates
or service.
the necessary members. A tester should be familiar with the
▲ It also stresses the system and may cause certain
technology, business, and user requirements or business logic.
defects to arise that may not normally be detected.
In a perfect world, testing should end when no more errors are
System testing can be performed informally by allowing
found, but there is rarely enough time in a project schedule for
small user groups to log on simultaneously, then allowing an-
this to occur. Testing usually ends when the acceptance thresh-
other small user group to logon within a specific time period,
old has been met (when management and the test group de-
e.g., 20 minutes later, and continue with additional user groups
cide that the existing errors are not showstoppers and can be
for a specified time period. The user groups are allowed to nav-
fixed at a later date). To ensure that a software product is thor-
igate through the application and the results are monitored.
oughly tested the proper testing method should be used, in
addition to an automated test tool, to ensure that all software
Database Testing and system requirements are met.
Database testing determines how well a database meets the
system requirements. Database testing is an ongoing process. References
When a database is created, a mirror of that database should [1] P. Dyson, Mastering Microsoft Internet Information Server 4, 2nd ed.
be developed and used as a test database with test data similar Sybex, 1997.
to actual data that will be entered into the application when it [2] R. Kramer, Glass Box Testing, Available http://www.cis.ysu.edu/
is implemented into production. Database testing identifies ~kramer/DataStructures/Intro/GlassBox.html
query response time, data integrity, data validity, and data re- [3] H. Hauser, Glass Box Testing, Available
http://www.issco.unige.ch/ewg95/node8.html
covery; it should always be performed when testing any soft-
[4] D. Howe, Glass Box Testing, Available
ware product. An example of this type of testing, using the
burks.bton.ac.uk/burks/foldoc/55/112.htm
order tracking system, is creating a new user and verifying
[5] B. Beizer, Black Box Testing: Techniques for Functional Testing of Soft-
that the data entered appears in the appropriate tables in the ware Systems. New York: Wiley, 2000, p. 8.
database and verifying that the record counts of those tables
have increased. Harrine Freeman has been a senior tester at McDonald Bradley
since May 2000. She has more than eight years of quality as-
Security Testing surance experience, including the use of automated software
Security testing attempts to violate built-in security controls. It test tools, SEI-CMM Level 2, and the software development
ensures that the protection mechanisms implemented in the life cycle process. She has four years of management experi-
software application secure it from abnormal penetration, e.g., ence leading various quality assurance teams in a myriad of
an order tracking system that locks out a user from the system environments. She has performed quality assurance of main-
when three logon attempts fail. This would prevent hackers or frame, client/server, and web applications. She is a member of
any unauthorized user from accessing the account. Another ex- the IEEE and Women in Technology.

50 IEEE Instrumentation & Measurement Magazine September 2002