Software Testing

and
Quality Assurance
Lecture 7 – Software Testing Levels
 2
1. Unit testing

 Unit testing is a process of testing the smallest building blocks of a software system, i.e. unit
or module or component, individually.

 This is because, unit or module must be validated before integrating it with other modules.

 Unit tests check module behavior against specifications or expectations and ensure that

 every unit of software has been built in the right manner in conformance with user
requirements and

 the software meets at least a baseline level of functionality prior to integration.

 With unit testing, debugging becomes easy, and parallel development and testing is
facilitated.
 3
Cont.

 To mention some of the errors that can be uncovered with unit testing :

 Comparison of different data types

 Incorrect logical operators or precedence

 Precision inaccuracy ( e.g. Expectation of equality when precision error makes

equality unlikely)

 Incorrect comparison of variables

 Improper loop termination

 Unit testing is simplified when a component with high cohesion is designed.

 4
2. Integration Testing

 In a modular design of a software system where the system is composed of different

modules, integration is the activity of combining the modules together.

 Integration testing is the process of combining and testing multiple components or

modules together.

 The technique combines all unit-tested modules and performs a test on their aggregation.

 The intention here is to uncover the bugs that arise when unit tested modules are
integrated.

 But one may ask, when we have tested all modules in unit testing, where is the need to
test them on aggregation?
 5
Cont.

 The answer is interfacing.

 When we unit test a module, its interfacing with other modules remain untested.

 So, interfacing between units must be tested when one module is combined with
another.

 The objective here is:

 To see if the modules can be integrated properly.

 To make sure all components collaborate correctly.

 To ensure that there are no errors in parameter passing or when one module invokes
another module.
 6
Cont.

 Why is this needed?

 It is because faults may occur in the data exchange or in the communication between
the components /modules.

 A component transmits syntactically incorrect or no data.

 The communication works but the involved components interpret the received data
differently (contradicting or misinterpreted specifications).

 Data is transmitted correctly but at the wrong time, or it is late (timing problem), or
the intervals between the transmissions are too short/long.
 7
Cont.

 So, Integration testing is necessary for the following reasons:

 It exposes inconsistency between the modules such as improper call or return

sequences.

 Data may be lost across an interface.

 One module when combined with another module may not give the desired result.

 Data types and their valid ranges may mismatch between the modules.

 Thus, overall Integration testing focuses on errors caused by interfacing between the
modules while integrating them.
 8
Approaches/Strategies to Integration Testing

 In which order should the modules be integrated?

 In practice, mostly different software components are completed at different times.

 There are 2 incremental integration strategies for integration testing.

 9
Approaches/Strategies to Integration Testing

I. Top-down Integration

 This integration method starts with the high-level modules and moves downward to the
low-level modules through the hierarchy.

 Modules subordinate to the top module can be integrated using Depth first or Breadth
first approach.

II. Bottom-up Integration

 As its name implies, this method begins testing with atomic modules (i.e., components at
the lowest levels in the program structure).

 The integration moves upward through the control hierarchy from bottom to top level
modules.
 10
3. System Testing

 Testing at this level focuses on testing the entire integrated system.

 This test validates that all elements mesh properly and that overall system
function/performance is achieved.

 System testing incorporates a series of different tests to test the whole system on various
grounds such as functionality, performance, security, etc.

 Overall, the goal of the system test is to validate whether the complete system meets the
specified functional and nonfunctional requirements and how well it does that.
 11
Some of Non Functional System Tests

A. Recovery Testing

 Recovery testing is the activity of testing how well the software is able to recover from
disasters such as crashes, hardware failures, and other similar problems.

B. Security Testing

 Security testing attempts to verify that protection mechanisms built into a system will, in
fact, protect it from improper penetration.

 Given enough time and resources, good security testing will ultimately penetrate a
system. The role of the system designer is to make penetration cost more than the value
of the information that will be obtained.
 12
Cont.

 The basic security elements that need to be covered during security testing are:

 Confidentiality: Protection against the disclosure of information to parties other than

the intended recipient.

 Integrity: Ensure that the information received has not been altered in transit.

 Authentication: Establish confidence that information or message received indeed

originates from a specific known source.

 Authorization: Determining that a requester is allowed to receive a service or perform

an operation.

 Non-repudiation: Support to prevent the later denial that an action happened.

 Cont.
C. Performance Testing 13
 This is testing the run-time performance of the software on the basis of various
performance factors.

 These performance factors are memory use, response time, throughput, and delays.

 This testing requires that performance requirements must be clearly mentioned and
quantified in the SRS.

D. Load Testing

 The normal system testing takes into consideration only nominal values of the resources.

 However, there is high probability that the system will fail when put under maximum load.

 With this in mind, Load testing tests a system with all its limits.

 A system is tested with a load that causes it to allocate its resources in maximum
amounts.
 Cont.
14
 The idea is to test a system under maximum load that are beyond the normal workloads
and determine the maximum sustainable load the system can handle without running out
of resources.

 For example, the maximum operating capacity of a system in terms of number of users or
requests.

E. Stress Testing

 Stress testing is also a type of load testing, but the difference is that the system is put
under loads beyond the limits so that the system breaks.

 This testing tries to break the system under test by overwhelming its resources in order to
find the circumstances under which it will crash.
 15
Cont.

F. Usability Testing

 The goal of usability testing is to verify that intended users of the system are able to
interact properly with the system while having a convenient experience.

 Usability characteristics against which testing is conducted are:

 Ease of use: easy entrance, navigation, and exit.

 Interface steps: steps should not be misleading and complex.

 Response time: request response should not be so high that the user is frustrated.

 Help system: precise help documentation

 Error messages: clear and meaningful error message

 16
4. Acceptance testing

 Software is being built to satisfy the user requirements and no matter how elegant its
design is, it will not be accepted by the users unless it helps them achieve their goals as
specified in the requirements.

 Ultimately, it’s the user that needs to be satisfied with the application, not the testers,
managers, or contract writers.

 So, Acceptance testing is the formal testing conducted to

 (1) determine whether a software system satisfies its acceptance criteria contracted
with the customer and

 (2) to enable buyers (customers) to determine whether to accept the system or not.
 17
Cont.

 This testing gives the end user a chance to provide the development team with feedback
as to whether or not the software meets their needs.

 Acceptance testing must take place at the end of the development process.

 Acceptance testing is classified into 2 categories.

 Alpha Testing: Tests are conducted at the development site by the end users.

 Beta Testing: Tests are conducted at the customer site or production environment.
 18
Regression Testing

 As stated, software testing and quality assurance activities goes throughout the evolution
of a software.

 According to IEEE software glossary,

 Regression testing is the selective retesting of a system or component to verify that

change or modifications have not caused unintended effects and that the system or
component still complies with its specified requirements.
 19
Cont.

 So, when changes are introduced, we must ascertain that

 the modifications work correctly and

 the modification has not introduced any new faults in the portion that was not subject
to modification.

 It may be possible that small changes in one part of a program may have subtle
undesired effects on other unrelated modules of the software.
 20
When is Regression Testing Done?

 The answer is after performing software maintenance activities. This can be after

 Corrective maintenance (Changes made to correct a system after a failure)

 Adaptive maintenance (Changes made to achieve continuing compatibility with the

target environment or other systems)

 Perfective maintenance (Changes designed to improve or add capabilities)

 21
How extensive a regression test has to be?

 However, the question is how extensive a regression test has to be.

 The amount of regression testing to perform after a change is dependent on issues

such as:

 The risk associated with the change

 The architecture of the system or product

 The nature of change introduced

 22
Cont.

 Based on this the following are possibilities:

1. Testing of all program parts or elements that were corrected, changed or newly
integrated (testing of altered and new functionality)

 Tests that execute only the area of modifications are not enough because it neglects
the consequences these alterations can have on unaltered parts.

 In software systems, simple local code changes can create side effects in any
other, arbitrarily distant, system parts.
 23
Cont.

2. Testing of the whole system (complete regression test)

 All the tests in the existing test suite should be re-executed.

 If necessary, new test cases must be specified to examine the modified or new
feature.

 However, a complete regression test is usually too time consuming and expensive.

3. Select a subset of the test cases from the existing pool to reduce the cost.

 A key question is how many and which test cases should be selected so that the selected
test cases are more likely to uncover new faults.