5SENG007CSoftware Engineering Principles and Practice

Lecture Week 10

Topic: Testing & Designing Test Cases

Intended Learning Outcomes:

ILO 1: To understand the Testing Process & Principles

ILO 2: To identify prominent Types of Testing

Disclaimer: Original Slides are enhanced by incorporating additional Images

Evaluation and Testing
•Testing process
o Setup test environment
o Test case execution
o Regression Testing

•Determine purpose
o Validation of requirements
o User experience Testing
o Performance & Scalability Assessment

•Allocate responsibilities
o Defect tracking, communication & reporting

•Document what happens

o Test Documentation reviews, Traceability Matrix
o Lessons learned
Levels of Testing
•Unit testing:
o Functions, Member functions etc

• Module testing:
o Classes, Lower Level System (LES) testing etc

• Sub-system testing:
o Libraries

• System testing
o Performance Testing

• Integration testing
o Incremental integration, Top-down Integration

• Acceptance testing
o alpha and beta testing

• Review and Maintenance

Some Testing Principles
•Complete testing is not possible
• Testing is creative
• Testing needs planning
o Test Plan
o Test Cases
o Test Suite

• Testing should be unbiased

• Ensure repairs don't create new errors
o Regression Testing
o Continuous Integration Testing
Test Case Template
Test Scenario: Verify the login functionality of the Gmail account.

Source: https://www.softwaretestingmaterial.com/test-case-template-with-explanation/
 Test Case Template
SCREEN NAME

PREPARED BY COMPANY LOGO

SoftwareTestingMaterial
DESIGNATION

DATE

SCENARIO ID SCENARIO DESCRIPTION

S.NO TEST CASE ID TEST CASE DESCRIPTION PRECONDITION TEST DATA EXPECTED RESULT POSTCONDITION ACTUAL RESULT STATUS DEFECT ID COMMENTS
Types of Testing
•Requirements testing
o Requirement Validation, Traceability Testing

• Design testing
o Interface Testing

• Functional testing: Black box testing

o Equivalence Partitioning, Boundary Value Analysis

• Non-functional testing, e.g. Performance Evaluation

o Stress Testing, Recovery Testing

• White box testing, Code inspections and walk-throughs

• Formal methods
Requirements testing
•Any missing requirements?
o e.g. lists ending with \etc"

• Any superfluous requirements?

o Prioritization

• Any over-complex requirements?

• Any contradictory requirements?
o e.g. specs with \and/or"

• Any ambiguous requirements?

o e.g. generalities: \where practicable“
o e.g. absolutes: \never"
o e.g. comparatives: \earliest"
Class Structure Testing
•Classes and Sequence Diagrams
•Interpret the class diagram and sequence diagrams side by side.
•Begin reading the sequence diagram, and as the messages are sent between
the objects, trace these messages back to the relationships on the class
diagram.
•Doing so should be helpful in understanding why the relationships on the
class diagram exist.
•After reading the sequence diagrams, if you find a relationship on a class
diagram that can't be mapped to a method call, you should question the
validity of the relationship.
Functional Testing
Black Box testing/Closed box testing:
• Knows what the code should do.
• The internal structure/design/implementation is not known to
the tester.

Equivalence Class Partitioning

• Break problem into Equivalence Classes.
• Then provide test data and expected results
• Test and ascertain actual results
 Functional Testing
Equivalence Classes (ECs):
• Program should behave the same for all members of the same EC
• E.g. the function IsOdd() should be TRUE for all odd numbers and
FALSE for all even numbers

Deriving ECs:
• A data item can belong to several ECs
• E.g. 'a' is both alphabetic and
Sample Test Plan
•Consider a function to convert lowercase characters to uppercase
characters

o Equivalence Class: Lowercase

o Purpose: Are lowercase chars converted?
o Data: ‘c'
o Expected Result: ‘C'
o Actual Result:

o Equivalence Class: Uppercase

o Purpose: Are uppercase chars ignored?
o Data: ‘C'
o Expected Result: ‘C'
o Actual Result:
Sample Test Plan
o Equivalence Class: non-alphabetic characters
o Purpose: are non-alphabetic chars ignored?
o Data: ‘5'
o Expected Result: ‘5'
o Actual Result:
Test varying data
•Boundary conditions (BCs):
o Check for lowest and highest values
E.g. 'a' and 'z'; '0' and MAX
Check for either side of BCs
E.g MAX-1, MAX+1
Files:
Empty, non-existent, no permissions
Numbers: zero, too big, too small
Exceptions:
Leap years, daylight saving
negative prices, double-booking etc
Fault Seeding
Technique that deliberately and secretly introduces faults (seeds) into a
program.
To observe how well the testing process can detect these artificially
created faults.
Useful to assess the effectiveness of the testing efforts and to
determine when to stop testing.
Phases of Testing
Plan and Design the Test:
o Start early, with every new prototype revisit to update any change in the
specification, create new test cases, new configuration support.
o Initially generate tests for the prototype with iterative builds

Prepare the test:

o Testing teams update their code, tests, documents and test environment
and align it with one another.

Perform the test:

o Run the test suit again. If any defect is found, test around the defect to
make sure that the bug is certain.
Phases of Testing
Report the results:
o Complete details about the bugs are reported. Communicate with the
development team.

Repair the Bug:

◦ The test team participates in this step by explaining the bug to the
developing team and provides direct testing to track the bug.

Return to step 1 and retest: A new build is produced after one cycle.
Testing Cycle
Design
Testing begins when the project begins.
Beginning of the project: Define the scope, Evaluate project design and
assets, Review and correct.
Software test plan document is also produced in this early phases.
The quality of the software is evaluated by code, graphics and sounds
where needed that are produced and compiled into the code.
Proper documentation testing (project plan, design and test plan) can
help in finding and solving problems sooner and cheaper.
Delaying testing till the end may cause some serious problems including
project failure.
Testing Stages
•The prototype is tested against criteria established in planning phase.

•Software developed by different programmers is linked together and

the design is tuned.

•Major feature of this stage is to run the software from start to finish
along some path and revise it.

•All modules of the software are tested at least for once.

•It is very critical to stick to the test plan, and carefully perform the
entire tests defined in the plan.
Improved Prototype
•Focus mostly on tuning the software, the developers has already
created.

•Identifies and fixes the remaining bugs.

•Usually performed by testers from outside the developing team.

•Run the software along all possible paths, the entire user interface is
final, software logic and algorithmic implementation AI are final, all HCI
issues and implemented.
Testing Techniques
•Adhoc Testing
• Informal Testing Approach, Exploratory Testing Nature

•Combinatorial Testing
• Systematically test different combinations of input values to see how they
interact.

•Test Flow diagrams

•Cleanroom testing
• A systematic and thorough approach to create high-quality software from the
start, emphasizing precision and reliability.

•Test Trees
• Breaking down a big task into smaller, more manageable parts. Each branch of
the tree represents a specific test, helping to ensure that every part of the
software is thoroughly checked.
Ad hoc testing
•Adhoc Testing is also sometime refers to as general testing. It is less
structured test.
•Ad hoc testing allows tester to explore paths based on their intuition.
•There are two kinds of ad hoc testing.
•First is free testing, which is testing without any planning or documentation.
•The second is direct testing, which is a single test, improvised to answer a
specific problem.
•The team's test strategy is to produce tests that will run/use the software
the way user will.
Combinatorial Testing
•How much testing should be enough?

•Software quality has to be good enough for customer, but testing has to stop
before the release date.

•It is neither practical nor economical, to test every possible combination of

software event, configuration, function and options. However, skipping some
testing or taking shortcuts is always risky.

•Combinatorial testing is a quick way to find defects earlier in the software

while keeping the test sets small, covering as much area as possible.
Combinatorial Testing
•Key idea of combinatorial Testing is to detect the interaction failures.
o Since not every parameter contributes to every failure, and most failures
are triggered by a single parameter value or interactions between two or
more parameters.

o In combinatorial testing, parameters (individual elements) are selected

from software elements, functions and choices such as software events,
software settings, hardware setting, customization choices etc.

o The test then created can be homogenous (test parameters of same type)
or heterogeneous (test parameters of different types).
Test Flow diagrams:
•Test flow diagrams (TDF) are used to create models representing software
behaviour from user's perspective.

•Testing takes place by exploring the model, along all possible paths to explore
the unexpected states.

•TDF is a formal approach to test design.

•Because TDF is graphical in nature, it is easy to review, analyse and provide

feedback on test designs.

•Complex features can be represented by complex TDF's, but usually small TDF
are preferable.

•Test: in particular, the state design.

Test Flow diagrams:
Cleanroom testing
Statistical use testing
◦ tests the actual usage of the program

determine a “usage probability distribution”

◦ analyze the specification to identify a set of stimuli
◦ stimuli cause software to change behavior
◦ create usage scenarios
◦ assign probability of use to each stimuli
◦ test cases are generated for each stimuli according to the usage
probability distribution
Test Trees
•Test tree is a usability technique for organizing test cases, which helps in
selection of proper set of tests for a given set of code change.

•Test Tree improves the overall understanding of complex software features

and take care of potentially chaotic function, especially when these
functions interact with other functions and elements.

•The test tree is constructed by decomposing the feature into subsets until
the bottom nodes identify elements to use or specification to perform
during testing.
Test Trees