Unit 4 – Domain Testing

Compiled with reference from:

Software Testing Techniques: Boris Beizer
Narasimha Rao.P
Craft of Software Testing: Brain Marrick
 Domain Testing U3

We will see in this part of Unit 3:

• Domains & Paths

• Nice & Ugly Domains

• Domain Testing

• Domains & Interfaces Testing

• Domains & Testability

ref boris beizer 2

 Transaction-Flow Testing U3

Contents of the unit

• Domains
• Model, as a set, paths
• Predicates
• closure, dimensionality, assumptions
• Nice & ugly domains
• specified & implemented domains
• Nice domains – properties, testing implications
• Ugly domains, simplification by programmers & testers
• Domain testing
• Bugs & testing for those bugs
• Testing procedure
• Variations, tools, effectiveness
• Domains & Interface Testing
• Range, closure, span
• Interface range, domain compatibility
• Finding the values
• Domains & Testability
• Linearizing, coordinate transformations. Canonical program form

ref boris beizer 3

 Domain Testing - Domains & Paths U3

Domain Testing Model

D1
D2
INPUT CLASSIFY DO CASE 1 OUTPUT
D3

(x,y,z,…) Dn { Outcome }
DO CASE 2
(1, 2, 3, 4, 5,…)

DO CASE 3

DO CASE n

4 ref boris beizer

 Domain Testing U3

Domain Testing

Views Programs as input data classifiers

Verifies if the classification is correct

5 ref boris beizer

 Domain Testing - Domains & Paths U3

Domain Testing Model

Two Views

• Based on Specs

• Functional Testing

• Based on Implementation information

• Structural Technique

6 ref boris beizer

 Domain Testing - Domains & Paths U3

Domain Testing Model

• Variables
• Simple combinations of two variables
• Numbers from - to +

• Input variables as numbers

• Loop-free Programs

7 ref boris beizer

 Domain Testing - Domains & Paths U3

Domain Testing Model

• Structural Knowledge is not needed. Only Specs.

• For each Input Case,

• A Hypothetical path for Functional testing

• An Actual path for Structural testing

8 ref boris beizer

 Domain Testing - Domains & Paths U3

Domain (simplified)

• A Single Connected Set of numbers D1

• No arbitrary discrete sets

• Defined by the boundaries

• One or more boundaries

• Specified by predicates

• Bugs likely at the boundaries

• One or more variables

9 ref boris beizer
 Domain Testing - Domains & Paths U3

Predicates

• Interpretation

• Structural Testing - CFG

• Functional Testing - DFG

• Specifies the Domain Boundary

• n Predicates in Sequence  2n domains

• Or, just two domains A .AND. B .AND. C

10 ref boris beizer

 Domain Testing - Domains & Paths U3

Paths

• At least One PATH thru the Program

• More paths for disconnected domains

11 ref boris beizer

 Domain Testing - Domains & Paths U3

Summary:
• Domain for a loop-free program corresponds to a set of
numbers defined over the input vector

• For every domain, there is at least one path thru the

routine, along which that domain’s processing is done

• The set of interpreted predicates traversed on that path

(ie., the path’s predicate expression) defines the
domain’s boundaries.

12 ref boris beizer

 Domain Testing - Domains & Paths - Domain Closure U3

Boundary: Closed / Open

X >= MIN X <= MAX

MIN MAX
D1 D2 D3

Closed

X >= MIN
D2 X < MAX
D1 D3

MIN Closed Open MAX

13 ref boris beizer

 Domain Testing - Domains & Paths - Domain Closure
Boundary: Closed / Open

MIN MAX
D1 D2 D3

Open
X > MIN X < MAX

14 ref boris beizer

 Domain Testing - Domains & Paths - Domain Dimensionality U3

• One dimension per variable

• At least one predicate

• Slices thru previously defined Domain

• Slicing Boundary

• N-spaces are cut by Hyperplanes

15 ref boris beizer

 Domain Testing - Domains & Paths – Bug Assumptions U3

• Processing is OK. Domain definition may be wrong.

 Boundaries are wrong.

 Predicates are wrong.

• Once input vector is set on the right path, it’s correctly

processed.

• More bugs causing domain errors …

16 ref boris beizer
 Domain Testing - Domains & Paths – Bug Assumptions.. U3

• Double-zero representation
• Positive and negative zero, errors for –ve zero

• Floating point zero check

• Zero by def, subtract, multiply

• Contradictory domains
• means that at least two supposedly distinct domains overlap

• Ambiguous domains
• union of the specified domains is incomplete
• either missing domains or holes in the specified domains
defined
• what happens to points on the domain boundary
undefined
17 ref boris beizer
 Domain Testing - Domains & Paths – Bug Assumptions.. U3

• Over-specified domains X > 3 .AND. X < 2 .AND. Y > 3

overloaded with so many conditions that the result is a null domain, path not achievable

• Boundary errors

Boundary closure

Shifted
18 ref boris beizer
 Domain Testing - Domains & Paths - Bug Assumptions U3

• Boundary errors….

Tilted

Missing

Extra boundary

19 ref boris beizer

 Domain Testing - Domains & Paths - Bug Assumptions U3

X >= 3
• Closure Reversal
• Incorrect implementation of logic

• Faulty Logic

Simplification of compound
predicates

20 ref boris beizer

 Domain Testing - Domains & Paths - Bug Assumptions U3

Strategy for domain testing in 2-dim

21 ref boris beizer
 Domain Testing - Domains & Paths - Restrictions U3

In General…

1. Coincidental Correctness
• DT cannot detect where outcome is correct for the wrong reasons
Example –
• if the domain is a disconnected mess of small subdomains, each of which has a binary
outcome or if outcomes are restricted to a few discrete values

F1(x) : +1

• Representative outcome D1

Partition testing – Any i/p represents

all i/ps in that domain

D2
Input Equivalence
F2(x) : -1

22 ref boris beizer

 Domain Testing - Domains & Paths U3

Domain Testing Model

Function f1
D1
INPUT CLASSIFY DO CASE 1 OUTPUT

(x, y, z, …) D2 { Outcome }
DO CASE 2
(1, 2, 3, 4, 5,…) f2

D3
DO CASE 3
f3

Dn
f4
DO CASE n

23 ref boris beizer

 Domain Testing - Domains & Paths - - Restrictions U3

3. Simple boundaries & Compound predicates

D1
X X >=0 .AND. X <= 16
0 16

D1
Y Y = 1 .AND. X >=0 .AND. X <= 16
0 16
X

24 ref boris beizer

 Domain Testing - Domains & Paths - - Restrictions U3

Compound predicates….

• Impact of .OR.

• Concave, Disconnected

• Adjacent domains with same function

• Example

ABC + DEF

• Eliminate compound predicates

25 ref boris beizer
 Domain Testing - Domains & Paths - Restrictions U3

4. Functional Homogeneity of Bugs

• Functional form still Retained

a x + b y >= c

• Bugs are only in a, b, c

26 ref boris beizer

 Domain Testing - Domains & Paths - - Restrictions U3

5. Linear Vector Space

• Linear boundary predicate, Interpreted

• Simple relational operators

• Conversion to linear vector space

• 2-d Polar co-ordinates

• Polynomials

• Problems with Non-linear Boundaries

27 ref boris beizer

 Domain Testing - Domains & Paths - Restrictions U3

6. Loop-free Software

Predicate for each iteration

Loop over the entire transaction

Definite loop

28 ref boris beizer

 Domain Testing - Domains & Paths – Nice & Ugly domains U3

Nice Domains
Requirements

• Bugs  ill-defined domains

Before DT

• Analyze specs

• Make the Boundary Specs Consistent & Complete

29 ref boris beizer

 Domain Testing - Domains & Paths – Nice domains U3

Implemented Domains

• Complete, Consistent & Process all inputs

Specified Domains

• Incomplete, Inconsistent

Programmer’s / Designer’s Effort

30 ref boris beizer
 Domain Testing - Domains & Paths – Nice domains U3

Nice Domains

• Linear, Complete, Systematic, Orthogonal, Consistently

Closed, & Convex

U1 U2

V1 D11 D12

V2 D21 D22

31 ref boris beizer

 Domain Testing - Domains & Paths – Nice domains U3

Nice Domains

Advantages

• Ease of DT

• Fewer Bug Occurrences

32 ref boris beizer

 Domain Testing - Domains & Paths – Nice domains U3

Nice Domains
Boundaries are

1. Linear
2. Complete
3. Systematic
4. Orthogonal
5. Closure Consistency
6. Convex
7. Simply Connected

33 ref boris beizer

 Domain Testing - Domains & Paths – Nice domains U3

1. Linear Boundaries
Interpreted linear inequalities
only two (test) points to determine a straight line
three points to determine a plane, and
n + 1 points to determine an n-dimensional hyperplane

n-dim Hyperplane:

n+1 Points

n+1 + 1 Test Cases

Non-Linear
more than 99.99% of all boundary predicates are either
directly linear or can be linearized by simple variable transformations
• Transform

34 ref boris beizer

 Domain Testing - Domains & Paths – Nice domains U3

2. Complete Boundaries

• Span the total number space (-, +)

• One set of Tests

Incomplete…

• Reasons
• Unachievable paths
• compound predicates that define a single boundary
• redundant predicates convert suchboundary values into a null set

• Tests
• Few number of tests for complete boundaries

35 ref boris beizer

 Domain Testing - Domains & Paths - Nice domains U3

3. Systematic Boundaries
• Linear Inequalities differing by a constant

fj (X) ≥ kj or, fj (X) ≥ g (j, c) g (j, c) = j + k * c

where fi is an arbitrary linear function, X is the input vector, ki and c are constants – Parallel lline
and g(i,c) is a decent function over i and c that yields a constant, such as k + ic. – Systematically spaced
parallel lines

• Parallel lines
• Identical Sectors in a Circle

36 ref boris beizer

• DT
 Domain Testing - Domains & Paths - Nice domains U3

4. Orthogonal Boundaries

• Two boundaries or, boundary sets

• Parallel to axes
every inequality in V is perpendicular to every inequality in U
If two boundary sets are orthogonal, then they can be tested independently
• DT

• Each Set Independently Uj

• # Tests  O (n)
The notation O(n) means of the order of. For example, O(n) is a linear growth,
O(n2) is a quadratic growth, and
O(2n) is an exponential growth Vj

37 ref boris beizer

 Domain Testing - Domains & Paths - Nice domains U3

Orthogonal Boundaries
• Tilted sets
• transformation
• Test cases: O(n)
Orthogonality can depend on the coordinate system chosen
• Concentric circles with radial lines

• Rectangular coordinates (Not

orthogonal)
• Polar (Orthogonal)
r ≥ aj .AND. r < aj+1 .AND.
 ≥ j .AND.  < j+1

38 ref boris beizer

 Domain Testing - Domains & Paths - Nice domains U3

Non-Orthogonal Boundaries

• Test Intersections  O ( n2 ) + O (n)

39 ref boris beizer

 Domain Testing - Domains & Paths - Nice domains U3

5. Closure Consistency

• A Simple pattern in all boundary closures

• Example

• Same relational operator for systematic boundaries

D3 D1 D2 D4

40 ref boris beizer

 Domain Testing - Domains & Paths - Nice domains U3

6. Convex Domain
• Line joining any two points lies with in the domain

• DT
• n on-points & 1 off-pt

Concave

• “ But, However, Except, Or … “ in Specs

• Handle with special care

41 ref boris beizer

 Domain Testing - Domains & Paths - Nice domains U3

7. Simply Connected Domain

• In a single piece D2 D1 D3

D2
D1

• 2 complementary domains

• D1: Convex  D2: Concave, not-Connected

• Programmer / Designer
• Convex part First
42 ref boris beizer
 Domain Testing - Domains & Paths – Ugly Domains U3

Generally,

From Bad Specs

Programmer / Designer Simplifies => Ugly to

Good

possibility of Introduction of more bugs ?!?

43 ref boris beizer

 Domain Testing - Domains & Paths – Ugly Domains U3

Causes

1. Non-linear Boundaries
2. Ambiguities & Contradictions
3. Simplifying the topology
4. Rectifying boundary closures

44 ref boris beizer

 Domain Testing - Domains & Paths – Ugly Domains U3

1. Non-linear Boundaries

• Transform

45 ref boris beizer

 Domain Testing - Domains & Paths – Ugly Domains U3

2. Ambiguities

• Holes in input vector space.

• Missing boundary

• Detected by Specification languages & tools.

46 ref boris beizer

 Domain Testing - Domains & Paths – Ugly Domains U3

2. Contradictions..

• Overlapping of

• Domain Specs

• Closure Specs

D3

D1 D2

47 ref boris beizer

 Domain Testing - Domains & Paths – Ugly Domains U3

3. Simplifying the Topology….

• Complexity !
• Generic cases
• Concavity, Holes, Disconnectedness
Programmers introduce bugs and testers mis-design test cases by
3. smoothing out concavities
4. filling in holes
5. joining disconnected pieces

6. Other Ways to simplify the domain’s topology

7. Connecting disconnected boundary segments and
8. extending boundaries out to infinity

48 ref boris beizer

 Domain Testing - Domains & Paths – Ugly Domains U3

3. Simplifying the Topology….

• Smoothing out concavity

• Filling in Holes

49 ref boris beizer

 Domain Testing - Domains & Paths – Ugly Domains U3

3. Simplifying the Topology….

• Joining the pieces

Correct:

• Connect disconnected boundary segments

• Extend boundaries to infinity

50 ref boris beizer

 Domain Testing - Domains & Paths – Ugly Domains U3

4. Rectifying Boundary Closures.

• make closures in one direction for parallel boundaries with

closures in both directions

• Force a Bounding Hyperplane to belong to the Domain.

Consistent Direction Inclusion / Exclusion Consistency

51 ref boris beizer
 Domain Testing - Domains & Paths – Domain Testing U3

General DT Strategy

1. Select test points near the boundaries.

2. Define test strategy for each possible bug related to

boundary

3. Test points for a domain useful to test its adjacent domain.

4. Run the tests. By post test analysis determine if any

boundaries are faulty & if so how?

5. Run enough tests to verify every boundary of every domain

52 ref boris beizer

 Domain Testing - Domains & Paths – Domain Testing U3

DT for Specific Domain Bugs

Generally,

• Interior point - point in the domain such that all points within an arbitrarily small distance (called
an
epsilon neighborhood) are also in the domain.
• Boundary point - within an epsilon neighborhood there are points both in the domain and not in
the domain.
• Extreme point - that does not lie between any two other arbitrary but distinct points of a (convex)
domain
• On point - is a point on the boundary
• Off Point - a point near the boundary but in the adjacent domain if the domain is closed
- a point near the boundary but in the domain being tested if the domain is open
You can remember this by the acronym COOOOI: Closed Off Outside, Open Off Inside

53 ref boris beizer

 Domain Testing - Domains & Paths – Domain Testing U3

DT for Specific Domain Bugs

Off Points

Extreme
point

Boundary point

Epsilon neighborhood
On Points

Domain D1
54 ref boris beizer
 Domain Testing - Domains & Paths – Domain Testing U3

1. 1-d Domains

2. 2-d Domains

3. Equality & inequality Predicates

4. Random Testing

5. Testing n-dimensional Domains

55 ref boris beizer

 Domain Testing - Domains & Paths – Domain Testing U3

Testing 1-d Domains : Bugs with open boundaries

B A B A
Closure Bug
x x

B A
Shift left Bug
x1 x

B A
Shift Right Bug
x x1

56 ref boris beizer

 Domain Testing - Domains & Paths – Domain Testing U3

Testing 1-d Domains : Bugs with open boundaries

B A B A
Missing Boundary x
x x

Extra Boundary B A C
x x
x x1

Bugs with Closed Boundaries : Similar to the above

57 ref boris beizer

 Domain Testing - Domains & Paths – Domain Testing U3

2. Testing 2-d Domains

• Closure bug

• Boundary Shift : up / down

• Tilted Boundary

• Extra Boundary

• Missing Boundary

58 ref boris beizer

 Domain Testing - Domains & Paths – Domain Testing U3

2 n : Domains share tests

3 n : no sharing of tests by domains

Strategy for domain testing in 2-dim

59 ref boris beizer
 Domain Testing - Domains & Paths – Domain Testing U3

3. Equality & Inequality Predicates

• An Equality predicate defines a line in 2-d

a
A
c c’
c
d
B

To avoid bugs

60 ref boris beizer

 Domain Testing - Domains & Paths – Domain Testing U3

4. Random Testing

• A Point in the center : verifies computation

61 ref boris beizer

 Domain Testing - Domains & Paths – Domain Testing U3

5. Testing n-Dimensional Domains (strategy)

n-dimensions, p boundary segments

• (n+1)*p test cases : n on points & 1 off point

• Extreme pt shared : 2 * p points

• Equalities over m-dimensions create a subspace of n-m

dimensions

• Orthogonal domains with consistent boundary closures,

orthogonal to the axes & complete boundaries

• Independent testing
62 ref boris beizer
 Domain Testing - Domains & Paths – Domain Testing U3

Procedure for solving for values

Simple procedure. Need tools.

1. Identify input variables

2. Identify variables which appear in domain-defining

predicates, such as control-flow predicates

63 ref boris beizer

 Domain Testing - Domains & Paths – Domain Testing U3

Procedure

3. Interpret all domain predicates in terms of input variables:

• Transform non-linear to linear
• Find data flow path

4. Predicate expression with p # predicates.

Find # domains : < 2 p

3. Solve inequalities for extreme points

4. Use extreme points to solve for nearby on points …

64 ref boris beizer

 Domain Testing - Domains & Paths – Domain Testing U3

Effectiveness

• Cost effective

• Bugs on boundaries, extreme points

• Hardware logic testing – tool intensive

65 ref boris beizer

 Domain Testing - Domains & Paths – Domain Testing U3

Domains & Interface Testing

• Domain

• Range

Variable Domain Range

Function /
Classify
Routine

66 ref boris beizer

 Domain Testing - Domains & Paths – Domain Testing U3

Domains & Interface Testing

Variable
Range for Domain for Range for
Routine 1 Routine 2
Domain Routine1 Routine2 Routine2

• Span compatibility

67 ref boris beizer

 Domain Testing - Domains & Paths – Domain Testing U3

Domains & Interface Testing

• Closure compatibility

68 ref boris beizer

 Domain Testing - Domains & Paths – Domain Testing U3

Interface Range/Domain Compatibility Testing

• Test each var independently

• Find an inverse function

• Build a super domain

69 ref boris beizer

 Domain Testing - Domains & Paths – Domain Testing U3

Finding the values / solving inequalities

70 ref boris beizer

 Domain Testing - Domains & Paths – Domain Testing U3

Domains and Testability

• Linearizing transformations

• Polynomial
• Rectangular to polar
• Generic & rational => Taylor series

71 ref boris beizer

 Domain Testing - Domains & Paths – Domain Testing U3

Domains and Testability

Perform transformations for better testing.

• Co-ordinate transformations

72 ref boris beizer

 Domain Testing - Domains & Paths – Domain Testing U3

Domains and Testability

• Canonical Program Form

73 ref boris beizer

 Domain Testing – Questions from the previous year’s exams U3

1. Explain Nice & Ugly Domains.

2. What is domain testing? Discuss applications of domain testing

3. Explain the domain boundary bugs for two dimensional domains

4. What is the purpose of Domain Testing? Give its schematic

representation

5. Define the following concepts.

1. Domains 2. Domain closure 3. Domain
dimensionality 4. Bug assumptions for Domain Testing

6. Explain simple domain boundaries & compound predicates.

7. Classify what can go wrong with boundaries, then define a test

strategy for each case in domain testing.
74 ref boris beizer
 Domain Testing U-3

To Unit 5…Paths, Path Products & Regular Expressions

75 ref boris beizer