Test configuration generation in Combinatorial testing - Example | Set-2

We have discussed the algorithm used for test configuration generation in the article Test Configuration Generation in Combinatorial Testing. Let's understand the algorithm in-depth with a solved example given below.

Example

Consider an application that works on any of the three browsers: Safari, Internet Explorer and Firefox. The application runs on three Operating systems including Windows, macOS, and Linux. The application is able to connect to the devices using three connection protocols including LAN, PPP, and ISDN. Also, the output is sent to a printer that is either local or networked or is sent to a screen by the application. For the given application, do the following: 

1. Identify the factors and levels in the given problem
2. Generate test configuration for the application using MOLS which satisfy the following constraints:
   • Safari supports only macOS
   • Internet Explorer works only on Windows
   • Firefox works on all the three OS, i.e. macOS, Windows, and Linux

Solution: 

1. Identify factors and levels

For the given problem, the factors and levels are: 

Therefore, there are a total of 4 factors and each factor has 3 levels.

2. Generate test configuration

(a) Relabel the factors

Factors have to be relabeled such that : 

Since, |F1| = |F2| = |F3| = |F4| = 3

So, the relation |F1| >= |F2| >= |F3| >= .........|Fn-1| >= |Fn| is satisfied.
Therefore, factors are labeled as: 
F1 = F1'
F2 = F2'
F3 = F3'
F4 = F4'

Also, as b = |F1| and k = |F2|, 
Therefore, b = 3 and k = 3

NOTE: |Fi| is the number of levels in factor Fi. Also, b denotes the number of blocks and k denotes the number of rows in each block.

(b) Prepare a table and fill columns F1 and F2

Let's now prepare a table containing 4 columns since there are four factors. Also, we need ( b x k ) number of rows, i.e. ( 3 x 3 ) = 9 rows divided into 3 blocks. Now, we fill Column F1 with 1s in Block 1, 2s in Block 2, and so on. Also, we fill the Column F2 using the sequence 1,2,3......k in Rows 1 through k. The table prepared is as follows: 

(c) Find MOLS of order k and fill the rest of the columns

Since k = 3,

So, MOLS of order 3 are required. 
We know that if k is a prime or power of prime then number of MOLS = k - 1
As k = 3 is prime, therefore 2 MOLS exists that are: 

    1  2  3            1  2  3
M1 =    2  3  1        M2 =    3  1  2
    3  1  2            2  3  1

We fill the remaining two columns using the columns of M1 for F3 and columns of M2 for F4: 

(d) Check if constraints are satisfied 

In the given problems, factors F1(Browser) and F2 (Operating system) are dependent on each other as:

• Safari supports only macOS
• Internet Explorer works only on Windows
• Firefox works on all the three OS, i.e. macOS, Windows, and Linux

So, an entry in the row is highlighted in yellow if it does not satisfy the constraints mentioned above: 

In the above table, there are four rows that do not satisfy the given constraints. For example, Row 1 of Block 1, F1 = 1 and F2 = 1, i.e. F1 is Safari and F2 is Windows, however, it is clearly mentioned in the first constraint that Safari only supports macOS and no other operating system. Hence, this row does not satisfy the constraints and is hence highlighted. 

(e) Remove configurations that do not satisfy given constraints

The configurations that are highlighted in yellow in the above table have to be removed and to do this, we will use a two-step procedure: 

1. Modify the highlighted rows so that the constraints are maintained
2. Add new configurations that cover the pairs that were left uncovered while replacing the highlighted rows

So, in the first step, we modify the above-highlighted rows such that these rows satisfy the given constraints. The modified table is shown below: 

For the second step, The new configurations are as follows: 

F1    F2    F3    F4    
-    1    1    1    ....(A)
-    3    3    2    ....(B)
-    2    3    1    ....(C)
-    3    1    3    ....(D)

Here, '-' denotes that F1 could take any value out of 1,2,3.

(A) and (C) is already satisfied in the above table since: 

• F2 = 1 and F3 = 1 is  occurring in Row 3 of Block 2. Also, F3 = 1 and F4 = 1 is occurring in Row 1 of Block 1. Hence, (A) is satisfied
• Similarly, F2 = 2 and F3 = 3 is occurring in Row 3 of Block 1 and F3 = 3 and F4 = 1 is occurring in Row 2 of Block 2. Hence, (C) is satisfied

However, (B) and (D) are not satisfied in the above table, so we add them to the unique configuration table. The updated table is shown below: 

(f) Replace the numbers in the columns with the given factor values

We now replace the values of columns F1, F2, F3 and F4 with their actual value i.e. levels. The final design configuration is as shown below: