ModelJUnit Tutorial

COM4506/6506 November 29, 2012

Model-based Testing

Model-based testing is a functional testing technique, where we create a model of the functionality we are testing, and then use that to automatically exercise the system under test (SUT). By comparing the behaviour described by the model and the SUT failures can be identied. In the lecture we already saw that for complex systems with dierent states and sequences of actions we can use nite state machines as models. Testing a system based on a nite state machine model amounts to exploring paths through the model and executing corresponding actions on the SUT, and along the way checking that the system behaviour matches the specied behaviour.

ModelJUnit

ModelJUnit is a Java library that extends JUnit to support model-based testing. Models are extended nite state machines that are written in Java. The second assignment comes as an Eclipse project with ModelJUnit integrated; if you want to download the latest version of ModelJUnit you can nd it on the ocial webpage: http://www.cs.waikato.ac.nz/~marku/mbt/modeljunit/ The link to the JavaDocs on the ocial webpage is broken, but there is a link to a zip le containing the JavaDocs and a set of example models illustrating what can be done with ModelJUnit. In ModelJUnit, an FSM model is a class implementing the FsmModel interface. This interface denes two methods: getState: When exploring a model, ModelJUnit uses this method to determine if it has already seen a state before. The method returns an Object, and ModelJUnit uses Object.equals to compare states. Typically, this function returns a String identifying states. reset: During model exploration, ModelJUnit will try to reset the model to its initial state. This method must ensure that all variables are set back

to their initial state, and the system under test also needs to be reset to its initial state. The model is implemented in terms of actions, which are public void methods annotated with @Action. Every action can have a guard condition, which is a public method returning a boolean value, and is named like the action with Guard appended (e.g. action action would have guard condition actionGuard). A model is explored using one of the Tester implementations provided by ModelJUnit. As extended nite state machines typically have large state spaces due to the use of variables systematic approaches to cover states are not applicable, and the standard procedure is to use a random or heuristic exploration of the model. For example, the RandomTester explores a model in terms of the generate(x) method, which performs x random transitions on the model. For example, for x = 10 the tester would randomly select and execute one action for which the guard condition evaluates to true ten times in sequence. Tester tester = new RandomTester(new YourModel()); tester.generate(10); To get an idea how much of the model has been covered, ModelJUnit provides coverage listeners that report the achieved coverage. For example, to get information about the achieved transition coverage one could use the following sequence of statements: TransitionCoverage coverage = new TransitionCoverage(); tester.addListener(coverage); tester.generate(10); System.out.println("Transition coverage: "+coverage); To ensure that the available states and transitions are known, it is recommended to call tester.buildGraph(); before running generate. To test a system using ModelJUnit, the actions in the model perform the corresponding actions on the system under test, and check the resulting behaviour using standard JUnit assertions. By default, ModelJUnit will just report failures by printing messages to stdout; to change this behaviour in order to stop the model execution when a failure occurs one has to add the following statement to the tester before the exploration: tester.addListener(new StopOnFailureListener()); By default, ModelJUnit does not provide any output during the exploration. ModelJUnit can be instructed to provide information about the transitions taken by adding the command tester.addListener("verbose"); before starting the exploration. In addition, while developing a test model it is often useful to add more output to both the SUT and the model using System.out.println commands.

Exercise

On the MOLE page of the course you will nd a link to ModelExamples.zip, which is an archive that contains an Eclipse project consisting of ModelJUnit and two examples.

3.1

Turnstile

The class TurnstileModel represents the extended nite state machine of a turnstile as used in the lecture on specifying systems (see lecture slides for more details). The class TurnstileTest contains a JUnit test that performs random exploration of this model. The directory also contains the class TurnstileImpl which is an implementation of this model.1 Your task is to extend the turnstile model with calls to the implementation and to check whether the observed behaviour is correct. This means that in the actions in the model you will need to add calls to sut.push() and sut.insertCoin(), and also add JUnit assertions checking whether sut.isLocked() has the expected value. Dont forget to reset the SUT in the reset method, for example by instantiating a new TurnstileImpl. To verify whether your model is exercising the implementation, you can use EclEmma, or manually change values in the implementation to see if the test cases can detect these faults.

3.2

Safety Injection System

The safety injection system is a controller from a nuclear system monitoring the pressure of the coolant. If the pressure is too high or too low, then appropriate actions have to be taken. The class SISModel again contains a nite state machine modelling the behaviour of the implementation, which is given in class SISImpl. One important dierence to the rst example is that the safety injection system has guarded transitions based on an input variable, rather than just actions as in the turnstile example; i.e. transitions depend on the current pressure value. To simulate changes in the pressure, our test model therefore includes an action that changes the value of the pressure by a random value (@Action public void pressureChanged()). Again there is a JUnit test case exploring the model in SISTest, and your task is to add calls to the SUT to exercise and to check its behaviour. The SUT is informed about changes in pressure using the sut.update method, which takes the current pressure value as parameter. The state of the implementation can be veried using the sut.isUnderPressure() and sut.isOverPressure() methods.

1 At this level of complexity, model and implementation are more or less the same; in real applications the model would represent a much simpler abstraction of the actual implementation.