Testing Framework

with Simulink Test

김종헌 부장

© 2015 The MathWorks, Inc.

1
2
 Test Harness
(Test Scenario)

Target algorithm
for verification

Test Harness (Plant model)

3
How to Test Your Model…?
Production model Do you
need to
modify it for
testing?

4
Building Test Harness Model using Model Reference

▪ Separated model not for code generation but only for testing
Test harness model
Production model

Signal Builder
Model Reference (Model block) Scope

5
Simulink Test

6
Why Simulink Test?

Saves you time:

▪ Creating / managing test infrastructure

▪ Generating & (re)-running multiple tests
▪ Reporting results
▪ Easy integration with other tools
(Requirements, Coverage, Test Generation, MATLAB Unit Test, Continuous Integration)
▪ A common test environment
– everyone doing things in a consistent manner

7
 Simulink Test Overview
1. Test Harnesses 2. Test Stimulus Integration 3. Test Manager

• Synchronized, simulatable test • Inputs and assessments based on logical, • Author, execute, manage test cases
environment temporal conditions • Review, export, report

Main Model

Component
under test

Test Harness
8
Agenda

▪ Creating Test Harnesses

▪ Creating Test Cases & Test Stimuli

▪ Testing against Requirements

▪ Reporting

▪ Coverage analysis

9
Creating Test Harness

10
What if you already have a harness model....

11
Agenda

▪ Creating Test Harnesses

▪ Creating Test Cases & Test Stimuli

▪ Testing against Requirements

▪ Reporting

▪ Coverage analysis

12
 Example 1: Create a test case
using the original signal builder

13
Create test cases with Signal Builder

14
What have we done so far....

▪ Created and imported test harnesses

▪ Created a test case for running multiple simulations (iterations) with

different scenarios

15
 Common questions...

When should I use iterations vs multiple test cases?

16
Comparison

▪ Use iterations if:

– Only changing parameters, inputs, or configuration settings
– Same model/harness & test type
– Same set-up (callbacks)
– Usually run together
– Relate to same requirements(s)
– Can use fast-restart

▪ Use separate test cases if:

– Need independent configuration control
– Different model/harness/test type or callbacks
– Relate to distinct requirements
– Distinct control of coverage
17
Example 2: Create a test case
using real-world recorded data

18
My data

19
Importing time-stamped data from Excel or text files

% pre-process .xlsx file

% get import options
importOptions = detectImportOptions('SiteWindDataRecorded.xlsx');
% set sheet
importOptions.Sheet = '2011_05_23';
% tell it that Time is in a date-time format
importOptions = setvartype(importOptions,'Time','datetime');
importOptions = setvaropts(importOptions,'Time', 'DatetimeFormat', 'HH:mm:ss.SSS');
% read data in
T = readtable('SiteWindDataRecorded.xlsx',importOptions);
% convert to timetable
TT = table2timetable(T);
% re-sample to 1sec intervals
TTT = retime(TT,'secondly','nearest'); Time WindSpeed WindDirection
0 14.59 214.9
1 15.25 218.2
2 16.46 212.2
3 16.08 207.3 20
21
What have we done so far....

▪ Created and imported test harnesses

▪ Created a test case for multiple simulations (iterations)

▪ Created a test case importing real-world data from Excel using root import
mapping

22
Agenda

▪ Creating Test Harnesses

▪ Creating Test Cases & Test Stimuli

▪ Testing against Requirements

▪ Reporting

▪ Coverage analysis

23
 Requirement based testing
Requirements

Input Scenarios Implementation Dynamic Testing

Baseline MATLAB Unit Test Assertions Test Sequence

and more!

24
 Requirements Editor in Simulink Requirements
Manage and Organize Requirements

Organize with
Requirement
Sets Import from
External Sources View and
Author

25
Requirements Perspective in Simulink Requirements

26
Track Implementation and Verification

27
 Example 1:
Baseline test

28
Test types in Test Manager

▪ Baseline Test Ex) Regression test

Ex) Back-to-Back test

▪ Equivalence Test like SIL, PIL

▪ Simulation Test Ex) Verifying algorithm

with logical criteria

29
▪ Challenges
– Not easy to predict expected result
– Hard to make time-series input data

▪ Solution
– Use data captured from simulation as baseline
1. Try to run a simulation for each case.
2. Capture output data from simulation result.
3. Review captured data to confirm whether it is valid as baseline.
4. Apply reviewed data to Test Manager as baseline

30
Baseline test using captured simulation result

31
 Example 2:
Using verify() to test against a requirement

32
 Test Sequence Block
Simulink Test

▪ A test sequence block can

– Drive inputs (considering feedback)
– Assess outputs with verify keyword

x
Define Inputs

33
34
C Caller Block Support

Verify model and hand code together

▪ C Caller block allows you to call a C function

directly from a model Handwritten C Code

▪ Test the C function by creating a test

harness for the C Caller block

▪ Author, manage and execute tests of the C

function with Simulink Test

Test Harness
» 14:30~150:00 Simulink를 통한 효율적인 레거시 코드 검증 방안 소개
35
Agenda

▪ Creating Test Harnesses

▪ Creating Test Cases & Test Stimuli

▪ Testing against Requirements

▪ Reporting

▪ Coverage analysis

36
Test Manager
Simulink Coverage
Simulink Design Verifier

37
Summary

▪ Benefits of Simulink Test

– Ease of creation, organisation & control of test harnesses

– Ease of driving your models with data from various sources

– Ease of in-harness/model verification of requirements

– Ease of reporting

– Ease of integration: requirements, coverage

38
39