STPA and Software

Verification

Sommersemester 2017

Dr. Asim
Abdulkhaleq

You can

copy, share and change,

film and photograph,

blog, live-blog and tweet

this presentation given that you attribute

it to its author and respect the rights and
licenses of its parts.
based on slides by @SMEasterbrook und @ethanwhite
 STPA Approach

Recall.

STPA Approach (Recall)

 The basic components in STPA are data lists (e.g. accidents, hazards, system goals, and design and safety requirements), data
tables (e.g. unsafe control actions, corresponding safety constraints and causal factors), and Diagrams (e.g. control structure
STPA Artefacts (Recall) diagrams and process models).
Data Tables
Unsafe Control Actions
A control structure diagram is made up of basic feedback control loops. When put together, they can be used to model the high-
Corresponding Safety
Constraints
level control structure of a particular system.
Causal Factors

Data Lists The main

Accidents components used
Hazards in STPA Diagrams
System Goals Control Structure Diagram
Design and safety Process Model
Requirements

In the STPA process, the safety analyst can derive different levels of the safety constraints. For example, the output of
STPA Data Flow (Recall) the STPA Step 1 is the safety constraints which are translated from system-level hazards. The output of the STPA Step 2
is the corresponding safety constraints which are translated from the Unsafe Control Actions. The output of the STPA
Step 2 is the new/modify safety constraints which are derived based on the causal factors.
 Software Verification

Software Verification and Validation (V&V)

Verification
Have we developed the software right?

Validation
Have we developed the right software?

Boehm, Barry (1989). "Software Risk Management".

 Software verification is a process to check whether software fully satisfies all the expected requirements. Two
fundamentals techniques to the software verification : 1) dynamic verification and 2) static verification. The dynamic
Software Verification verification aims at checking the software behaviour dynamically and it is performed during the execution of software.
Approaches to Verification
The static verification is the process to check if that the software meets its requirements by inspecting the software
Dynamic Verification (Test, experimentation) code before it runs.
Module Test Formal verification entails a mathematical proof showing that a system satisfies its desired property or specification.
Integration Test
System Test Formal verification techniques are used to prove the correctness of software and check whether the software satisfies
Acceptance Test its requirements. Three types of formal verification exist: model checking, theorem, proving and deductive methods

Static Verification (Analysis)
Code Conventions Verification
Bad Practices (anti-Pattern) Detection
Software Metrics Calculation
Formal Verification
Ghezzi, et al. Fundamentals of Software Engineering, Prentice Hall, ISBN 0-13-099183-X

Some artefacts of a software are requirements documents, design models and source code. To formally verify a
software, we need the target software to be formally modelled in the input language (SMV language or Promela, etc) of
Formal Verification a formal verification technique e.g. model checker. and the software requirements to be formalised in a formal
What we need to formally verify a software?
specification e.g. Temporal logic.
Software

Control Actions? Feedback?

Autonomous vehicle

Software Software Design

Software Code
Requirements Models

written in natural modelled in graphical written in high-level

language e.g. English notations e.g. State- programming language
machine, stateflow. e.g. C, C++ or Java

Formalisation Transformation
 The property (requirement) of interest must be modeled in a mathematical structure (e.g. temporal logic). Temporal Logic has been
proposed by Pnueli as an appropriate formalism in the specification and verification of concurrent programs. Many different
Formal Specification and Verification versions of temporal logic have been used in the verification process such as Linear-Time Temporal Logic (LTL), and Computation
Specify and verify properties about how a system changes over time
Tree Logic (CTL) which have been broadly used to express safety properties in a formal notation.
Temporal Logics (TL)
is a convenient formalism technique for specifying and verifying properties
of reactive systems

Linear Temporal Logic (LTL)

is a mathematical language for description linear-time properties

Properties of interest include :

- Safety : Nothing bad will happen
- Liveness: Something good will happen.
- Fairness : Independent subsystems make progress

An LTL formula consists of atomic propositions, Boolean operators and temporal operators (next, always/Globally, eventually, U
until, R release).
Formal Specification and Verification
An LTL consists of:
- atomic proposition symbols [p, r, q,..], Boolean operators , and temporal
operators:

TL Operator LTL Operator Meaning Diagram

G means globally/always. is true now
and forever in the future
X means at the next time. is true in
the next state
F means eventually. is true in some
future state
means releases. is true until and
R including the point where become
true. If never becomes true, then
must remain true forever
means until. is at least true until
U happens at the current or a future
position
Exercise: Formal Specification
Which LTL formula specifies the following requirement
The train door software must not close (reopen) the
train door when there is a person in the doorway?
1. LTL= G ((Person_In_Door_Way) -> G (ControlAction=Close))

2. LTL = G (!(Person_In_Door_Way) -> (ControlAction=Close) )

3. LTL = G ((Person_In_Door_Way) -> X ! (ControlAction=Close))

4. LTL = G ((Person_In_Door_Way) -> X! (ControlAction=Open))

3 Minutes

Software verification is a process to check whether software fully satisfies all the expected requirements. Formal
verification and testing are two fundamental approaches to software verification. Formal verification techniques are
Software Formal Verification Process used to prove the correctness of software and check whether the software satisfies its requirements. Three types of
How to verify my software design model/code?
formal verification exist: model checking, theorem proving and deductive methods
Software Software Input
Design/Code Requirements

check
Errors
Track Software
Verification
No Yes

Not Satisfied No Satisfied Output

Results

Software Model Checking
A technique to automatically verify whether the software meets its requirements.
Model checking is a well-established formal verification technique to verify whether the software meets its requirements through
exhaustive exploration of the state space of the software. The model checking process involves that the target software to be
formally modelled in the input language of a model checker and specifications/properties to be formalised in a temporal logic such
as Linear Temporal Logic (LTL).

There exist two ways to construct or extract the input model required by software model checkers: (1) at the design
Software Model Checking level, a verification model can be constructed from state machine diagram (e.g. SMV model ) and (2) at the implementation level, a
A technique to automatically verify whether the software meets its requirements. verification model can be extracted directly from software code (e.g. Promela/SPIN model ).
At the design level
The verification model can be constructed/extracted from the software
behaviour models such as finite state machine.
Model checkers can be used at this level are:
1. SMV (Symbolic model Verifier) model checker
2. NuSMV Model checker
3.
At the implementation level
The verification model can be constructed/extracted directly from
software code written in C or Java.
Software model checkers can be used at this level are:
1. SPIN (Simple Promela Interpreter)
2.
 The SMV language allows to describe the finite state systems. It requires that the given system to be verified is modeled in a
suitable model/diagram (e.g. Finite state machines). This model of a given system should be written in the specification language
SMV Language called SMV and the given property (requirement) should be expressed in CTL to check the validity of the model against its
Structure of the SMV model
specifications. The output of the SMV model checker are Boolean values: true or false with a counterexample which shows why the
formula does not validate.

The basic structure of the SMV module contains the following sections: 1) The name of the model with the optional state variable
parameters, 2) The declaration of the state variable and their possible values,
3) The initial values of variables and the states variable, 4) The sub-modules of the super module deceleration, 5) The transitions of
the module, and a list of the LTL formulae or both.

To verify the software design against its requirements, software controllers behaviour must be modelled in a suitable behavioural
model (e.g. stateflow model). The Stateflow diagram is a visual notation for describing dynamic
SMV Language Example behaviour, including the hierarchy, concurrency and communication information.
Train door controller
MODULE main
VAR
states: {OpenDoor , CloseDoor}; In this example, we modelled the software behaviour of the train controller in Simulink/stateflow and transform it into the SMV model
Person_In_Door_Way: boolean;
ControlAction:{Open, Close}; as an input to model checker e.g. NuSMV.
ASSIGN
init (states):=OpenDoor;
init(Person_In_Door_Way):=bool(0);

next (states):=case
states=OpenDoor & !(Person_In_Door_Way) : CloseDoor;
states=CloseDoor& (Person_In_Door_Way): OpenDoor;
TRUE: {CloseDoor ,OpenDoor};
esac;

next(ControlAction):=case
states=OpenDoor: Open;
states=OpenDoor & !(Person_In_Door_Way) : Close;
states=CloseDoor& (Person_In_Door_Way): Open;
TRUE:{Open, Close};
esac;

Software behavioural model (stateflow) SMV Model (verification model)

 To verify the SMV model against the the software requirements which are specified in LTL, we can use the NuSMV model checker.

NuSMV Model Checker

It is an open source tool developed by ITC-IRST and

UniTN 1999
It is an extension and re-implementation of SMV
model checker.
It based on Binary Decision Diagram (BDD) to verify
finite state machine against its specifications
expressed in LTL

Only the LTL formula no. 3 is ture. The other formulae are not satisfied and counterexamples are generated.
Exercise: Formal Specification -- specification G (Person_In_Door_Way -> X !(ControlAction = Close)) is true

Which LTL formula is satisfied by the SMV model of

the train door controller ?
1. LTL= G ((Person_In_Door_Way) -> G (ControlAction=Close))

2. LTL = G (!(Person_In_Door_Way) -> (ControlAction=Close) )

3. LTL = G ((Person_In_Door_Way) -> X ! (ControlAction=Close))

4. LTL = G ((Person_In_Door_Way) -> X ! (ControlAction=Open))

3 Minutes
 In case that the SMV model does not satisfy a given LTL of a software requirement, the NuSMV model checker will return a
counterexample. A counterexample contains information that shows why the given LTL formula of a software requirement is not
NuSMV Model Checker Result satisfied. The LTL G (Person_In_Door_Way -> G (ControlAction=Close) is false (not satisfied) and a counterexample is generated.
Counterexample
it contains information that shows why the given LTL is not satisfied As you can see in line 14 Loop starts here, that means the model return infinite counter-example that starting point of the loop.
1. -- specification G (Person_In_Door_Way -> G
2. ControlAction = Close) is false
3. -- as demonstrated by the following execution sequence
4. Trace Description: LTL Counterexample
5. Trace Type: Counterexample
6. -> State: 1.1 <-
7.
states = OpenDoor
8.
9. Person_In_Door_Way = FALSE
10. ControlAction = Open
11 -> State: 1.2 <-
12. states = CloseDoor
13. Person_In_Door_Way = TRUE
14. -- Loop starts here
15. -> State: 1.3 <-
16. states = OpenDoor
17. Person_In_Door_Way = FALSE
18.
-> State: 1.4 <-
19.
20.
states = CloseDoor
21. -> State: 1.5 <-
22. states = OpenDoor

Formal Verification Challenges
Formal verification focuses on proving the functional
The main limitation of model checking is the state explosion problem which makes it impossible to scale up and it might be
applicable only to finite state programs [AK86]. In some cases, the model checker cannot reach to state in the model and cannot
provide an answer. Therefore, the model checker alone is not sufficient to assess the safety of software. The use of formal
verification methods can reduce the likelihood of certain software errors or help to detect them, therefore the formal verification
methods must complement with appropriate testing.

correctness of software.
It can not make the software safe or reduce the risk.
It needs expert users
The requirements can not be directly verified by
software model checker

State explosion problem

 STPA in Software
Verification

If we apply STPA to the software controller in order to identify the unsafe control actions and develop the safety requirements of the
software controller. To recognise the software risks, we need to check if the software satisfies its safety requirements which are
STPA in Software Verification derived during the safety analysis process. However, there are some challenges on verifying the STPA software safety
How to verify software against STPA results?
requirements:
1. Safety requirements are wrist in natural language
STPA Safety STPA Results 2. Difficult to transform the safety requirements manually into formal specification
Requirements
satisfied
3. It is requires a model to visualise information derived during an STPA safety analysis
not satisfied Safety constraints

Software

Control Actions? Feedback?

Software Verification
Train Door
(Formal Verification & Testing)

STPA SwISs: A System-Theoretic Safety Engineering Approach for
Software-Intensive Systems
Developing safety-critical software requires a more systematic software and safety engineering process that enables the software
and safety engineers to recognise the potential software risks. For this purpose, Abdulkhaleq, Wagner and Leveson proposed a
system-theoretic safety engineering approach based on STPA including software verification activities called STPA SwISs (STPA-
based Approach for Software-Intensive Systems).

1. STPA Swiss provides seamless safety and verification activities

2.STPA SwISs provides an algorithm to automatically transform the software safety requirements into formal specifications in Linear
Temporal Logic (LTL) to facilities the
verification activities of software design and implementation with the model checker against the STPA results.
3. STPA SwISs provides a concept of automatically generating safety-based test cases from information derived during STPA safety
analysis to test the software system against the STPA results.

The STPA SwISs approach defines the following roles:

1. Safety analyst will apply STPA and derive the software safety requirements at the system level.
2. System analyst and designer will model the system specification and model the results of STPA in a suitable model (e.g. Simulink
Stateflow).
3, Safety tester will verify the constructed model against the STPA-generated software safety requirements and generate the safety-
based test cases.

The STPA SwISs approach is carried out in four major steps:

1) deriving the software safety requirements of a software controller at the system level, automatically generating the unsafe
Detailed STPA SwISs Approach scenarios based on the extended approach to STPA by Thomas, and automatically expressing the STPA-generated safety
requirements in formal specifications in LTL;
2) constructing the safe behavioral model of the software controller with the statechart notations in Simulink. A safe behavioral
model is a statechart notation that models the process model variables of a software controller in the STPA control structure
diagram as states and the control actions as the state actions, and it is constrained
by the STPA-generated software safety requirements (transitions);
3) transforming the safe behavioral model into an input model of the NuSMV model checker and checking the correctness of the
generated model against the STPA and safety requirements expressed in LTL;
and 4) automatically generating a safety-based test model and deriving the safety-based test cases from this model.
Exercise: Apply STPA to Train Door System
Disuse with your colleague how to apply STPA to the
train door system and do the following:
1. Write one accident that the software controller of train door
can contribute in.
2. Write 3 hazards which maybe cause this accident.

3. Translate the identified hazards into the safety constraints.

4. Draw the control structure diagram of the train door system.

5. Identify one unsafe control action and translate it into a

corresponding safety constraint.

10 Minutes

This step starts by applying STPA to the system specification to identify STPA software safety requirements and the potentially
unsafe scenarios which the software can contribute to. The algorithm starts by establishing the fundamentals of analysis by
Deriving the STPA Software Safety Requirements
determining the system-level accidents (ACC ) and the associated system-level hazards (HA ) which the software can lead to or
Accidents contribute in. Next, the algorithm demands that the safety control structure diagram of the system shall be constructed from the
AC1- A person is injured
system specifications. The software here is the controller in the control structure diagram.
while the train closed the
door.
.
The control structure diagram of the train door system includes: 1) software door controller; 2) door actuator; 3) physical door; 4)
Hazards
H1- Door close on a person and door sensor.
in the doorway.

System-level safety constraint

The train door controller
must not close the door safety control structure diagram of train door system
while a person is in the
doorway

 For each software controller component in the control diagram, its software safety requirements can be derived by performing the
following steps:
Unsafe Control Actions
Control Action 1)Identify all safety-critical control actions that can lead to one or more of the associated hazards (e.g. close, open, stop, )
C1- Close
2)Evaluate each control action with four general types of hazardous (not provided, provided incorrect, wrong time or order, stooped
.
Unsafe Control Actions too soon or applied too long)
UCA1.1- Train door controller
closes the door while a person in
3)Translate each unsafe control action into the corresponding safety constraint
the doorway

Corresponding Safety Constraints

CSC 1.1- Train door controller
must not close the door while a
person or object is in the
doorway

4) Identify the process model and its variables and include them in the software controller in the control structure diagram to
understand how each unsafe control action could occur.
Process Model & Variables
For example, the process model variables that have an effect on the safety of the providing or not providing the control action close
door are : door state, train position, personInDoor and train status.

Generating the Unsafe Scenarios & Requirements
Unsafe Control action
UCA1.1- Train door controller closes the door while a person in the doorway
Basic Scenario
RUCA1.1- The door software controller provided the control action close door while the train is
stopped, train position is aligned, door state is open and a person is in the doorway.
Refined software safety constraint
RSC1.1- The door software controller must not provide the control action close door while the train
is stopped, train position is aligned, door state is open and a person is in the doorway.
Based on the results of the pervious steps, we can identify the unsafe software scenarios for each unsafe control action and refine
its software safety constraint as follows:
1) Identify the potential unsafe critical combinations of unsafe control actions and evaluate it to identify the potential unsafe
scenarios of the software controller that cause accidents (e.g. UCA 1.1). We use the combinatorial testing algorithm to
automatically generate the combination sets between the process model variables of each control action. For example, the control
action close has 4 process model variables: door state (2 values), train position (2 values), person in door way (2 values) and
train status (2 values). The total number of the combinations (all combinations) is 2 x 2 x 2 x 2 =16 combination sets.
2) Generate the unsafe scenarios based on the critical combinations of each control acton. For example, the unsafe scenario RUCA
1.1: The door software controller provided the control action close door while the train is stopped, train position is aligned, door
state is open and a person is in the doorway.
3)Translate each unsafe scenario into a software safety constraint.

The causal factors and causal scenarios that lead to the unsafe control actions are identified by analysing the control loops in the
control structures diagram of train door system.
Causal Factors & Causal Scenarios
How each unsafe control action could occur in the system?
 Formalisation of STPA
Results

Formalisation of STPA Results

Providing or not providing a control action (CA) is based on the occurrence of
the set of values of process model variables and higher inputs (CS).

Rule 1: When CS occur in the execution path, the software must not (!) provide CA.
Then, LTL formula can be expressed as: LTL = G ( CS ! CA).
Rule 2: When CS occur in the execution path, the software must provide CA at the
next step. then LTL formula can be expressed as:LTL = G ( CS X CA)

Rule 3: The software must always not (!) provide CA too early (U) before CS the
occurrence of CS still not become true. Then, the LTL formula can be expressed as:
LTL = G ((CA CS) & (!CA U CS))

Rule 4: The software must always not (!) provide CA too late (U) while the occurrences
of the critical set of combinations CS has become previously true in the execution
path. Then, the LTL formula can be expressed as: LTL = G ((CS CA) & (!CS U
CA))
 1. LTL1:1= G (((trainstatus= stop) && (doorstate = close) && (trainposition=Aligned) && (PersonIndoorway= TRUE)) -> !
Exercise: Formalisation of STPA Results (controlAction= close door)).
1. LTL 1.2 = G (PersonIndoorway =TRUE) -> X (ControlAction=open door)
Write the corresponding LTL formula for the following
software safety requirements:
1. The train door software controller must not
provide the control action close door while the
train is stopped, train position is aligned, door
state is open and a person is in the doorway
2. The train door software controller must provide
the control action open door when there is a
person in doorway.

5 Minutes

Verify STPA Results

within XSTAMPP
XSTAMPP
www.xstampp.de

It is an open source tool developed at our institute

through student project and job (2013-2017).
Designed specially to serve the widespread adoption
and use of STAMP in different areas.
It extended with an plugin called STPA verifier to
support the software safety and verification (formal
verification and testing activities).

STPA Verifier is an Eclipse plug-in developed to verify the STPA safety requirements with model checking tools such as SPIN and
NuSMV. The STPA generated safety requirements are automatically transformed into formal specification in LTL (linear Temporal
STPA Verifier Process Logic).
www.xstampp.de
STPA Verifier Main Window

Generating Safety-
based Test Case from
STPA Results
 The last step of the STPA SwISs is generating the safety-based test cases from the STPA results.

Detailed STPA SwISs Approach (Recall)

The algorithm has the following steps:

Safety-based Test Case Generation Algorithm 1. Modelling the output of the STPA (e.g. process model and safety constraints, control actions) into a suitable software behavioural
Algorithm LTL formulae STPA
Results model (e.g. state flow notation) called safe behavioural model.
1 Modelling STPA Results model 2. Transforming the safe behavioural model into formal model (e.g. SMV model).
Safe Behavioral 3.Check the correctness of the formal model (e.g. SMV model) with model checker against the STPA safety requirements which are
2 Transforming into a Formal Model Model
transform specified in LTL.
3 Checking Correctness with Model Checker
SMV Model modify 4.Automatically converting the safe behavioural model of the software controller into the input model of the safety-based testing tool
check
not satisfied e.g. STPA TCGenerator.
Verify
4 Generating Runnable Safe Test Model
?
satisfied transform
Traceability
matrix
5.Using the safety based testing tool STPA TCGenerator to generate the safety-based test cases for each STPA safety requirement.
Safe Test Model
5 Generating Safety-based Test Cases
traverse
Safety-based Test
Cases
export

Test case sheet


Modelling STPA Results
A Safe Software behavioural Model
Mapping STPA Results into Stateflow Model
To generate the safety-based test cases, the information derived from the STPA safety analysis must be integrated into a suitable
model which should visualise the process
model variables of each software controller and their relations in a control structure diagram. For example, the Stateflow diagram
notations are used to visualise the automation model of each software controller. The Stateflow diagram is a visual notation for
describing dynamic behaviour, including the hierarchy. concurrency and communication information. The idea here is to build a
model from STPA results with a modelling editor (e.g. Simulink) that supports the export of the statechart notations as XML
specifications.
We identify the rules of constructing a safe software behavioural model from the STPA results as follows:
1. The process model of each software controller in the STPA control structure diagram will be visualised by a chart in the Stateflow
model.
2. The safe behavioural model should contain all internal states S and the safety-critical process variables P of the software
controller in the STPA control structure diagram.
3. All process model variables of the software controller in the STPA control structure diagram should be declared in the safe
behavioural model.
4. The safe behavioural model should constrain the transitions using the STPA software safety requirements (constraints) which are
identified based on the corresponding LTL rules 3
and 4.
5. Define an enumeration data type variable named controlAction in the safe behavioural model which takes all control actions of
the software controller in the STPA control structure diagram as its value.
6. The controlAction variable will be used as an entry action of internal states of the safe behavioral model to show which control
action will be issued when the software controller enters a state

Safe behavioural model of train door controller
For example, the Stateflow model (safe behavioural model) of the door software controller has one chart which contains one
superstate door state. The train door software controller contains one internal
state doorstate which has two substates close and open. The three process model variables are declared as follows: Trainposition
(Enum ), Trainstatus (Enum) and the personIndoorway (Boolean). The train door software controller must provide the control action
open door when the train is stopped and aligned with platform. We used this information derived during STPA analysis process to
constrain the default transition of the state open. The control actions of the train door software controller are open door, close door
and stop are defined as as enumeration data type in the safe behavioural model.

Control structure diagram Safe behavioural model of

with process model train door controller

STPA TCGenerator is a stand-alone tool written in Java based on the NetBeans platform. STPA TCGenerator is developed to
STPA Test Cases Generator generate the safety-based test cases directly from STPA safety analysis results. STPA TCGenereator parses the STPA file project
created in XSTAMPP and the safe behavioural model which is created with Simulinks Stateflow editor to automatically generate the
SMV model and check the correctness of the safe behavioral model, eliminate the safe test model and generate safety-based test
cases
References

ISBN: 9780262026499 ISBN-10: 3736994923

ISBN-13: 978-3736994928

Dr. Asim Abdulkhaleq

e-mail asim.abdulkhaleq@informatik.uni-
stuttgart.de
phone +49 (0) 711 685-88458
WWW www.iste.uni-stuttgart.de/se
Twitter @AbdulkhaleqAsim

Institute of Software Technology

Pictures Used in this Slide Deck

V-Model https://commons.wikimedia.org/wiki/File:V-model-en.png
Model checker: Advanced topics in Combinatorial Methods for Testing by Rick
Kuhn http://slideplayer.com/slide/4058348/
NuSMV logo: http://nusmv.fbk.eu
Train door photo is taken from http://www.bbc.com/news/uk-england-
tyne-24634884
Principle of model checker photo : https://mitpress.mit.edu/books/principles-
model-checking