Sie sind auf Seite 1von 23

AUTONOME

VALIDATION OF HIGHLY
AUTOMATED AND
AUTOMOBILSYSTEME –
AUTONOMOUS AUTOMOBILE
GROßE
SYSTEMS
HERAUSFORDERUNGEN!
AUTOMOTIVE SAFETY AND SECURITY WEEK
TESTING ADAS AND SELF-DRIVING CARS
FRANKFURT OBERURSEL
27. – 30. MARCH 2017
Autonomous Automobile Systems
Overview
 Motivation for highly
automated and
autonomous Vehicles

 V-Modell and ISO 26262

 Verification and Validation

 Confidence on decisions of
cognitive autonomous Systems

 Comparison to other Industries

 Future Prospects – Look-Out

2 Bosch Engineering | Valisure - BEG/EPV | xx.03.2017


© Bosch Engineering GmbH 2016. Alle Rechte vorbehalten, auch bzgl. jeder Verfügung, Verwertung, Reproduktion, Bearbeitung, Weitergabe sowie für den Fall von Schutzrechtsanmeldungen.
Autonomous Automobile Systems
More comfort for drivers and passengers
 Are you used to work as driver?
Life is just to short, to spent it as driver.

3 Bosch Engineering | Valisure - BEG/EPV | xx.03.2017


© Bosch Engineering GmbH 2016. Alle Rechte vorbehalten, auch bzgl. jeder Verfügung, Verwertung, Reproduktion, Bearbeitung, Weitergabe sowie für den Fall von Schutzrechtsanmeldungen.
Autonomous Automobile Systems
The daily economical damage of the traffic jam!
Daily millions of hours wasted in traffic jams …

4 Bosch Engineering | Valisure - BEG/EPV | xx.03.2017


© Bosch Engineering GmbH 2016. Alle Rechte vorbehalten, auch bzgl. jeder Verfügung, Verwertung, Reproduktion, Bearbeitung, Weitergabe sowie für den Fall von Schutzrechtsanmeldungen.
Autonomous Automobile Systems
Accidents – Casualities – Fatalities
 Development of accidents
in Germany
from 1970 to 2014

 Increasing vehicle
mileage 193%

 Reduction of
fatalities 84%

 Goal:
Significant reduction
of (serious) accidents
1970 1975 1980 1985 1990 1995 2000 2005 2010

5 Bosch Engineering | Valisure - BEG/EPV | xx.03.2017


© Bosch Engineering GmbH 2016. Alle Rechte vorbehalten, auch bzgl. jeder Verfügung, Verwertung, Reproduktion, Bearbeitung, Weitergabe sowie für den Fall von Schutzrechtsanmeldungen.
Autonomous Automobile Systems
(Double-)V-Model according to ISO 26262:2011
 V-model acc. ISO 26262:2011 Requirements Phases Design Phases Test Phases

 Shown are the requirements phase 4-9


3-7 3-5
and the design phase on the left Safety
goals
Item
definition
System
safety
validation
hand side of the V-model
3-8 3-8 4-8
 Test phases and test cases are Functional
safety
Preliminary
architectural
Vehicle
Integration

shown on the right hand side of the requirements assumptions testing

V-model 4-6
Technical
4-7
4-8
System
System
safety integration
design
requirements testing

5-6 / 6-6 HW- / SW-


5-7 / 6-7
Safety integration
HW- / SW-
requirements testing
design
HW / SW

Detailed HW- / SW-


HW- / SW-
HW / SW unit
detailed
Safety testing
design
requirements

6 Bosch Engineering | Valisure - BEG/EPV | xx.03.2017


© Bosch Engineering GmbH 2016. Alle Rechte vorbehalten, auch bzgl. jeder Verfügung, Verwertung, Reproduktion, Bearbeitung, Weitergabe sowie für den Fall von Schutzrechtsanmeldungen.
Autonomous Automobile Systems
(Double-)V-Model according to ISO 26262:2011
V-model acc. to ISO 26262:2011 with the requirements phase and the
design phase on the left hand side of the V-model - Test phases and
test cases on the right hand side of the V-model

 A) The requirments phase covers a complete set of all safety requirements on all hierarchical
levels. This is achieved by a deductive derivation of safety requirements (Top-Down).

 B) Design phases realise the safety requirements on the corresponding hierarchical level of the
desing.

 C) Test phases probe the correct realisation of the safety requirements for the different
hierarchical levels within the step-wise integration.

7 Bosch Engineering | Valisure - BEG/EPV | xx.03.2017


© Bosch Engineering GmbH 2016. Alle Rechte vorbehalten, auch bzgl. jeder Verfügung, Verwertung, Reproduktion, Bearbeitung, Weitergabe sowie für den Fall von Schutzrechtsanmeldungen.
Autonomous Automobile Systems
(Double-)V-Model according to ISO 26262:2011
Basic assumptions for the V-model:

 Systems can be derived completly in a deductive-deterministic


manner (a situation or stimuli X, results in a system reaction Y).

 Safety requirements are to be completely and atomically described


(complete set of safety requirements).

 The test cases are also derived completely from the safety requirements
(complete set of test cases).

 Result: After all test cases are passed, the functional safety of the item /
system is achieved.

8 Bosch Engineering | Valisure - BEG/EPV | xx.03.2017


© Bosch Engineering GmbH 2016. Alle Rechte vorbehalten, auch bzgl. jeder Verfügung, Verwertung, Reproduktion, Bearbeitung, Weitergabe sowie für den Fall von Schutzrechtsanmeldungen.
Autonomous Automobile Systems
Set of Problems for Autonomous Systems
 The driving situations to be controlled are manifold and highly complex and for this not
completely describable in models. This results in an incomplete deductive-deterministic
system specification. Further it may result in a funktional inadequacy, this means the described
system specification is fulfilled, but not 100% of the driving situations are controlled by the system.

 Driving situations require not „the one and single reaction“, multiple different safe reactions are
possible.

 Corrections or (development) enhancements of the autonomous system may result (on the
base of the system-complexity) in a decrease of decision quality and degradation in certain
driving situations.

9 Bosch Engineering | Valisure - BEG/EPV | xx.03.2017


© Bosch Engineering GmbH 2016. Alle Rechte vorbehalten, auch bzgl. jeder Verfügung, Verwertung, Reproduktion, Bearbeitung, Weitergabe sowie für den Fall von Schutzrechtsanmeldungen.
Autonomous Automobile Systems
Consequences for Autonomous Systems
 Safety requirements require itself partly an abstract discription (of the
requirement) (this is only valid for software with decision competence for highly
complex systems).

 Every single abstractly described safety requirement offers a variety of solutions


in a wide solution space. For this there are inductive-iterative solution
processes required to approach to an optimum.

 It is up to the development departments to identify in this solution space the best


possible set of (sensor data) attributes to be used for the system decision. These
attributes have to be weighted (according to their relevance) and to be given to the
decision algorithm (which is also to be developed).

 Possibly there are self-learning algorithms and artificial intelligence required to


do this.
10 Bosch Engineering | Valisure - BEG/EPV | xx.03.2017
© Bosch Engineering GmbH 2016. Alle Rechte vorbehalten, auch bzgl. jeder Verfügung, Verwertung, Reproduktion, Bearbeitung, Weitergabe sowie für den Fall von Schutzrechtsanmeldungen.
Autonomous Automobile Systems
Verification and Validation of Trained Systems
 Not all possible driving situations and scenarios can be tested
completely. This is because of the unlimited amount of different driving
situations.

 For this it is required to define a representative subset (equivalence class),


which is to be used for the verification and validation .

 The crucial point and difficulty is to find this representative subset!

 This representative subset will be (in general) distrubuted to different test


environments for autonomous systems. See following slides.

11 Bosch Engineering | Valisure - BEG/EPV | xx.03.2017


© Bosch Engineering GmbH 2016. Alle Rechte vorbehalten, auch bzgl. jeder Verfügung, Verwertung, Reproduktion, Bearbeitung, Weitergabe sowie für den Fall von Schutzrechtsanmeldungen.
Autonomous Automobile Systems
Verification and Validation of Trained Systems
4 base scenarios of testing environments:
 Hardware-In-The-Loop-Tests

 Disney Scenario (virtual driving test area) – Verification of the system with unknown and virtual
realised driving and test situations.

 Hollywood Scenario – Validation with physical realised scenarios on real test areas.

 The validation in real traffic situations is used complementary to the upper test methods / and
test scenarios. The validation in real traffic situations is limited in time and it is limited according
to the amount of driving situations.

Conclusion: The numerical amount of verification and validation test has to be done (due to safety
reasons and due to time limitations) in virtual test areas and real test areas.
12 Bosch Engineering | Valisure - BEG/EPV | xx.03.2017
© Bosch Engineering GmbH 2016. Alle Rechte vorbehalten, auch bzgl. jeder Verfügung, Verwertung, Reproduktion, Bearbeitung, Weitergabe sowie für den Fall von Schutzrechtsanmeldungen.
Autonomous Automobile Systems
Disney Scenario – Virtual driving test area
 Overall modeling of static objects (traffic signs, streets, parking cars, crash barriers, etc.) and of
dynamic / moving objects (passenger car, bike, humans, etc.).

 Out of the 3-D-Modeling are the sensor signals computed (this includes disturbing signals as
reflexions and visual limitations – as far as possible to modelate them).

 The dynamic reaction of the autonomous system is


implemented in the 3-D-model, as it is for
3-D-video-games (like „Need for Speed“).

 In this scenario it is possible to interact with human


drivers in the virtual space (like in a video game).

 The creation of this virtual driving test area will afford


big investigations and efforts in time and money.
13 Bosch Engineering | Valisure - BEG/EPV | xx.03.2017
© Bosch Engineering GmbH 2016. Alle Rechte vorbehalten, auch bzgl. jeder Verfügung, Verwertung, Reproduktion, Bearbeitung, Weitergabe sowie für den Fall von Schutzrechtsanmeldungen.
Autonomous Automobile Systems
Hollywood Scenario – Real driving test area
 Real simulated driving situations on testing areas.
 Provocation of highly complex driving situations.
 Optional with several autonomous and independently acting
vehicles.
 Continous test runs in the mode „24h/365d“ are in principle
possible.

14 Bosch Engineering | Valisure - BEG/EPV | xx.03.2017


© Bosch Engineering GmbH 2016. Alle Rechte vorbehalten, auch bzgl. jeder Verfügung, Verwertung, Reproduktion, Bearbeitung, Weitergabe sowie für den Fall von Schutzrechtsanmeldungen.
Autonomous Automobile Systems
700 thousend miles driving through the desert and more …?
Assumption: 99,x % of the driving situations are easy to control!
– To test easy driving situations repeatedly is inefficient and expensive.

For this collecting a lot of driving hours and high numbers of kilometers gives only
a good feeling for safety, but without a real plus of safety.

Approach of a risk based verification and validation:


It is required to concentrate on less frequent, but not
easy to control driving situations.
Definition of appropriate driving profiles with complex
situations to control in the different scenarios.
This results in saving time and ressources.

15 Bosch Engineering | Valisure - BEG/EPV | xx.03.2017


© Bosch Engineering GmbH 2016. Alle Rechte vorbehalten, auch bzgl. jeder Verfügung, Verwertung, Reproduktion, Bearbeitung, Weitergabe sowie für den Fall von Schutzrechtsanmeldungen.
Autonomous Automobile Systems
Real defensive driver
A real defensive driver chooses his
driving speed and his driveway
according to his confidence his ability to
control the driving situation for the next
few seconds.
For this he takes into account his driving
capability, his instantaneous
constitution (healthyness, tiredness), his
mental state and his sensory perception
(sight field, ability to see).

16 Bosch Engineering | Valisure - BEG/EPV | xx.03.2017


© Bosch Engineering GmbH 2016. Alle Rechte vorbehalten, auch bzgl. jeder Verfügung, Verwertung, Reproduktion, Bearbeitung, Weitergabe sowie für den Fall von Schutzrechtsanmeldungen.
Autonomous Automobile Systems
Autonomous defensive system
 The quality of its decisions while driving should
be evaluated and reflected by the autonomous
system on its own.

 Attributes representing the quality of the decisions


of the autonomous system have to be identified
and elaborated. These might be signal disturbences,
image quality, signal noise, etc.).

 Further the autonomous system might take into account the sensor avaiability, the sensor
healthyness and the principle quality of the sensor data.

Result:
Defensive, self-reflecting, autonomous system, according to the defensive driver.

17 Bosch Engineering | Valisure - BEG/EPV | xx.03.2017


© Bosch Engineering GmbH 2016. Alle Rechte vorbehalten, auch bzgl. jeder Verfügung, Verwertung, Reproduktion, Bearbeitung, Weitergabe sowie für den Fall von Schutzrechtsanmeldungen.
Autonomous Automobile Systems
Comparison to other branches and industries
The comparison with other branches is partly advantageously
(Aerospace, aviation, rail, shipping, storage logistics, robotics,
process automation, agriculture, medical engineering, …).
In comparison with the road traffic there is the context and
the environment less complex:
• No humans in the direct vicinity or environment (process automation,
underground railway / metro tube tunnel and agriculture)
• Lower complexity of the environment (storage logistics, robotics)
• Lower velocities (storage logistics, robotics, agriculture)
• Highly standardized infrastructure (rail, aviation)
• Less objects to consider (rail, aviation)
• Higher financial resources (aerospace, aviation)
18 Bosch Engineering | Valisure - BEG/EPV | xx.03.2017
© Bosch Engineering GmbH 2016. Alle Rechte vorbehalten, auch bzgl. jeder Verfügung, Verwertung, Reproduktion, Bearbeitung, Weitergabe sowie für den Fall von Schutzrechtsanmeldungen.
Autonomous Automobile Systems
Look-out – …still on the travel to the future…
 Concrete solutions for the verification and validation
are still under development. And these solutions for the
verification and validation have to be evaluated itself.

 There is a high need for collaboration inside the


automotive industry with OEMs and suppliers.
(for example in the SotiF* working group correlated
within the 2nd edition of the ISO26262).

 Further there is a need for collaboration cross boarder


of the automotive industry – because autonomous
systems are not restricted to the automotive industry.
The whole society is or will bed affected by
autonomous systems in future.
* © Nicolas Becker: „Safety of the intended Functionality“, Safetronic, Stuttgart, 2015
19 Bosch Engineering | Valisure - BEG/EPV | xx.03.2017
© Bosch Engineering GmbH 2016. Alle Rechte vorbehalten, auch bzgl. jeder Verfügung, Verwertung, Reproduktion, Bearbeitung, Weitergabe sowie für den Fall von Schutzrechtsanmeldungen.
Autonomous Automobile Systems
Literature and References
Topic:
Release of Autonomous Systems

 Walther Wachenfeld, Hermann Winner (Technische Universität Darmstadt, Fachgebiet Fahrzeugtechnik – FZD) – Kapitel 21 „Die Freigabe des
autonomen Fahrens“, „Autonomes Fahren“ (M. Mauer, et al.), 2015

Topic:
Extraction of Critical Scenarios & Condensing of Field Testing

 Philipp Glauner, Axel Blumenstock, Martin Haueis (Daimler AG; Böblingen) –


Workshop Fahrerassistenzsysteme: „Effiziente Felderprobung von Fahrerassistenzsystemen“, 2012

 Lutz Eckstein (Institut für Kraftfahrzeuge, RWTH Aachen University), Adrian Zlocki (Forschungsgesellschaft Kraftfahrwesen mbH Aachen),
“Safety Potential of ADAS – Combined Methods for an Effective Evaluation”, Paper Number 13-0391

20 Bosch Engineering | Valisure - BEG/EPV | xx.03.2017


© Bosch Engineering GmbH 2016. Alle Rechte vorbehalten, auch bzgl. jeder Verfügung, Verwertung, Reproduktion, Bearbeitung, Weitergabe sowie für den Fall von Schutzrechtsanmeldungen.
Autonomous Automobile Systems
Literature and References
Topic:
Simulation & virtual test environments

 Schmidt, F. – „Funktionale Absicherung kamerabasierter Aktiver Fahrerassistenzsysteme durch Hardware-in-the-Loop-Tests“,


Diss. Universität Kaiserslautern, 2012

 Helmer, T.; Kühbeck, T.; Gruber, C. & Kates, R. Development of an Integrated Test Bed and Virtual Laboratory for Safety Performance Prediction in
Active Safety Systems Proceedings of the FISITA 2012 World Automotive Congress, Springer Berlin Heidelberg, 2013, 197, 417-431

 Nentwig, M.; Schieber, R.; Miegler, M. & Stamminger, M. – „Möglichkeiten und Grenzen der simulationsbasierten Validierung von kamerabasierten
Fahrerassistenzfunktionen“, VDI Berichte, 2010, 2106, 141-152

 Roth, E.; Dirndorfer, T.; v. Neumann-Cosel, K.; Fischer, M.-O.; Ganslmeier, T.; Kern, A. & Knoll, A., –
“Analysis and Validation of Perception Sensor Models in an Integrated Vehicle and Environment Simulation”
Proceedings of the 22nd Enhanced Safety of Vehicles Conference, Technische Universität München, 2011

 Belbachir, A.; Smal, J.-C.; Blosseville, J.-M. & Gruyer, D. – “Simulation-Driven Validation of Advanced Driving-Assistance Systems”
Procedia - Social and Behavioral Sciences , 2012, 48, 1205 – 1214

21 Bosch Engineering | Valisure - BEG/EPV | xx.03.2017


© Bosch Engineering GmbH 2016. Alle Rechte vorbehalten, auch bzgl. jeder Verfügung, Verwertung, Reproduktion, Bearbeitung, Weitergabe sowie für den Fall von Schutzrechtsanmeldungen.
THANK
YOU FOR
ATTENTION!
AUTONOME
VALIDATION OF HIGHLY
AUTOMATED AND
AUTOMOBILSYSTEME –
AUTONOMOUS AUTOMOBILE
GROßE
SYSTEMS
HERAUSFORDERUNGEN!
AUTOMOTIVE SAFETY AND SECURITY WEEK
TESTING ADAS AND SELF-DRIVING CARS
FRANKFURT OBERURSEL
27. – 30. MARCH 2017