Lean Quality Assurance

by Gilb
Copyright: Gilb 2010, Slide owner: Kai@Gilb.com @kaigilb Tom@Gilb.com @imtomgilb www.gilb.com These slides will be at: http://www.gilb.com/tiki-download_file.php?fileId=392 www.Gilb.com

Version 8- Sep. 2010

Main Take-away Points

Quality Assurance is far more than test,
and it can be far more cost-effective

Quality is far more than bugs You probably have a lot to learn,
if you want real competitive quality

www.Gilb.com

Version 8- Sep. 2010

Begin: Quality Assurance is far more than test

and it can be far more costeffective

www.Gilb.com

Version 8- Sep. 2010

a story

www.Gilb.com

Version 8- Sep. 2010

Inspection Effectiveness
Capers Jones

www.Gilb.com

Version 8- Sep. 2010

 www.Gilb.com

Version 8- Sep. 2010

 www.Gilb.com

Version 8- Sep. 2010

 www.Gilb.com

Version 8- Sep. 2010

 www.Gilb.com

Version 8- Sep. 2010

All Defects

Best Practice Testing Combined

Design

Remaining Defects
www.Gilb.com

Version 8- Sep. 2010

10

Little hope of zero defects

Between

8 10
and defect removal stages required to achieve removal effectiveness of

95%

www.Gilb.com
Version 8- Sep. 2010

11

Testing Capability (C. Jones)

120

%
100 80 60 40 20 0 1st year

In Field Test

2nd year

5th year

www.Gilb.com

Version 8- Sep. 2010

12

Defect Detection Capability (C. Jones)

120

%
100 80 60 40 20 0 1st year

In Field Test Inspection

2nd year

5th year

www.Gilb.com

Version 8- Sep. 2010

13

IBM Defect Avoidance Experience

120

%
100 80 60 40 20 0 1st year

In Field Test Inspection Defect Prevention Effectiveness

2nd year

5th year

www.Gilb.com

Version 8- Sep. 2010

14

Design Quality In

www.Gilb.com

Version 8- Sep. 2010

15

 www.Gilb.com

Version 8- Sep. 2010

16

but by Engineering Quality In

Reliability Work hours $ Kr. Usability Maintenance Performance Security

www.Gilb.com

Version 8- Sep. 2010

17

Setting Quality Goals

Usability.Learn Scale: average time to Learn how to operate the computer, from .. to .. Status [today] 3 hours Goal [next year] 10 min.

www.Gilb.com

Version 8- Sep. 2010

18

Designing to meet Quality within Costs

Design Ideas

Qualities

www.Gilb.com

Version 8- Sep. 2010

19

End: Quality Assurance is far more than test

and, QA can be far more cost-effective

www.Gilb.com

Version 8- Sep. 2010

20

Start: Quality is far more than bugs

www.Gilb.com

Version 8- Sep. 2010

21

a story

www.Gilb.com

Version 8- Sep. 2010

22

System Performance

Capacity How Much

Quality How Well

Resource Saving Efficiency

www.Gilb.com

Version 8- Sep. 2010

23

Qualities are many and variable

Usability Adaptability Integrity Availability

Learning Doing Error Rate Portability Enhancability Compatibility Threat Type and Frequency Security Mitigation

Reliability Maintainability (fault fix speed)

www.Gilb.com

Version 8- Sep. 2010

24

Quantify the Quality to Assure It

I often say that when you can measure what you are speaking about, and express it in numbers, you know something about it; but when you cannot measure it, when you cannot express it in numbers, your knowledge is of a meagre and unsatisfactory kind;
- Lord Kelvin, 1893
www.Gilb.com

Version 8- Sep. 2010

25

End: Quality is far more than bugs

www.Gilb.com

Version 8- Sep. 2010

26

7
Competitive Lean QA methods to Learn
www.Gilb.com
Version 8- Sep. 2010

27

1.

Stakeholders Decide Qualities

Suzanne Robertson & James Robertson

www.Gilb.com

Version 8- Sep. 2010

28

2.

www.Gilb.com

Version 8- Sep. 2010

29

3.

Assuring that Designs give Qualities

Usability

www.Gilb.com

Version 8- Sep. 2010

30

4.

Measure Quality Levels in Specifications with Inspection

www.Gilb.com

Version 8- Sep. 2010

31

5 a.

Numeric Quality Gateways

www.Gilb.com

Version 8- Sep. 2010

32

5 a.
100 80 60 40 20 0 0

Numeric Quality Gateways Improve Quality of work

Gary at McDonnell-Douglas

Defects/Page

80 Majors Found
(~160-240 exist!)

40 23 8 0 5

February

April 33

Inspections of Garys Designs

www.Gilb.com
Version 8- Sep. 2010

6
43% 40% 35% 30% 25% 20% 15% 10% 5% 0%

DPP Improves Quality by 10x: Raytheon

Start of Effort The individual learning curve ??
% CONC % COC

COC Cost of Conformance

CONC Cost of Rework (non-conformance)

Bad Process Change

5%

1st year 2nd year

4rd year 5th year 6th year 7th year 8th year 34

www.Gilb.com Version 8- Sep. 2010 www.sei.cmu.edu/publications/%20documents/95.reports/95.tr.017.html

Frequent feedback and improvement a assure quality

Potential Value Plan Do Study Perceived-Value Info Realized-Value Informa5on Stake-

holders

Realized Value

Stakeholders

Act

Stakeholders

Stakeholders

Stakeholders

Stakeholders

Other Cri5cal Factors

2 Kinds of Feedback from Stakeholders, when value increment is really exploited in practice after delivery. Combined with other information from the relevant environment. Like budget, deadline, technology, politics, laws, marketing changes. www.Gilb.com Version 8- Sep. 2010

35

7b
Measure

Learn

Stakeholders

Deliver

Value Management Process

Develop Decompose

Values

Solutions

www.Gilb.com

Version 8- Sep. 2010

36

7b
Measure

Learn

Stakeholders

Values

Deliver

Solutions

Develop

Decompose

www.Gilb.com

Version 8- Sep. 2010

37

7b
Measure

Learn

Stakeholders

Values

Deliver

Solutions

Develop

Decompose

www.Gilb.com

Version 8- Sep. 2010

38

7b
Measure

Learn

Stakeholders

Values

Deliver

Solutions

Develop

Decompose

www.Gilb.com

Version 8- Sep. 2010

39

7b
Measure

Learn

Stakeholders

Values

Deliver

Solutions

Develop

Decompose

www.Gilb.com

Version 8- Sep. 2010

40

7b
Measure

Learn

Stakeholders

Values

Deliver

Solutions

Develop

Decompose

www.Gilb.com

Version 8- Sep. 2010

41

7b
Measure

Learn

Stakeholders

Values

Deliver

Solutions

Develop

Decompose

www.Gilb.com

Version 8- Sep. 2010

42

7b
Measure

Learn

Stakeholders

Values

Deliver

Solutions

Develop

Decompose

www.Gilb.com

Version 8- Sep. 2010

43

7b
Measure

Learn

Stakeholders

Values

Deliver

Solutions

Develop

Decompose

www.Gilb.com

Version 8- Sep. 2010

44

7b
Measure

Learn

Stakeholders

Deliver

Value Management Process

Develop Decompose

Values

Solutions

www.Gilb.com

Version 8- Sep. 2010

45

End

7
Competitive Lean QA methods to Learn
www.Gilb.com
Version 8- Sep. 2010

46

What you can do immediately

Identify the 5 most critical qualities of your system. Quantify the 5 qualities. For each quality,
set a Current level and a Goal level

www.Gilb.com

Version 8- Sep. 2010

47

Main Take-away Points

Quality Assurance is far more than test,
and it can be far more cost-effective

Quality is far more than bugs You probably have a lot to learn,
if you want real competitive quality

www.Gilb.com

Version 8- Sep. 2010

48

Thanks!

Thanks!
Questions: now, briefly After lecture, all during the conference. Kai@Gilb.com Tom@Gilb.com Mobile: +47 920 66 705 www.Gilb.com Copy of these slides will be in Downloads/Slides: http://gilb.com/tiki-list_file_gallery.php?galleryId=14
www.Gilb.com
Version 8- Sep. 2010

49

The Lean Quality Assurance Methods

Everything not adding value to the Customer is considered to be waste.
This includes:
unnecessary code and func?onality Delay in the soAware development process Unclear requirements Bureaucracy Slow internal communica?on The learning process is sped up by usage of short itera?on cycles each one coupled with refactoring and integra?on tes?ng. Increasing feedback via short feedback sessions with Customers helps when determining the current phase of development and adjus?ng eorts for future improvements.

Amplify Learning

Decide as late as possible Deliver as fast as possible Empower the team Build integrity in See the whole

separate components work well together as a whole with balance between exibility, maintainability, eciency, and responsiveness. Think big, act small, fail fast; learn rapidly
www.Gilb.com
Version 8- Sep. 2010

50