Lauri Vtinen

ICSM 2013, 26.9.2013

Tanja Toroi, Anu Raninen and Lauri Vtinen :

Identifying Process Improvement Targets in

Test Processes: A Case Study

Agenda
1.

Problems in the test processes

2.

Motivation to light-weight process improvement

3.

Case study and A Short Introduction to LAPPI technique

4.

Results of the case study with conclusions

26.9.2013

Problems in the Test Processes (earlier results)

Test planning, test cases and unit testing are often vague and
neglected

Lack of testing know-how and insufficient testing tools often hinder

testing development
Lack of testing expertise prevents organizations from using
software testing methods
Verification and validation are first processes to neglect when time
runs out
26.9.2013

Motivation to Light-weight Process

Improvement
Software testing is considered important factor in quality assurance
Continuous efforts on improving a testing process is important
E.g. CMMI and ISO/IEC 15504 standards are often considered too
heavy-weigh in small software companies
Light-weight process modeling is one possible solution

26.9.2013

Case study

How to identify software test process problems in a cost-efficient

way?
What are the major problem areas in software testing?
LAPPI technique was applied to model the test processes and
identify process problems
Research article available:
A. Raninen, J. J. Ahonen, H.-M. Sihvonen, P.Savolainen and
S. Beechman: LAPPI: A light-weight technique to practical
process modeling and improvement target identification,
J. Softw. Evol. And Proc. 2012, DOI: 10. 1002/smr.1571
26.9.2013

Backbone of the LAPPI technique (1/3)

Workshop 1: Roles and information flow modeling
Information flow

Role
PROGRAMMER

Complete features
of the next release

Test cases to implement

into a test automation

TEST DESIGNER

A problem in the
information flow
Dont understand
the test case

TESTER

Test report

26.9.2013

Backbone of the LAPPI technique (2/3)

Workshop 2:

Process modeling

26.9.2013

Backbone of the LAPPI technique (3/3)

Workshop 2:

Process modeling

26.9.2013

Features of case organizations

Case study was conducted in three different case organizations
A

Market

Farming

Metal Industry

Telecom

Size of
organization

Small

Large

Medium

Emplyees in sw.
Developmet /
system testing

9 / 4-6

24 / 2

30 / 1

Country

Finland

Multinational

Finland

Age of company

24

43

13

26.9.2013

Results of the case study (1/2)

Problem

1. No exit criteria defined for testing

(*)

2. Test automation is not integrated with

manual test process

(*)

X
X

3. Level of unit testing conducted by

programmers vary a lot

4. Lack of proper test case specification

5. All repercussions of the code changes are not

known

6. Documentation processes are vague

X
X

(*) less reported problem

26.9.2013

10

Results of the case study (2/2)

Improvement suggestions

1. Define minimum level of unit testing

2. Pay attention to test case specification.

3. Provide training in software testing.

4. Streamline the usage of defect tracking system.

5. Define exit criteria for testing.

6. Consider software testing already in the

requirement specification phase.

7. Integrate test automation with the manual test

process.
8. Improve and monitor documentation.
9. Provide more test resources.

X
X

X
X

26.9.2013

11

Conclusions

Problems of testing are similar across different kinds of companies

Process modeling can be used to initiate process improvement
LAPPI technique is a simple and cost-effective tool to identify
process problems

26.9.2013

12

Identifying Process Improvement Targets in Test Processes:

A Case Study

Thank you for your attention!

Lauri.Vaatainen@uef.fi

www.uef.fi