Sie sind auf Seite 1von 67

0hapter 1

ë  

ë ë  



| 
  

1 Principles 2 Lifecycle 3 Static testing

4 Dynamic test
5 Management 6 Tools
techniques
Principles

1 2 3 ë ë  




4 5 6

0ontents
Why testing is necessary
Fundamental test process
Psychology of testing
Re-testing and regression testing
Expected results
Prioritisation of tests
      

£ ºo generally accepted set of testing


definitions used world wide
£ ºew standard BS 7925-1
u 
 

  


 
u  


u 


   
 
u 
 
 
G   
£ Error: a human action that produces an
incorrect result
£ Fault: a manifestation of an error in software
u   

 ! 
u 
"
!
#! !
 !

£ Failure: deviation of the software from its


expected delivery or service
u ! 

 
Failure is an event; fault is a state of
the software, caused by an error
Ô     
A   
 

   
  
 

  
 
  
^  
 

£ Reliability: the probability that software will


not cause the failure of the system for a
specified time under specified conditions
u  

!
 

! uu

$%
! #    


u   



 
!  

! $
u &!
&! uu

' 
   

 
$
G  
   
 

£ software is written by human beings


u  
 #!   

 
u 
 #! 
( 


u 
  


£ under increasing pressure to deliver to strict
deadlines
u  
 
! !  

u 

 

£ if you have ever written software ...


G   
 
   

£ huge sums
u ) 
*+,   
u -


 .
!+/*0
u )
  ) 
+*0
£ very little or nothing at all
u    



u   
 
 
 
£ software is not ³linear´:
u  ! 





ë
    

£ software faults can cause death or injury


u     
 
 


    
 
 

uu/*
u   
 

u   
) !12
 ) 

u  

 

!
!  

ë      

u 
!
 
  
 
! 
u 
 !  

    
 

u   
 




 
 
 
 






u 
   
 
 ! 
u 
!

   !
  
3
 
u 
!
 !
 


"
 

u  
!
! 

u   !

G        
Avr. 4 menus
3 options / menu

system has Average: 10 fields / screen


20 screens 2 types input / field
(date as Jan 3 or 3/1)
(number as integer or decimal)
Around 100 possible values

Total for 'exhaustive' testing:


`  `     
xf 1 second per test, 8000 mins, 133 hrs, 17.7 days
(not counting finger trouble, faults or retest)

               


Ô     

£ What is exhaustive testing?


u 
 




"!

u 
 


 



"
!

u
"
      !  

   
£ ±ow much time will exhaustive testing take?
u 
 

u  ! 

u  !  

±       

u (


! 
u 
!

 !

u 
!! 
!
 
u 
!

   



 
u 
!
  
   



 
u 

  
 !

±     

£ xt depends on ^ë
u        ! 
u    !  !
 
u   

 !

! 



 !

! 
uu

 

u    
    


u     
 
u   


uu
 #

 


ë        

£ test time will always be limited


£ use ^ë to determine:
u   
  
u
u
  
 
 !  
 

 È i.e. where to
place emphasis
u     
    

£ use ^ë to
u 
 
 
 

 
   
 444
r     

Prioritise tests
so that,
whenever you stop testing,
you have done the best testing
in the time available.
   

£ testing measures software quality


£ testing can find faults; when they are
removed, software quality (and possibly
reliability) is improved
£ what does testing test?
u 
!   #


 
u 
 uu!   5!
6
   #!  #
    #
!  #
   #
4
ß  
   
  

£ contractual requirements
£ legal requirements
£ industry-specific requirements
u
4 4
!  ! 78)# 

  
 #
uu 
u
  
 #
  
u

!
   #    

xt is difficult to determine
how much testing is enough
but it is not impossible
Principles

1 2 3 ë ë  




4 5 6

0ontents
Why testing is necessary
Fundamental test process
Psychology of testing
Re-testing and regression testing
Expected results
Prioritisation of tests
 |  

  
Ô

 
Ô
  

 ! "  #$%%% `


 !
Ô  #   " 
Ô 

 & Ô    #$%%% `


 &
 &
Ô 
 & #        " 
Ô 
Ô
Ô 
 

     

   

|   !

 

        
   
 

£ how the test strategy and project test plan


apply to the software under test
£ document any exceptions to the test strategy
u
4 4 

 

 
 5!




 !   

!
  
 
£ other software needed for the tests, such as
stubs and drivers, and environment details
£ set test completion criteria
 
  

|   !

 

        
   

$&  & 


  
'&  
A   
Finds faults
£ effective

£ exemplary
Represents others

£ evolvable
Easy to maintain
£ economic

0heap to use
 
  

£ test specification can be broken down into three


distinct tasks:
24 
6 


9 (  


 


       

/4 
 6 


9( 
9 (  



 4
4
 
 

:4 ! 6 

 

  #  #
4
"# 
   



9 (  


   

£ list the conditions that we would like to test:
u !
 

 
 
 5!

 
  

 
u 


    

!   
  !

u
4 4
       (
)   && * (
&   `
`)
)` (
£ prioritise the test conditions
u ! 
!
       



ë       
xmportance

]
Best set

] First set Time


$#   



9( 
9 (  



£ design test input and test data
u

 
"
 



    
£ determine expected results
u 
   
! 


 
#  
! ! #  
     

£ design sets of tests
u  


 
 

3
 
!


  #!    

#   ! 
r  
i      & 
!  
xmportance  & 
Ô 

Time
%#  
 

 

 

£ prepare test scripts
u 

 



 


 


   
 

u    


 
 


 
£ prepare test data
u    ! 
"    
  
  
  
 

 
£ prepare expected results
u !




 

  
"
!

    

|   !

 

        
   
Ô   

£ Execute prescribed test cases


u     
 
u !  
"
!

 
 
    
    & )   
 
u  


 !! 

   

|   !

 

        
   
    "

£ The test record contains:


u 

 
  !  !!
  & 
  
£ Follow the plan
u  
 
  
u !
 !! 
 


u  !
  
 
!


 

u 
   



!
 
    

  




! 
 

    $

£ 0ompare actual outcome with expected


outcome. Log discrepancies accordingly:
u  
!
u
 ! 
4 4
"


!  
u
  

  !
u
 !  
 
£ Log coverage levels achieved (for measures
specified as test completion criteria)
£ After the fault has been fixed, repeat the
required test activities (execute, design, plan)
0      

|   !

 

        
   
0      

£ Test completion criteria were specified in the


test plan
£ xf not met, need to repeat test activities, e.g.
test specification to design more tests

  

 

        
   

+,
       

£ 0ompletion or exit criteria apply to all levels


of testing - to determine when to stop
u 

#! 
!


 5!
#
4 4
)     
 - 
 -  &  
u ! ! 
4 4
!
"


u   

&   
0   
 
 
  

|   


a a
ë
    & 
   
Ô  
 
  
^  0 
Principles

1 2 3 ë ë  




4 5 6

0ontents
Why testing is necessary
Fundamental test process
Psychology of testing
Re-testing and regression testing
Expected results
Prioritisation of tests
G   

£ build confidence
£ prove that the software is correct
£ demonstrate conformance to requirements
£ find faults
£ reduce costs
£ show system meets user needs
£ assess the software quality
0 

0onfidence
  
   

Time

ºo faults found = confidence?


A 
   You think
you are here

Many ±igh Few


Few
Faults Faults
Faults

Low ±igh
Software Quality

Few Test Few


Faults Quality Faults

You may
be here

Low
A       

£ Show that the system:


u 
  !
u 
 ;   ! ;
Goal: show working
Success: system works

Fastest achievement: easy test cases

Result: faults left in


A      

£ Show that the system:


u 
  ! ;
u 
 ;   !
Goal: find faults
Success: system fails

Fastest achievement: difficult test cases

Result: fewer faults left in


    

Purpose of testing: to find faults


Finding faults destroys confidence
Purpose of testing: destroy confidence

Purpose of testing: build confidence

The best way to build confidence


is to try to destroy it
G      

£ A destructive process
£ Bring bad news (³your baby is ugly´)
£ Under worst time pressure (at the end)
£ ºeed to take a different view, a different
mindset (³What if it isn¶t?´, ³What could go
wrong?´)
£ ±ow should fault information be
communicated (to authors and managers?)
  '     #

u !
   !  
 

u   


! 
 
 

u 
 




   

  
u 



   !
<
u  ! <
u 


     
  
u 


!  

 ! 



!  

 !  uu
! 

u 

   ! ! !
! 


u 
! 
 

       #

u  

  #  
4!


u 
 ! 3
 
  ! !
<
u 

 

 


 ! 
u 


   
 
#   
 
#
 !


u 
 3
 

u   
 !
 
u ! 
 
 ! 


£ Test your own work?


u  :0=
 :0=uu *0=! ! 
u 
!    !  


u 

 !
   

#     


u
   

&.   &  
   /+Ô  &  
@


£ ºone: tests designed by the person who


wrote the software
£ Tests designed by a different person
£ Tests designed by someone from a different
department or team (e.g. test team)
£ Tests designed by someone from a different
organisation (e.g. agency)
£ Tests generated by a tool (low quality tests?)
Principles

1 2 3 ë ë  




4 5 6

0ontents
Why testing is necessary
Fundamental test process
Psychology of testing
Re-testing and regression testing
Expected results
Prioritisation of tests
^   
 
  
 

£ Run a test, it fails, fault reported


£ ºew version of software with fault ³fixed´
£ Re-run the same test (i.e. re-test)
u ! 

" 

 

u 

  
#
  
"
  
 

 


 
<
u 
 !  
   
£ xf test now passes, fault has been fixed
correctly - or has it?
^      
  !

ºew faults introduced by the first


fault fix not found during re-testing

x
x

x
Ú
Fault now fixed
Re-test to check
^   

£ to look for any unexpected side-effects

x
x

x
Ú
0an¶t guarantee
to find them all
^     "

£ misnomer: "anti-regression" or "progression"


£ standard set of tests - regression test pack
£ at any level (unit, integration, system,
acceptance)
£ well worth automating
£ a developing asset but needs to be
maintained
^     $

£ Regression tests are performed


u 
 

# ! !  "

u 
 

  

#

    
!      


u 


 "
 !

£ Regression test suites
u


 

u 
! 

u 

 


^     %

£ Maintenance of the regression test pack


u
  




 
  
  



    
u 

 

4 4  

! 



u 

  

!
 
!

  !

! 
 


  
  



u
  

  
  ! ! 
  

4 4!  "
 
^        

£ Test execution tools (e.g. capture replay) are


regression testing tools - they re-execute
tests which have already been executed
£ Once automated, regression tests can be run
as often as desired (e.g. every night)
£ Automating tests is not trivial (generally takes
2 to 10 times longer to automate a test than to
run it manually
£ Don¶t automate everything - plan what to
automate first, only automate if worthwhile
Principles

1 2 3 ë ë  




4 5 6

0ontents
Why testing is necessary
Fundamental test process
Psychology of testing
Re-testing and regression testing
Expected results
Prioritisation of tests
Ô    

£ Should be predicted in advance as part of the


test design process
u 9>
)!  (!
  
 ! 

 

 
4
£ Why not just look at what the software does
and assess it at the time?
u !  !
 
 

  
!  !
 
 

  uu 

#   

  
!
u ! 
# ! 
>2
! 
# ! 
>2uu 
 !
 !   
 
A  expected
inputs outputs

A Program:
3 6?
Read A
xF (A = 8) T±Eº
PRxºT (³10´)
ELSE 8 10?
PRxºT (2*A)

0   1   


Principles

1 2 3 ë ë  




4 5 6

0ontents
Why testing is necessary
Fundamental test process
Psychology of testing
Re-testing and regression testing
Expected results
Prioritisation of tests
|      

£ We can¶t test everything


£ There is never enough time to do all the
testing you would like
£ So what testing should you do?
r     

Prioritise tests
so that,
whenever you stop testing,
you have done the best testing
in the time available.
±     

£ Possible ranking criteria (all risk based)


u
 

 !
!
 



u
 

 !
!
   

u
 

 !

  

u  
! 
   
 

5! 


u       
! 
(!

u 

 

u 
  
  

u  
"
#
  
Principles

1 2 3 ë ë  




4 5 6

Summary: Key Points


Testing is necessary because people make errors
The test process: planning, specification, execution,
recording, checking completion
xndependence & relationships are important in testing
Re-test fixes; regression test for the unexpected
Expected results from a specification in advance
Prioritise to do the best testing in the time you have