PLANNING PROCESS

CBOK Domain 10
By Marta Heinrich
Process Cycle

Comprised of 4 parts:
■ Plan (P): Devise a plan
■ Do (D): Execute the plan
■ Check (C): Check the results
■ Action (A): Take the necessary action
Process Cycle
■ The role of testing is to fulfill the ‘Check’
responsibilities

■ It is not to determine whether software can

be placed into production
Process Cycle
■ 90% of all defects are caused by process
problems
• Dr. W. Edwards Deming

■ Anytime a process is used, defects of

approximately the same type and frequency
occur
Process Cycle
■ Testing, as performed in most
organizations, is a process designed to
compensate for an ineffective software
development process.
Process Cycle
■ The test process must be designed to
provide as much value as possible in the
time available
Process Cycle
■ Testing is no longer an ‘after-thought’ to
the software development life cycle, but
rather is a key part of it.
Test Planning Activities

Comprised of 3 parts:
■ Pre-Planning Activities
■ Test Planning
■ Post Planning Activities
Pre-Planning Activities

■ Success Criteria/Acceptance Criteria

■ Test Objectives

■ Assumptions

■ Entrance/Exit Criteria
Pre-Planning Activities
Success Criteria/Acceptance Criteria
■ Criteria that must be validated through
testing to provide the information needed to
make an acceptance decision.
Pre-Planning Activities
Success Criteria/Acceptance Criteria
■ Met during each phase of development by
entrance/exit criteria
■ Pre-determined in test plan and/or project
plan
■ Examples
Pre-Planning Activities
Test Objectives
■ Objectives to be accomplished through
testing
Pre-Planning Activities
Test Objectives
■ Should restate the project objectives from
the project plan
■ Should determine whether the projected
objectives have been achieved
■ If not stated clearly in the project plan, test
team must develop their own
Pre-Planning Activities

Test Objectives
■ Set objectives to minimize risks
■ Brainstorm to identify project objectives
■ 10 or fewer is a good number
■ Relate to the test policy
Pre-Planning Activities

Test Objectives
■ Itemize the objectives
■ Write in a measurable statement
■ Assign a priority
■ Define completion criteria
Pre-Planning Activities
Assumptions
■ Establishing conditions that must exist for
testing to be comprehensive and on
schedule.
Pre-Planning Activities
Entrance/Exit Criteria
■ Criteria that must be met prior to moving to
the next level of testing or into production.
Pre-Planning Activities
Entrance Criteria
■ Required conditions and standards for work
product quality that must be present or met
for entry into the next stage of the software
development process
Pre-Planning Activities
Exit Criteria
■ Standards for work product quality, which
block the promotion of incomplete or
defective work products to subsequent
stages of the software development process
Pre-Planning Activities

Entrance/Exit Criteria
■ Defined in Test Plan
■ Provides guidelines
■ Examples of different criteria
Test Planning
■ Test Plan ■ Staffing

■ Requirements and ■ Approach

Traceability
■ Test Check Procedures
■ Estimating

■ Scheduling
Test Planning

Test Plan
■ The deliverables to meet the test’s
objectives
■ The activities to produce the deliverables
■ The schedule and resources to complete the
activities
Test Planning

Test Plan

■ Why create a test plan?

■ When do you start the test plan?

Test Planning

Test Plan
Documented tests should be:
 Repeatable
 Controllable
 Insure Adequate Coverage
Test Planning
“The act of designing tests is one of the most
effective error prevention mechanisms
known…The thought process that must take
place to create useful tests can discover and
eliminate problems at every stage of
development.”
– Boris Beizer
Test Planning
Test Plan
■ IEEE standard defines a hierarchy of test
documents
– Test Plan
– Test Design Specification
– Test Case Specification
– Test Procedure Specification
■ View two sample plans
Test Planning
Types of Tests
■ A key step in developing a test strategy is to
determine the right tests to execute

■ Determined by the objectives of the test and

the type of application
Test Planning
Types of Tests
■ White Box (Glass Box) ■ Vendor validation
■ Black Box ■ Conversion
■ Functional ■ Stress/Load
■ Integration ■ Performance
■ Unit ■ Recovery
■ System ■ Configuration
■ Requirements-based ■ Benefits realization
■ Usability (User Acceptance)
Test Planning
Requirements/Traceability
■ Defines the tests needed and relates those
tests to the requirements
Test Planning
Requirements
■ A description of a condition or capability
of a system -- Also known as stakeholder
needs.
■ Derived from user needs
■ Stated in a contract, standard, or
specification
Test Planning
Traceability
■ Ability to trace requirements is critical to
ensure user needs have been delivered

■ Primary goal of software testing is to prove

requirements are actually delivered in the
final product
Test Planning
Traceability
Accomplished using any of the following:
 Use cases
 Activity diagrams
 Class diagrams
 Test cases
 Scenarios
Test Planning
Estimating
■ Determines the amount of resources
required to accomplish the planned
activities
Test Planning
Scheduling
■ Establishes milestones for completing the
testing effort
Test Planning

Scheduling
Four key objectives:
■ Provide predictability
■ Identify opportunities to improve or protect
the project schedule
■ Be fair to the team
■ Maximize productivity
Test Planning

Scheduling
Tips to achieve goals
■ Measure performance and productivity
■ Identify and estimate every task
■ Identify tasks as fixed or recurring
■ Allow extra time
Test Planning

Scheduling
Tips to achieve goals (continued)
■ Recognize individuality of the team
■ Beware of late hiring in the schedule
■ Take deliberate shortcuts
■ Be careful of too many meetings
Test Planning
Staffing
■ Selecting the size and competency of staff
needed to achieve the test plan objectives

■ Also covered in CBOK Domain 7

Test Planning
Staffing
■ What types of personnel should be
considered for inclusion on the test team?
■ What attributes and skills make up a good
tester?
■ What is a good ratio of testers to
developers?
Test Planning

Staffing
Steps to form the best team:
■ Identify potential test team members
■ Recruit test team members and develop
tentative test assignments
■ Define individual work assignments
Test Planning
Approach
■ Methods, tools, and techniques used to
accomplish test objectives

■ Covered in CBOK Domain 5 and 9

Test Planning

Test Check Procedures

■ Test quality control
■ Set of procedures based on the test plan and
test design
■ Incorporating test cases that ensure tests are
performed correctly and completely
Test Planning
Test Check Procedures
Includes the following:
– Test coverage evaluation
– Product error evaluation
– Test effectiveness evaluation

■ Covered in CBOK Domain 16

Test Planning
Test Check Procedures
Test coverage evaluation
■ The process of assessing the thoroughness
of the collective set of test cases for the
product, and deciding whether or not to
develop more tests.
Test Planning
Test Check Procedures
Product error evaluation
■ The process of assessing the quality of the
product, with respect to the test execution,
and deciding whether or not to develop
more tests.
Test Planning
Test Check Procedures
Test effectiveness evaluation
■ The process of assessing the overall
effectiveness of the current testing effort
relative to the test completion criteria, and
deciding whether to stop testing, or add
more tests and continue.
Post Planning Activities
■ Change Management

■ Versioning
Post Planning Activities
Change Management
■ Modifies and controls the plan in
relationship to the actual progress and scope
of the system development
Post Planning Activities
Versioning
■ Methods to control, monitor, and achieve
change

Includes:
– Change Control
– Change Management
– Configuration Management
Post Planning Activities
Versioning
■ The ability to maintain control over the
changes made to ALL project artifacts is
critical to the success of a project
Post Planning Activities
Versioning
The Configuration Management (CM) portion
of the project facilitates:
■ Product baselines
■ Review
■ Change
■ Defect tracking
Post Planning Activities
Versioning
Other duties of CM:
■ Write the CM Plan
■ Report change request based progress
statistics
■ Support product development activities
■ Allow all project artifacts to be
available as required
Post Planning Activities

Versioning
 Once testing begins, the project must ensure
that the correct versions of software
components are being tested.
 Migration and Back Out procedures must
be developed
Planning Process -- Summary
Consists of 3 parts:
■ Pre-Planning Activities
■ Test Planning
■ Post-Planning Activities

■ Testing covers the ‘C’ in the PDCA

Continuous Process Cycle
Bibliography
■ CSTE Study Guide, QAI, 2000
■ Kaner, Polk, and Nguyan. Testing Computer
Software, Wiley Computer Publishing, 1999.
■ Kit, Edward. Software Testing in the Real World,
Addison Wesley, 1997.
■ Perry, William E. Effective Methods for Software
Testing, Wiley Computer Publishing, 2000.