Rolls-Royce EVMS Implementation using SAPs Project

Systems

Industry & Government Working Together for

Successful EVMS Implementation on the
Joint Strike Fighter (JSF) Program

Presenters
!

Martin Eveleigh
! Rolls-Royce North America
! Focal Point for Rolls-Royce Defense
Europe EVMS Implementation
! Based in GEAE Evendale, Cincinnati
! JSF F120 Project Engineer
Jim Henderson
! NAVAIR EVMS Focal Point
! Team Chief for EVMS Validation
Luis C Contreras
! AzTech International - Consultant to
Rolls-Royce during EVMS implementation
2

What well cover

!
!
!
!
!

!
!
!

Program Background
Why Rolls-Royce Sought DoD Validation
Team philosophy/approach
3 Top Reasons for successful implementation
Review Process:
! Pre-Review (before any on-site visits)
! On-Site Reviews
! Post-Review
System Description & Instructions
! How SAP was integrated into the
documentation
SAP Challenges we overcame
5-10 Minutes for Q&A
Contact information for future Q&A

Please hold your

questions or
well never get
through this!

Major Achievement for Rolls-Royce & UK/US Review Team

!

!
!

First DoD EVMS Validation in the UK (first

overseas, actually)
Rolls-Royce is the largest SAP user:
! for Aerospace & Defense (A&D)
! Project Systems (PS) module
! Computer Automated Time Sheets (CATS)
module
First EVMS Validation using SAP
Started validation process 12 months before
contract award
Completed validation process 6 months prior
to E&MD contract

Did we validate
EVMS or SAP?

Global Implementation Team

!

Wealth of Experience from these Team Members:

Contractor: Rolls-Royce Defense (Europe)
!
! Rolls-Royce Defense North America - Indy
! Consultants (AzTech International)
Government Customers:
!
! UK Ministry of Defence (MoD) SPS & DPA
! JSF Programme Office
! DoD:
! NAVAIR
! DCMA
! Canada MHP
Okay, we didnt
always agree,
Commercial Customers:
!
but
! General Electric (GE)
! Pratt & Whitney (PW)

Spirit of Implementation
!
!

Everyone recognized the importance

Government, GE, & Pratt Whitney were
partners / stakeholders in our success
Open to critiques, advice, & experience

Not another
#@%&*
audit

Rolls-Royce Team Key Players

!
!

!
!

!
!
!
!
!
!
!
!
!
!
!
!

John Whatley (JSF Project Executive)

Martin Eveleigh (JSF Programme Management / F120 Project
Engineer)
Alan Bye (IPM/SAP)
Dave Inman (JSF F120 Programme Manager / Assistant Chief
Engineer)
Nick Millar (F120 Fan Systems CAM)
Hamish Constable (F120 Transmission CAM)
Dave Hopkins (F120 Systems Engineering CAM)
Gary Selby (F120 Programme Management CAM / PBT)
Bob Ford (Ops Controller Budgets/Resources)
Martyn Brown (JSF Project Accountant)
Mike Bryant (Fan Systems Function Head RM)
Gary Davis (Fan Systems RM)
And my boss said
John Fitchett (JSF Procurement Executive)
I could do this
Dave Luscombe (Procurement)
alone?!
Bob Alcock (MRP Controller)
AzTech International EVMS Consultants
!
Luis C Contreras
!
Oscar Banda
7

DoD EVMS Validation - Why did Rolls-Royce seek it?

!

Integrated Programme Management

culture (processes, SAP tool, training,
need) was already underway, but:
! Validation is recognition of
achieving project management
Best Practice
! Future contractual requirement

The
contract
required it.

EVMS Validation Why did Rolls-Royce seek it before

a contract (Phase 3b) required it?
!
!
!

"

To be pro-active
To mitigate risks
Felt that starting implementation prior
to contract award would minimize
impact to initial project planning

The
contract
required it.

Forced issue to make Phase 3a

contract require full EVMS
compliance

Rolls-Royce EVMS Implementation: 18 months

from kick-off through Initial Compliance Review

Did someone say

we could do it in
3 months?@!#

Implementation Milestones
!

!
!
!

!
!

!
!

18 Months In Preparation - (F120 Phase 3a after seven

months)
4 DC Summits
Jan 2000 Kicked-off in Washington DC
Feb 2000 Initial RR-Allison/AzTech International
involvement
May 2000 JSF JPO/NAVAIR/DCMA Interim Progress Visit
Sep 2000 JSF JPO/NAVAIR/DCMA/UK MoD/GE/P&W
Progress Assessment Visit (PAV)
Achieved DR Closure Agreement - January to April 2001
May 2001 JSF JPO/NAVAIR/DCMA/UK MoD/GE/P&W
Initial Compliance Review (ICR)
September 2001 - Formal Acceptance Letter

Isnt this
overkill?

11

Why didnt Rolls-Royce Self-Certify?

!

!
!

!
!

Relatively new to EV; wanted outside,

unbiased observations (yes, audit)
Avoid inherent bias/subjective opinion
DoD compliance is a globally marketable
recognition
Potential scope/cost growth from third-party
consulting
Self-certification might be poorly regarded
Could not proclaim world-class
benchmarking unless a formal validation
Wanted people (CAMs, PMs, internal
functions) to face the impact of proving the
system to drive the point home that EVMS is
important and here to stay

Nobody said
we could do
that!@#

Yeah, right. as
if you could
have even if
you wanted to?

12

Pre-Review Process
!

!
!

Pre-Contract Award Meetings with

Government/Customers
Why go to all that
Decided to use GE F120 Phase 3A
trouble for such a
tiny contract?
Wanted to get system in place with a
manageable project:
! $5M contract
! 5 CAMs
! Period of Performance - 16th Oct 2000 to
15th Nov 2001
! Six (6) Control Accounts
! Mostly Engineering with minor Tooling &
Manufacture

13

Innovations / Implementation Tools

!

Stanmore Directory
Set up a folder on the LAN with subfolders for
!
all Project data including:
! MS Project files
! MS Excel files
! WBS Dictionary, SOW
! Purchase Orders
! Training Materials
! System Description/Instructions
Had laptops with the data available to Review
Team and to Project Team (PM, CAMs, PBT).
Electronic System Description

14

This data not available to Primes

Stanmore Directory - F120 P3a

15

Review Process
!
!

Before contract was in place

SAP (Project Systems Module and other modules
already in place)
Requirements Assessment: AzTech International &
Rolls-Royce North America (Allison) EV expert
brought in for gap analysis
Used validated system design (documentation)
!
from Rolls-Royce Allison (Indianapolis) as a
starting point
Decided to re-write around Rolls-Royce
!
Defense Europe organization and SAP
functionality/processes
Identified typically difficult areas (Accounting,
!
MRP, Baseline Control, Scheduling)
Identified potentially weak SAP areas
!

Why reinvent the

wheel?

Had to replace
those #$*&
Zs with Ss!

16

On-Site Review Process

!
!
!

Advanced Visit
Progress Assessment Visit
Initial Compliance Evaluation
! Purchasing/Manufacturing
demonstrated from other programs;
some pending for Production;
mitigated.

17

On-Site Review Process

18

Standard On-Site Review (Old School

Implementation/Evaluation Tools Used):
!

Same old
same old

!
!

Were not
telling you
what to say,
but
!

Storyboard Data Traces, Exhibits, flow, binders

don't work; Index to Storyboard and
SD/Instructions; Links between index: easier to
present; Easier to review; Used to audit the SD
pages, references, etc.
CAM Notebooks - Used them; encouraged to use
them.
CAM Storyboards
Interview preparation - no canned answers; did
pass out questions, then answers and went over
them with CAMs (all together or one-on-one).
MS Access Database for generating Control
Account & Activity (Work Package) Statistics;
other data validation reports

Do CAMs ever
really use these
Notebooks?

SAP cant
do that!!??

19

Post-Review (following May 2001 Review)

!

Incorporated System Description /

Instructions document changes
Issued CD ROM with new
documentation to Government
Review team members
Internally, reviewed Lessons Learned
from implementation process

20

Major Reasons for Success #1: Company Commitment

!

!
!

!
!
!

Rolls-Royce Integrated Programme Management (IPM) Initiative:

Fostered corporate culture change (already started when
!
SAP was implementedbefore EVMS requirement)
Rolls-Royce JSF Senior Management set a priority to meet the
future contract requirement
Senior Manager EVMS sponsorship
Key Players:
EV focal point for implementation
!
Released from prior duties
!
Given budget, responsibility, & authority
!
Tool (SAP) Expert
!
Hands-on PM
!
Dedicated CAMs who got involved
!
MRP, Finance, Purchasing, & Other Functions got involved
!
Outside EVMS Consultants
!
Refresher SAP & EVMS training; One-on-one support
Rolls-Royce, top to bottom, saw the benefits
Rolls-Royce did not aim to just pass or to please

Future
programme
required it.

Sometimes
had to beg
for support.

21

Rolls-Royce Vision for

Programme Management
!
!
!

!
!
!
!

State of the art Programme Management

Electronic System - paperless
Integrated data - Customers, Partners, and
Suppliers
Universally recognized best practice
Create system that exceeds the JSF requirements
Works both for current & future programmes
Accreditation is secondary. Rolls-Royce seeks
A system that works for CAMs
!
A system that works for all programmes
!
Substantiated historical and projected data
!
An effective management tool
!

Paperless hasnt
happened yet

On-going
process

22

Major Reasons for Success #2: Communication &

Direction
!

Expectations were clear

Requirements were understood
!
Map for success
!
Hundreds of eCommunication Tools:
mails. Time
Teleconferences
!
zone differences
were a pain
Video conferences
!
Face-to-face Meetings
!
E-Mails
!
Memo to Bill Gates:
Document versions routed to all team
!
MS Word Document
members on a regular basis
Compare has bugs
Direct e-mail and phone questions and
!
answers between Process Group leaders from
Government/Customer Review Team and
Rolls-Royce
23

Major Reasons for Success #3: Government &

Customer Approach
!

Brought in experts as enablers, not

dictators
Process Group leaders had time
limits for reviewing and responding to
inputs and DRs
Quiet discussions rather than
showdowns

How many times

did we debate
the use of 50/50
method?!!#@

24

Contractor Lessons Learned

!
!
!
!

!
!
!
!
!

!
!
!

M -> J

Tell it like it is
Use the government. DoD Review teams are free consultants
Think: What are the Benefits to OUR Company?
Implementation Team members will not always be there after
implementation, so it is important to spell out as much as you
Free? What about
internal implemention
can.
effort, system design,
Review all standards documents
consulting,
Ask other companies.
Ask other review team experts from the government agencies
It is a big job, but use it to your company's advantage
Respond to all DRs/Issues. Don't necessarily agree, but definitely
Respond.
Neither member can have oversized ego about what's RIGHT
Focus on high value, high impact Control Accounts
Share tools between CAMs

25

Initial Compliance Review Team Goals

Assess Rolls-Royce Bristol compliance with the Industry Guidelines
! Management system design and operation
Prepare evaluation report
! Document compliance using the EVMIG Appendix B as a guide
Present findings/recommendations to Rolls-Royce Bristol
Establish process for closure of outstanding issues and Acceptance

!
!

All-expense
paid trip to
Europe!

26

Initial Compliance Review Team

Team Member

Team Assignments

Organization

Gayle Brooks
Lt. Col Paul Waugh
Jim Henderson
Kristen Kehrer
Monique Anneker
Harvey Johnson
Dave Kester
Rocky Galletta
Bill Jackson
Terry Bowd
Lorene Brown
Mike Lowry
Stephen Pears
Judith Yeaman
Bruce Campbell
Bev Deachin
Dennis Chapman
Ed Swain
Barbara Dobbins
Bob Gall
CDR Marion Eggenberger

Co-Review Director
Co-Review Director/Deputy PM
Team Leader
Assistant Team Leader
Organizing Process Leader
Organizing Process
Scheduling Process Leader
Scheduling Process
Work/Budget Authorization Process Leader
Work/Budget Authorization Process
Work/Budget Authorization Process
Accounting/Indirect Mgt Process Leader
Accounting/Indirect Mgt Process
Accounting/Indirect Mgt Process
Managerial Analysis Process Leader
Managerial Analysis Process
Matl Mgt/Change Inc. Process Leader
Matl Mgt/Change Incorporation Process
Matl Mgt/Change Incorporation Process
Subcontract Management Process Leader
Subcontract Management Process

DCMA
JPO
NAVAIR
NAVAIR
NAVAIR
DCMA
NAVAIR
CA MHP
GE
UK DPA SPS
NAVAIR
DCMA
UK DPA SPS
UK DPA SPS
GE
P&W
NAVAIR
GE
GE
P&W
DCMA
27

Validation Process Benefits

!

Beats holding
your thumb to
the sky...

Provides the buyer:

! Early identification of supplier management system issues and
risk reduction
! Verification that the suppliers management system meets the
intent of the EVMS guidelines
! Confidence that the supplier has and uses effective internal
management control systems
! Special emphasis on schedule networks and resource
management
Provides the supplier (contractor):
! A non-biased assessment of its management system
! Recognition for achieving rigorous business disciplines

28

May 2001 Initial Compliance Evaluation

!

Team Preparation:
# Developed review planning schedule
# Reviewed system description
# Worked through Progress Assistance Visit (PAV) issues prior to
Initial Compliance Evaluation (ICE)
# Developed team handbook with review methodology
# Conducted team training
# Cooperation between Government & Industry team was key
Conducted on-site review:
# Documented system description compliance
# Presented story board system traces to review team
# Evaluated compliance with CAM interviews, documentation
verification, and traces
So much for
glamourous
# Prepared evaluation report
European
vacation...

29

Government Lessons Learned

!
!
!
!

!
!
!

J -> M

Isnt this
common
sense Project
Management?

Communication and co-operation between industry & government

are keys to success
Establish clear roles and responsibilities
Emphasis placed on documenting and following up on all findings
Set and revisit deadlines
Government PM Co-Review Director provided leadership and
program perspective
Build consensus
! Agree on the intent of EVMS Guidelines
Ensure review team has requisite expertise
Address all relevant management system characteristics
Maintain a positive approach
30

System Description / Instructions

!

System Description:
! Process-oriented
! Includes guidelines and grey areas - for
PMs PMO, to resolve future conflicts
! 90% non-tool-specific
! Addresses management system
characteristics (App. B of EVMIG)
Instructions:
! Roles & Responsibilities
! Step-by-step, more detail, mechanics for
CAMs
! Essential for new CAMs
! More likely to change down the road
! More tool-specific
31

System Description / Instructions - Innovations

!
!

!
!
!
!
!

Screen shots with SAP menu paths

Advanced Scheduling Methodologies
& Analysis
Resource Profiles (Curves)
Risk Management
Earned Value tools
Advanced Variance Analysis
MRP

32

New & Improved Digitized System Description

!
!
!
!
!
!
!
!
!
!
!
!
!

Home Page - Web look-alike

Links to System Description
Links to Instructions
Links to Exhibits
Links to Appendices
Links to DoD Reports
Links to Index
Colour-Coded Sections
User Friendly
Excellent Training/Reference Aid
Meets the goals set internally by Rolls-Royce
Held on Stanmore - F120 Team Usage
CD ROM
33

Electronic EVMS Home Page

34

Electronic EVMS System Description

35

Electronic EVMS Instructions

36

Electronic EVMS Appendices

37

Electronic EVMS
DoD Reports

38

System Description / Instructions - Lessons Learned

!

!
!
!
!
!
!
!

Define Table of Contents early on and

include all subsystems in best order possible
One document
Show integration and importance of System
All contracts
Consistency of best practices
Involve Ownership
Make it useful for new users (staff changes)
Do not be afraid to change it; it must and it
will as subsystems evolve and improve
Integrate the Tool (i.e., SAP) when it
enhances usability

But will anyone

even read this
tome?!

39

Expert Support
!
!

!
!

Worked closely with Rolls-Royce EVMS Focal Point

AzTech International brought in past experience with
other industries, government agencies, and past
validations
Initial Evaluation Where we stood; Weak areas
Developed System Description / Instructions merging in
R-R and Review Team comments
Conducted EVMS Training tailored to SAP and other inhouse tools
Developed tools for work-rounds to SAP (Excel and
Access)
Worked one-on-one with Project Team (PM, CAMs,
PMO and other interviewees: Accounting, Finance, MRP,
Purchasing, Contracts, etc.)
Helped negotiate Discrepancy Report/Findings
responses

Consulting
scope creep...

40

SAP Innovations - Why did Rolls-Royce choose SAP?

!

!
!

M->L

SAP (now at Version v4.0b) was implemented as part of the Rolls-Royce IPM before EVMS
requirement came in
All planning, earned value, actual cost collection, and forecasting done at the Activity (work
package) level
Roles - PM, RM, CAM roles required within the tool
One-stop
shopping...
CATS - Cross-Application Time Sheet
Each Activity is automatically a charge number
!
Each Activity can have labor, purchase orders, or manufacturing orders against it
!
CAMs can drill down to see cost details
!
MRP - Material Requirements Planning
Ability to drill down to the manufacturing operations or the purchase order detail
!
Project Systems
WBS down to Control Account levels
!
Activities (Work Packages) linked (cost and schedule integration)
!
Planning & Forecasting: Resource management & Resource Profile Curves
!
Earned Value
!
Aimed to be able to do weekly Earned Value
Most costs accumulated daily/weekly
!
Earned value and forecasting can be done daily/weekly
!

41

One-stop
shopping?!

Rolls-Royces Earned Value Management System

MRP

Variance
Analysis
Reports
(VARs)

Finance

SAP
Purchasing

Modules
(Among
others)

Project
Systems (PS)

Cost
Performance
Report (CPR)
Formats

Contract Funds
Status Report
(CFSR)

Customer Reporting
Schedule export
SAP

CATS
(Timesheets)
Initial
Planning
(iterations)
L

Earned
Value
Worksheets

Database for
Control Account
& Work Package
Statistics

MS Excel
Legend:
MS Project
MS 42
Access

Tool Discussion: SAP & Other Tools

!
!
!
!

Not an "SAP" validation

SAP representatives had no participation in the review
1700 legacy systems eliminated
PS, MRP, CATS & other integration easily outweighed
lack of reporting and other deficits
Some non-SAP form and reporting tools:
!
Control Account Authorisations
!
MS Project Schedules for baseline input
!
Excel Variance Analysis Reports
!
Excel Earned Value Worksheets
!
Excel CPR Worksheets; DIDs: CPR, CFSR,
CCDR;Replanning; Adders (C&A, COM, Fee)
!
Excel Baseline Logs, RAM
!
Programme Change Requests

No CPR?!!!

43

Needed SAP Enhancements (Wish List)

!

!
!

!
!
!

L -> M

Next release
will do all
thisNOT!

Required Reporting that most commercial EV applications easily do:

US DoD Reporting
!
!
Cost Performance Report (CPR) Formats (Five of them)
1 - WBS, 2 - Function, 3 - Baseline, 4 - Manpower Loading, 5 - Problem
!
Analysis
ANSI X.12 Export
!
Variance Analysis Reports (VARs) at Control Account WBS (and even at Summary WBS
!
Elements)
Variance Thresholds
!
!
Amount (Cost) and % for Current and Cum SV, CV and for VAC
Easily Export Time-Phased (by A/P) BCWS, BCWP, ACWP, and ETC in Hours and Cost by WBS
Export to WinSight (A package made by C/S Solutions that the DoD Analysts use and often flow
down to contractors/subcontractors)
Easy Labour Rate & Usage Variance Analysis
Easy Material Price & Usage Variance Analysis
MRP Earned Value - automatically summarize % complete from underlying manufacturing
operations and purchase order details
Most EV applications do not do these, but it would be awesome if SAP could generate:
Contract Funds Status Report (CFSR)
!
Contractor Cost Data Report (CCDR) - Requires summary by WBS and by Function and Cost
!
Element (e.g., Engineering, Manufacturing, Tooling, Labour vs. Material, etc.)
!
1921, 1921-1, 1921-2, 1921-3

44

Cynics Challenges
!
!
!

!
!

Can we maintain system discipline?

Can we keep project team trained?
Will all projects that require it adopt full
EVMS?
How will we incorporate SAP Version
Changes?
How will we conduct internal surveillance?
How can we work with the government and
customer to maintain and improve the
system?
What happens when best Team Members
move on?

45

Contact Information

Martin Eveleigh (Rolls-Royce)

513 243.3675
!
Martin.Eveleigh@ae.ge.com
!
Jim Henderson (NAVAIR)
301 342.2392
!
HendersonJH@navair.navy.mil
!
Luis C Contreras (AzTech International)
949 249.9349
!
AzTech@pobox.com
!
http://www.goAzTech.com
!
46