Gartner Final Health Check Report and Recommendations

Independent Health Check of
DMV System Modernization

Project
Final Health Check Report
and Recommendations
29 January 2018
CONFIDENTIAL AND PROPRIETARY

This presentation, including any supporting materials, is owned by Gartner, Inc. and/or its
affiliates and is for the sole use of the intended Gartner audience or other intended
recipients. This presentation may contain information that is confidential, proprietary or
otherwise legally protected, and it may not be further copied, distributed or publicly
displayed without the express written permission of Gartner, Inc. or its affiliates.
© 2016 Gartner, Inc. and/or its affiliates. All rights reserved.
Table of Contents
1. Executive Summary
2. 120-Day “Get Well” Plan for the SysMod Project
3. Analysis of Three Go-Forward Options
4. Detailed Findings and Recommendations
5. Appendix A: Project Health Assessment Approach and Timeline
6. Appendix B: Risk Category Definitions
7. Appendix C: Interviewees and Documents Reviewed
8. Appendix D: Sample Position Descriptions
Engagement: 330045594
Gartner is a registered trademark of Gartner, Inc. or its affiliates. 2
Executive Summary
Bottom Line Upfront
NV DMV’s SysMod Project is one of the most troubled public sector modernization efforts that Gartner
has seen in recent years and is at high risk of failure without an overhaul in many key areas
§ Unproven DMV system solution, with no known successful implementation in a similar environment
§ Nonexistent project management controls and discipline on both the DMV and Vendor sides, with no
approved project schedule until October 2017 — nearly 1 ½ years into the project
§ A low performing vendor to date, having produced low-quality deliverables, an unrealistic project schedule,
and provided resources who either lacked core skills or simply could not drive work forward effectively
§ No formal definition of business benefits to map system functionality back to expected results
§ Low team morale and lack of communications/organizational change management, leading staff to
believe the project is at high risk of failure
Based on Gartner’s experience, projects with these types of attributes typically

go in one of the following directions
§ Software is forced into production based on an arbitrary date in order to publicly claim success, with little
tangible business benefit, and in many cases, severe disruption to the business
§ The project continues on with ever-changing end dates and vendor change orders to adapt to emerging
business requirements that were not initially captured at the outset of the project
§ End users continue to use the old system in parallel with the new one in order to validate data and perform basic
legacy functions.
§ In order to shore up missing components, such as missed requirements, augmentation of staff and a re-
baselining of the plan — even if it requires creating a project pause
Final Report Context
Gartner’s initial assessment of SysMod Project provided DMV leadership with an early, clear-eyed
assessment of the challenges facing the project. This has culminated in the Final Report, where Gartner
provides go-forward options and actions for DMV to take in order to get the SysMod Project back on track.
1 2
A graphic depiction of the current Detailed findings mapped against

state of SysMod across 4 key areas Gartner project best practices
3 4
Options for Moving Forward on the Final Report & Detailed

System Modernization Project Recommendations
Executive Summary
Current State of the SysMod Project
In late November 2017, Gartner conducted interviews with nearly 40 members of the SysMod team, including
representatives from the Nevada DMV, as well as its system integration vendor. Based on our assessment
framework which looks at 22 project attributes, Gartner assesses that in 15 areas, the project is at high risk of failure.
Executive Summary
Current State of the SysMod Project
1. Technical Solution and Scope
§ The software solution selected for SysMod (leveraging Oracle Enterprise Business Suite and Siebel Customer Relationship
Management) is an unconventional and unproven technology choice to support the business processes of a typical state
motor vehicle agency
§ Gartner is not aware of a single instance where this solution strategy has been successfully deployed by another state
§ The vendor has not conducted a comprehensive requirements validation and fit-gap analysis to demonstrate the degree
of alignment between its proposed solution and SysMod requirements
§ The decision to acquire hardware for an on-site hosting environment (vs. a cloud-based option) has already begun to lock the
DMV into a costly, long-term obligation for maintenance and support of the SysMod solution
2. Project Management and Governance
§ Project management by both the DMV and the vendor leadership teams has been abysmal
§ The DMV’s PMO permitted the vendor to proceed without an approved project schedule for nearly 1.5 years after the
kickoff — therefore, it had no clear plan or tool to:
• Organize tasks and activities
• Determine project staffing needs and assign resources to tasks and activities
• Track project progress
§ Critical planning documents have still not been submitted and approved, including:
• Resource Management Plan
• Change Management Plan and Control Procedures
• Risk Management Plan
• Information Security Plan
• Data Conversion Plan
Executive Summary
Current State of the SysMod Project (Cont)
2. Project Management and Governance (Cont)
§ The lack of a viable governance framework is problematic:
§ No clear direction — or accountability — is defined regarding decision making rights, responsibilities, processes and
documentation standards at all levels of the project
§ It was unclear, due to the lack of a governance plan, who is responsible for making decisions for the direction of the
project
§ There is an executive committee composed of DMV executives and administrators, however, they have not met to make
any decisions at this point
3. Vendor Performance Management
§ The vendor’s overall performance to date has been poor
§ Vendor is very late in producing key planning deliverables defined in its Scope of Work
§ The project schedule (which was just approved in October 2017) is already outdated and no longer relevant
§ Deliverables that the vendor has produced often are of poor quality and must undergo numerous review cycles prior to
acceptance by the DMV
§ Many key staff resources identified in the vendor’s proposal were not assigned to the SysMod project or have rolled off
the project already
§ Many of the resources that the vendor has assigned to the project lack acceptable written and verbal communication
skills, as well as appropriate subject matter expertise in the technology products that the vendor has proposed
Executive Summary
Current State of the SysMod Project (Cont)
4. Requirements Management and Benefits Realization
§ The quality of functional requirements prepared by the DMV for the SysMod project, and the manner by which these
requirements have been validated and managed by the vendor, has been fair
§ Many SysMod requirements are nested and would need significant rework to decompose them to support development
and testing
§ The vendor has not conducted a complete validation and fit-gap of requirements with respect to its solution strategy to
determine software configuration and customization needs
§ In instances where a requirement has been added or updated, these changes have not been recorded in a centrally
managed Requirements Traceability Matrix (RTM)
§ Quantifiable benefits of the SysMod project have not been formally defined and monitored — or linked to requirements to
ensure that the project actually achieves its objectives and meets stakeholder needs
5. Communications and Organizational Change Management.
§ Although the DMV has reviewed and approved the vendor’s Communications Plan, like many other deliverables, this
document was late in delivery — over a year after the project kickoff
§ The DMV has not developed an Organizational Change Management (OCM) plan to help ensure that its stakeholders are
appropriately prepared and have realistic expectations for using the new system
§ Neither the vendor nor the DMV have conducted a formal stakeholder assessment to determine end user needs with respect
to communications, OCM and training strategies
Risk Level
High Risk of Failure
Significant Risk
Executive Summary Manageable Risk
Recommendations to Address the Findings Managed Risk
1. Technical Solution and Scope

A. Technical Solution
§ Revise Business Case to include metrics for tracking anticipated benefits and an explanation
for how this will be done
1. Business Case
§ Perform a cost-benefit or ROI analysis, including total cost of ownership calculations,
comparing on-site solution vs. cloud-based solution 2. Technology Infrastructure &
Process Strategy and Planning
§ Before deciding on the technology architecture, consider whether DMV can acquire and keep
technical staff with skills needed to maintain and support the solution 3. Security Planning and Execution
§ Cease all development work until an Information Security Plan and Security Architecture are
4. Data Conversion Planning and
accepted by the DMV Execution
§ Immediately develop and finalize key deliverables — including Data Conversion Plan, 5. Integration/Interface Planning
Interface Architecture, Solution Architecture and Transition Approach and Implementation
6. Operational Transition Planning
B. Project Management and

2. Project Management and Governance Governance
§ Strengthen executive support through improvements in project governance and by
1. Executive Support
formally defining leadership involvement in OCM and Communications Plans
§ Develop a comprehensive governance framework that includes an Executive Steering 2. Program/Project Governance
Committee with a clear charter and a RACI/RAID matrix to define decision making
authority and responsibility at all levels of the project 3. Risk Mitigation Strategy and
Management
§ Reassess and streamline processes for reviewing and approving (or rejecting) project
deliverables 4. Schedule Management
§ Immediately develop and finalize key deliverables — including Schedule Management 5. Budget Planning and
Plan, Resource Management Plan and Change Management Plan Management
§ Reorganize, realign and streamline DMV’s PMO and overall project organization to 6. Resource Planning and
Management
optimize staffing levels and capabilities
Risk Level
Significant Risk
3. Vendor Performance Management

§ If DMV determines it must rebid SysMod system implementation services the Department C. Vendor Performance
must more strongly weigh a vendor’s prior experience and performance in similar system
modernization projects in its evaluation model
1. Vendor Management Strategy
§ DMV should reference independent research reports for additional insight into the
capability and vision of potential vendors
2. Vendor Performance
§ Restructure the vendor’s Statement of Work to provide stricter project controls and Management
performance expectations, including compliance with schedule and quality requirements 3. Development Planning and
§ Schedule and conduct periodic review sessions to address vendor performance issues Execution
and risks
4. Test Planning and Management
§ Immediately develop and finalize key deliverables — including Development Management
Plan, Master Test Plan and Training Plan
5. Training Strategy & Planning
4. Requirements Management and Benefits Realization

D. Requirements Management and
§ Rewrite requirements so that they meet accepted industry standards (e.g., IEEE) for Benefits Realization
clarity and testability and validate requirements with DMV business and technical staff
§ Vendor must conduct a comprehensive fit-gap analysis and document results in a 1. Scope Definition and Refinement
centrally managed Requirements Traceability Matrix (RTM)
§ The vendor must establish processes and tools, subject to DMV review and approval, for
2. Requirements Management
centrally managing all requirements for the new project
§ DMV needs to clearly define anticipated benefits and determine how benefits will be 3. Benefits Realization Planning and
measured over the course of the project Execution
Risk Level
Significant Risk
5. Communications and Organizational Change Management

E. Communications &
§ DMV should conduct a stakeholder assessment and the results used to drive a Organizational Change
redesign of the current communication plan, as well as provide better direction for Management (OCM)
OCM and training planning and execution
1. Communication Planning and
§ Project updates should be regularly updated and disseminated and Execution
approaches/timelines should be followed by all stakeholders
2. OCM Planning and Execution
§ DMV needs to prepare an OCM plan that is based on results from a stakeholder
assessment
Executive Summary
Pause the Project, Fix Core Issues, Restart Execution
Gartner has identified three options for DMV to consider in moving forward with the SysMod
project. Regardless of which alternative is chosen, Gartner recommends DMV take a 120 day
1
“Get Well” pause and correct some foundational issues identified in this assessment.
Implement these Reassess & Renegotiate.

foundational fixes during the Reassess the vendor’s current Statement of
pause: Work & revise to establish stronger project
controls, improved accountability for vendor
Options
1. Restructure the DMV PMO and performance and restructured deliverable
project staffing model. Establish and payment schedules.
2
core Project Management
discipline: processes, tools &
templates. Re-procure.
Forward
Take revised requirements and craft a
2. Establish project governance new RFP and award to another vendor.
bodies & processes. Develop a
Institute a “get well project charter focusing on
pause” on the SysMod expected business benefits.
3
Project for up to 120
days Go
3. Revise and validate current
SysMod requirements. Transfer.
Based on revised requirements,
conduct a fit-gap analysis with AZ
MvM or other states solution and
transfer to NV.
Executive Summary
Three Options to Consider
Go Forward Options High-Level Actions Pro’s Con’s
Reassess & § Fit-gap to revised/validated § The vendor is already under § Could negatively impact staff morale
Renegotiate. requirements contract — can get restarted quickly and commitment to the project if
Reassess the vendor’s if the vendor agrees to new they perceive little has changed
§ Reassess solution strategy
current Statement of Work contract/SOW and staffing
§ Prior track record for failing to
& revise to establish § Renegotiate vendor contract, SOW, requirements deliver suggests the vendor may not
stronger project controls, and demand A-Team
have the ability to course correct —
improved accountability
even with revised SOW/contract
for vendor performance
and restructured § Uncertainty that the solution is
deliverable and payment viable and not in-line with State’s
schedules. strategic shift toward cloud
Re-procure. § Terminate vendor contract § Opportunity to select solution and § $27M invested in the vendor’s
Take revised vendor based on clearer project would be sunk cost
§ Develop and release new RFP
requirements and craft a requirements
§ Minimum of another year to get
new RFP and award to § Procure services of another
§ Ability to apply lessons learned from through project pause and procure
another vendor. vendor/system integrator and
vendor procurement to get DMV services of a new vendor
negotiate a contract
more favorable terms and pricing
§ Additional cost for new solicitation
and possibly higher cost for new
vendor and solution
Transfer. § Conduct a site visit of AZ MvM § Potential for millions of dollars in § $27M invested in the vendor’s
Based on revised project office cost savings to the State project would be sunk cost
requirements, conduct a
§ Conduct Fit-gap of AZ MvM (or other § MvM solution can be delivered
fit-gap analysis with AZ
State) solution to revised/validated quickly and likely at lower cost than
MvM solution and transfer requirements Options 1 or 2
to NV.
§ Examine contracting vehicles § MvM is a Microsoft-based solution
available to procure services of — complements current NV DMV
integration vendor(s) skill set today
§ Terminate vendor contract
120-Day “Get Well” Plan for the
SysMod Project
SysMod Project “Get Well” Pause: Timeline for the Next 120 Days
February 2018 March 2018 April 2018 May 2018
W1 W2 W3 W4 W5 W6 W7 W8 W9 W10 W11 W12 W13 W14 W15 W16
1. Restructure DMV PMO and Project Staffing Model
2. Establish Clear Project

Governance
3. Revise and Validate Requirements
Foundational Activities Conduct Fit-Gap to Assess Define Staffing, Contract &

Proposed Solution Schedule Requirements
Optional Paths
(Options 1 and 3) (Options 1 and 3)
Begin Preparing RFP for New System Integrator and Solution

(Options 2)
120-Day “Get Well” Plan Activities
Task 1: Restructure the DMV PMO & Staffing Model
OVERVIEW: Establish organizational structure to provide project leadership and staffing and define and leverage processes, governance
provisions, plans, templates and tools necessary to effectively and efficiently manage the SysMod project
DURATION: DELIVERABLES/OUTCOMES:
§ 12 weeks § PMO Charter
§ Updated Business Case for the new project (including quantifiable
KEY ACTIVITIES: target benefits)
§ Quickly assess existing PMO practices in place today and use § PM tools and templates
processes/tools/templates that work and make sense § Project Plan — including:
§ Document appropriate PMO structure for DMV and work with DMV § Preliminary Schedule
leadership to map DMV staff to PMO roles § Scope Management Plan
§ Develop charter to formally define the PMO’s mission, § Schedule Management Plan
organization, key responsibilities and authority § Budget Management Plan
§ Revise the SysMod business case to clearly outline business § Change Management Plan
benefits for the new project and measurements of success for the § Resource Management Plan
project § Quality Management Plan
§ Identify and acquire or adapt PM tools and templates (e.g., to § Risk/Issue Management Plan
support collaboration, executive dashboards, documentation) § Analysis and Reporting Standards
§ Develop project staffing structure and confirm resource needs, § Project Staffing Model
including individuals for the following roles: § Governance Framework
§ Project Manager
§ Assistant Project Manager
§ Contract Manager
CRITICAL RESOURCE REQUIREMENTS:
§ Project Coordinator
§ Business Analysts § Program Manager must have prior experience leading a project of
§ DMV Subject Matter Experts comparable scope and complexity. See Appendix for a description
of this position.
§ DMV Team leads (Technical, Data Conversion, Testing,
Communications, OCM, Training) § Business Analysts should have demonstrated skills in requirements
definition/use case development, process analysis, data modeling,
§ DMV Functional Leads (multiple — to represent various
business functions) etc. See Appendix for a description of this position
§ Prepare Project Plan (includes preliminary schedule and plans for § Subject Matter Experts (SME) should be carefully selected based
managing scope, schedule, quality, budget, resources, on a balance of specific knowledge/expertise and ability to think
risks/issues) outside the box.
§ Define project analysis and reporting requirements to track and
communicate
Engagement: progress vs. plan
330045594
Examples of Project Management Tools, Practices and Capabilities
Table of Contents
1.0 PMO Statement of Purpose .................................................................................. 2

1.1
1.2
Mandate ...................................................................................................................2
Organizational Context and Executive Sponsorship .................................................2
The Project Charter
2.0 Goals and Objectives ............................................................................................ 3
3.0 Benefits .................................................................................................................. 6
authorizes and
4.0 IT PMO Scope ........................................................................................................ 7 communicates the
4.1 Services ...................................................................................................................7
4.1.1 Facilitate Governance for IT Project Demand ..............................................8 goals, objectives,
4.1.2
4.1.3
Communicate IT PMO Services ..................................................................9
Deliver IT PMO and Project Performance Metrics ..................................... 10 services, processes,
4.1.4
4.1.5
Provide Up-to-Date Project Management Methodology & Content ............ 11
Manage Approved IT Projects ................................................................... 11
and activities
4.1.6 Provide Knowledge Management Guidance to Capture IT Project
Deliverables .............................................................................................. 12
necessary to achieve
4.2
4.1.7 Provide IT PMO and PM Training .............................................................. 12
Out-of-Scope .......................................................................................................... 13
the desired “target
5.0 IT PMO Organization Structure .......................................................................... 14
5.1 Roles, Responsibilities, Accountabilities and Authorities ........................................ 14
state.”
6.0 IT PMO Stakeholders / Relationship Management............................................ 16
6.1 Business Stakeholders ........................................................................................... 16
6.2 IT Stakeholders ...................................................................................................... 16
6.3 External (non-Rex) Stakeholders ............................................................................ 17
7.0 IT PMO Governance............................................................................................. 18
7.1 What Constitutes a Project, Portfolio and Program ................................................. 18
7.2 Project Complexity Matrix ....................................................................................... 18
7.3 Governance Roles..................................................................................................19
8.0 IT PMO Metrics ..................................................................................................... 21
9.0 IT PMO Implementation Roadmap...................................................................... 23
9.1 Maturity Evolution ................................................................................................... 23
9.2 Near-Term Roadmap ............................................................................................. 23
9.3 Medium-Term Roadmap......................................................................................... 25
9.4
9.5
Long-Term Roadmap ............................................................................................. 26
Key Identified Risks ................................................................................................ 27
The project governance structure will be formalized in the
10.0 Reporting .............................................................................................................. 29 Project Charter. Each governance committee will have its
Appendices .................................................................................................................. 31
own charter to formalize purpose, membership, decision-
making models, meeting frequency and other key items.
Project Processes (e.g., risk/issue management, change

management) are documented. These would be
documented as part of the Project Management Plan
(Phase II).
Task 2: Establish Clear Project Governance
OVERVIEW: Develop a Governance framework for the SysMod Project to help ensure that:
1. Project level decisions are made by the individuals or groups who have the appropriate information and authority.
2. Actions and behaviors of all project staff (DMV and vendor) and stakeholders are aligned with project objectives, as well as
defined governance standards and principles.
3. Decisions are made quickly, yet prudently.
§ 4 weeks § Governance Framework
§ RAID or RACI matrix
KEY ACTIVITIES: § Executive Steering Committee Charter
§ Prepare overarching governance framework document to formally define the
following:
§ Guiding Principles
§ Key Decisions (e.g., deliverable acceptance, change requests, etc.)
§ Governance groups and individuals
§ Decision making processes
§ Documentation standards and processes
§ Establish and name individuals to governance groups, including:
§ Executive Steering Committee
§ Project Management Office
§ Ad hoc groups — for example: CRITICAL RESOURCE REQUIREMENTS:
• Standing Architecture/Modification Review Board § Executive Steering Committee should be composed of
• Business Process Review Board between 3–5 senior level business and IT leaders
• Policy/Standards Review Board representing the DMV
§ Define decision making protocols § The PMO to be assigned responsibility for developing
§ Prepare RACI or RAID matrix to assign decision making authority and maintaining the governance framework and ensuring
§ Prepare Executive Steering Committee Charter to outline group organization, it is followed
authority/responsibility, decision making protocols, documentation standards,
meeting frequency, etc.
§ Develop plan for communicating/sharing governance provisions with entire
project team
Example Project Governance RACI Matrix
Executive Architecture Review
Steering Committee Project Sponsor Program Manager State IT Oversight Board
Decision Domain/Governance Participant
Project Management
Budget/Cost (changes) A R C
Approach (initial + changes) A R
Schedule (baseline) A C R
Schedule changes - minor/non-cost I C A
Schedule changes - material A C R
Assignment of Staff/Personnel I A C
Scope of Work/Functional Requirements (changes) A R
Technical Deliverables
Solution Architecture (approving) A R C R
Hardware/Software (selecting) A R C R
Interface Design (approving) A R C R
System Design Modifications (approving) A R R
Other Key Deliverables
Quality Phase Gate Exit Document (approving) A R
Training Strategy/Plan A C R
Deployment Strategy/Plan A C R
Other Deliverables - Major A R
Other Deliverables - Minor I C A
Configuration and Release Management
Release Management Plan A C R
Configuration Management Plan I A
Issue and Risk Management
Issue Resolution - Major A C R C
Issue Resolution - Minor I C A
Risk Mitigation - Major A C R
Risk Mitigation - Minor I C A
Final Acceptance of mDL Solution
Final Acceptance A R C
KEY The sample Decision Matrix above indicates how key decisions will
R = Recommend
A = Approve/Accountable
be made by each individual or group involved in a project, as well
C = Consult as those who provide input or will simply be informed.
I = Inform
Task 3: Revise and Validate Requirements
OVERVIEW: Review and revise SysMod functional and technical requirements and document in an updated Requirements Traceability Matrix
(RTM) that may be used for more accurate fit-gap analyses, as well as to support more effective testing activities.
§ 8 weeks § DMV Modernization Business Capabilities Model
§ Revised Requirements Traceability Matrix (RTM), validated with DMV
KEY ACTIVITIES: business and technology staff (to be used in fit-gap analyses)
§ Document and agree upon desired DMV Modernization
Business Capabilities
§ Review and revise existing SysMod requirements, organizing
around core Business Capabilities. Identify gaps and
document new requirements if needed
§ Revise existing requirements to clarify current requirements
statements for the new project. Reword according to industry-
accepted standards:
§ Eliminate any “including, but not limited to…” clauses and list
specific items/elements
§ Eliminate ambiguous terms, such as “be capable of”, “as CRITICAL RESOURCE REQUIREMENTS:
appropriate”, ”adequate”, “user friendly”, etc. § PMO and DMV subject matter experts and technical staff will need to
§ Separate requirement statements that currently represent participate in requirements validation sessions
two or more distinct specifications (often identified by the
term “and”)
§ Logically organized (e.g., by business function, SDLC
phase)
§ Check for any missing types of requirements (e.g.,
performance, security, interface, facility, training, etc.)
§ Determine which requirements (if any) may be considered
optional or desirable
§ Finalize updated requirements in Revised Requirements
Traceability Matrix (RTM)
Sample DMV Modernization Business Capabilities
Manage Vehicle Operator Manage Regulation and

Manage Motor Carrier Records Support Enforcement Activities
Credentials Compliance
Update Motor Manage
Issue New Set up New Motor Investigate
Revoke or Suspend Carrier Account Inspection
Operator Carrier Accounts Potential Fraud Identify/Clear Identify/Clear
Operator License Information Processes
License Liens Stops
Replace Manage Driver Issue New Carrier Renew Truck Assess Assist in
Operator Information/ Registration Registration/Plates Penalties/Fines Prosecution Identify Tax Set Fees and
License Records Exemptions Penalties
Manage IRP/IFTA Collect/Maintain

Cancel Truck Inspect Salvage
E
Renew Reinstate Revoked Applications and Crash Record
Operator Registration/Plates Vehicles Data License Vehicle Issue OW/OS
or Suspended Records
Dealers
L
License License Vehicle Permits
Manage Vehicle Records Provide Customer Service

P Receive/Process Regulate Motor
Validate Operator
Insurance
M
Applications Carrier Safety Coverage
A
Change or Maintain/
Issue New Manage Lobby Maintain/Suppo Process Title &
Transfer Support Online Validate Medical
Registration Queues rt Kiosks Registration Validate Vehicle
S
Registration Services Qualifications
Applications Safety Inspections
(CLP/CDL)
Maintain Manage/ Receive and

Create New Administer Process Driver
Odometer Support IVR Record Validate Vehicles Operate and
Title Driver Tests License
Statements Systems Payments Emissions Maintain Weigh
Applications
Inspections Stations
Change or Issue Salvage Develop Issue License

Survey Customers Process
Transfer Title Title Manuals and Plates and Regulate Carrier
OW/OS Permit
Guides Stickers Weight and Tax
Applications
Compliance
Maintain Damage
Disclosure Manage Address Customer
Add New Liens Develop and Process Motor
Statements Inventory Issues and
Maintain Forms Carrier Registration
Complaints
Applications
Analysis of Three Go-Forward Options
Approach for Comparing each of Three Options for Moving Forward
Gartner assessed each of the three options for moving forward with the SysMod Project against the five criteria shown below
Most Costly Least Costly
Financial Impacts — the degree to which the option is likely to 4 3 2 1

positively or negatively impact the total cost of DMV’s
modernization initiative.
Negative Perception Positive Perception
Stakeholder Perception — the degree to which the option
4 3 2 1
chosen might be perceived — either positively or negatively
— by DMV’s internal and external stakeholders.
Most Difficult Least Difficult

Ease of Contracting — the relative complexity of the overall
process involved in establishing a contract with DMV’s external 4 3 2 1
vendors.
Most Time Least Time

Time to Implement — a comparative measure of the
overall time required for DMV to acquire, test and deploy a 4 3 2 1
solution that fully addresses its requirements.
Vendor Track Record — an assessment of the likelihood Unknown or Limited Track Record Strong Track Record
that the DMV’s selected vendor can successfully deliver a 4 3 2 1

modernized solution on time and within budget.
Option 1 — Reassess and Renegotiate Contract with Vendor
Risk Assessment Summary and Risk/Impact Considerations
Risk Rating - Option 1 The Total Weighted Risk Score for Option 1 is 2.45, representing a
Reassess and Renegotiate Tech Mahindra Contract medium/moderately high risk of going forward. This score is based on
risk/impact considerations outlined below.
Financial Risk RISK/IMPACT CONSIDERATIONS

4
3 § Financial Impacts
3
§ If DMV continues with an on-premise infrastructure for the vendor’s
2 solution, future transition to a cloud based solution would be more
Vendor Track Stakeholder
Record 4 1 Perception complex and costly
0 3 § The vendor may expect additional payment for the 120-day pause
§ Stakeholder Perception
1 § DMV Staff may perceive that nothing has changed and level of
2 commitment to the project may continue to be very low
Time to Ease of § Ease of Contracting
Implement Contracting
§ Because the contract between DMV and the vendor already exists,
amending the current contract could be the simpler option
§ The vendor may not agree to new contract terms — especially if DMV
Weight of Risk Rating Weighted approval of its key personnel is a contingency for moving forward
Assessment Category
Category Score Score § Time to Implement
§ After 90-120 days, DMV may determine that the proposed solution is a
Financial Risk 25% 3 0.75
poor fit for revised SysMod requirements and would need to pursue
other options
Stakeholder Perception 10% 3 0.30
§ Uncertainty in revised time to implement based on current state of
Ease of Contracting 15% 1 0.15 requirements
§ Vendor Track Record
Time to Implement 25% 2 0.50 § The vendor may not be able to field an A-Team
§ Based on past performance, there is potential that the vendor may still
Vendor Track Record 25% 4 1.00 be unable to meet its contract/performance expectations
Total Weighted
100% 2.70
Score
Option 1 — Reassess and Renegotiate Contract with Vendor
Key Activities
Option 1 — Reassess and Renegotiate Current Contract and SOW
Re-evaluate the vendor’s current Contract and Statement of Work (SOW) and revise to establish stronger project controls,
improve accountability for vendor performance and restructure deliverable and payment schedules.
Key Activities:
§ Require the vendor to produce Deliverable Expectation Documents for all past due deliverables and give 15-day notice for
the vendor to produce DED’s and corresponding deliverables
§ Conduct fit-gap analysis to assess the extent to which the vendor’s proposed solution addresses the DMV’s revised and
validated requirements
§ Review fit-gap analysis results and decide if DMV still prefers to proceed implementing the original proposed solution
§ If the original solution is still acceptable, DMV will draft a new SOW that includes the following key terms:
§ Identification of the vendor’s key personnel (e.g., project manager, solution architect, testing lead, etc.)
§ Provision for DMV to interview and approve all key personnel proposed by the vendor
§ Changes in project governance that assigns all decision making authority regarding scope, acceptance criteria, etc. to
DMV
§ Payment aligned with completion of defined phase gates — and not tied to individual deliverables
§ Provision for regular formal evaluation of vendor performance vs. SOW expectations and project plan
§ DMV to provide revised SOW to the vendor and negotiate accordingly
§ If DMV and the vendor cannot agree on revised SOW terms, or if the solution proposed does not address the DMV’s
requirements, pursue Option 2 or 3
Option 2 — Procure New Solution and System Integrator
Risk Rating — Option 2 The Total Weighted Risk Score for Option 2 is 2.75, representing a
Procure New Solution and System Integrator medium/moderately high risk of going forward. This score is based on
risk/impact considerations outlined below.

4 § Financial Considerations
3 3 § $27 million already invested in the vendor’s solution/infrastructure and
2 implementation work would be a sunk cost
§ The total contract costs for a new vendor and solution could be higher
Record 2 1 2 Perception
than the vendor’s
0
§ DMV may have challenges retaining/securing SysMod project funding
for a new procurement, based on negative perception of its
management of the vendor’s project
2 § Stakeholder Perception
Time to Ease of
Implement Contracting § DMV may experience poor public perception if it terminates its
4
contract with the vendor and goes back out to bid again
§ Staff may welcome new vendor and solution — however, they may
experience burnout from the current project
Weight of Risk Rating Weighted § Ease of Contracting
Assessment Category
Category Score Score § The procurement life cycle associated with issuing a new RFP and
selecting a new vendor/solution is complex
Financial Risk 25% 3 0.75
§ DMV could leverage lessons learned from the vendor’s project to
Stakeholder Perception 10% 2 0.20 negotiate more favorable terms and pricing
§ Time to Implement
Ease of Contracting 15% 2 0.30 § Option 2 presents the longest time to implement — involving 1–2
years for the RFP and contracting process, followed by another 3–5
Time to Implement 25% 4 1.00 years to develop, test and deploy the new solution
§ Vendor Track Record
Vendor Track Record 25% 2 0.50 § The DMV would have another opportunity to properly assess and vet
potential vendors and solutions — based on a clearer set of
Total Weighted
100% 2.75 requirements
Score
Option 2 — Procure New System Integrator and Solution
Key Activities
Option 2 — Procure New System Integrator and Solution
Prepare a new Request for Proposal (RFP) document, including revised and validated requirements, to identify an alternative
solution and different system integration vendor to deliver a modernized DMV solution to the State.
Key Activities:
§ Prepare new RFP document, including revised/validated functional and technical requirements
§ Respond to vendor questions during proposal preparation period
§ Develop evaluation criteria/model to be used in assessing vendor proposals — including relative fit of proposed solutions to
the DMV’s requirements
§ Evaluate/score vendor proposals and identify top-rated vendor/solution
§ Negotiate contract and Statement of Work (SOW) with top rated vendor/solution
Option 2 — Procure New Solution and System Integrator
Related Activities
Magic Quadrant for CRM and Customer Experience
DMV should review Gartner’s Magic Quadrant Implementation Services, Worldwide
and related research to assess the capability of
leading system integration vendors, should it
decide to pursue Option 2. Gartner’s most
recent Magic Quadrant for CRM and Customer
Experience Implementation Services is
presented at right.
The Magic Quadrant compares each vendor’s
overall capabilities and provides a relative
ranking of each firm’s “Completeness of
Vision” and “Ability to Execute.”
Completeness of Vision encompasses a
vendor’s market, product and industry strategy
and capacity for innovation.
Ability to Execute accounts for the quality and
competitiveness of a vendor’s products and
services, its operational capacity and overall
customer experience.
DMV should be very cautious in selecting any
niche vendor to implement such a large and
complex modernization initiative as SysMod.
Option 3 — Transfer MvM Solution from Arizona MVD
Risk Rating — Option 3 The Total Weighted Risk Score for Option 3 is 1.40, representing a
Transfer MvM Solution from the Arizona MVD low/medium risk of going forward. This score is based on risk/impact
considerations outlined below.

4
§ Financial Considerations
3
1 § Because the MvM solution is already developed, the cost to DMV should be
2 far less than the vendor’s $75M contract cost. A restructured PMO and
Record 2 1
1 Perception streamlined staffing model would further reduce costs
0 § $27M already invested in the vendor’s solution/infrastructure and
implementation work would be a sunk cost
1 § Stakeholder Perception
2 § This could be perceived as an innovative example of cost-effective
Time to Ease of
partnership with other state DMV
Implement Contracting
§ Though still in development MvM solution has been very well received by
ADOT customer service reps. Given this the DMV staff may be more
confident and supportive of the transfer solution than the current solution
§ Ease of Contracting
Weight of Risk Rating Weighted
Assessment Category § An agreement would need to be executed between AZ and NV to define
Category Score Score
roles, responsibilities, terms/conditions, costs, resources, etc. associated
Financial Risk 25% 1 0.25 with the transfer
§ NV would need to execute a contract for system implementation services to
Stakeholder Perception 10% 1 0.10 support work involved in transferring the MvM solution
§ Time to Implement
Ease of Contracting 15% 2 0.30
§ The AZ MvM solution can be considered a COTS solution that would need
Time to Implement 25% 1 0.25 to be configured or customized for NV DMV, therefore time to implement
should be shorter than the other two options — possibly in 3 years or less
Vendor Track Record 25% 2 0.50 § Vendor Track Record
§ DMV may need to wait until qualified MvM contract personnel are available
Total to work on the new project — these details would need to be
100% Weighted 1.40 evaluated/explored
Score
§ The AZ MvM approach and tools used for project management have
resulted in one of the most well executed projects currently in the public
sector
Key Activities
Conduct a fit-gap analysis of revised and validated requirements to assess to potential for transferring the Arizona MVD’s MvM
solution to Nevada — and assuming the MvM solution addresses the new project’s requirements, plan and execute a transfer of
this solution.
Key Activities:
§ Conduct a site visit to the Arizona MvM project office and a representative field office to gain a better understanding of how the
solution is being developed and how deployed functionality is already working
§ Conduct fit-gap of revised and validated requirements to determine how well the Arizona MvM solution addresses these
requirements. Identify gaps and determine how these requirements will be addressed
§ If the MvM solution proves a reasonably good fit to the new project’s requirements, the DMV commits the next 120 days to
planning the transfer, including the following key activities:
§ Assess alternative approaches for defining the affiliation between AZ and NV, including consideration of the following
options (see next page for additional details):
• Direct Intergovernmental Agreement
• Joint Venture
• Private Enterprise
§ Estimate costs, including:
• Total Cost of Ownership of the transfer solution for Nevada
• Costs shared with Arizona in completing the MvM solution and executing the transfer
§ Determine contract staffing needs and timing
• Skills/positions required
• Options for recruiting/contracting required staff
• Alternatives for leveraging staff involved in Arizona MvM project
§ Determine tools, licenses, etc. that Nevada would need to acquire
§ Execute the agreement that best addresses the mutual needs of both states and begin the transfer project
Transfer Agreement Alternatives
Comparison of Alternative Affiliation Models for Executing a Transfer Agreement between AZ and NV
Affiliation
Option Overview Pros Cons
Direct An IGA is a contract between two § Likely the most familiar type § All details of the agreement
Intergovernmental public agencies. Requirements for of agreement for DMV to must be specified upfront in the
Agreement (IGA) developing and executing an IGA execute IGA (e.g., cost sharing,
are typically defined by statute and organization, staffing, etc.)
requires review and approval by the
§ Execution of the IGA may
Attorney General, and sometime by
require legislative approval in
the State Legislature as well.
both states
Joint Venture (JV) A JV agreement defines the § Cost sharing arrangements § Requires facilitation/
objectives of both parties, each and initial investments by intervention by both State
state’s contribution, the purpose of both parties are defined — Attorney Generals — which
the relationship and a time-line of but, other details regarding may slow creation of the JV
the venture. The agreement also execution of the transfer can
establishes the governance be hammered out once the
structure and includes provisions JV is set up
for defining intellectual property
rights and termination/dissolution
processes.
Private Enterprise Similar in principle to a JV — § Transfers some of the § May require a competitive
Agreement however, a contract is established execution responsibilities and procurement to select an
between a private vendor and both risk from the States to a third- implementation vendor —
states. party vendor requiring additional time and
effort
Side-By-Side Comparison of the Three Options
In comparing the risk assessment results for each of the three options, the option to transfer
the Arizona MvM solution to Nevada DMV was scored as the most favorable.
Option 1 Option 2
Option 3
Reassess and Procure Different System
Risk/Impact Dimension Transfer MvM Solution
Renegotiate Contract Integrator and Solution
from Arizona MVD
with the vendor (New RFP)
Financial Impact 3 3 1
Stakeholder Perception 3 2 1
Ease of Contracting 1 2 2
Time to Implement 2 4 1
Vendor Track Record 4 2 2
Weighted Risk Score 2.70 2.75 1.40
Detailed Findings and Recommendations
Gartner’s Risk Assessment Framework
Gartner’s rating of the risk areas highlighted below reflect our Summary Findings and
Observations, as well as our detailed assessment results, where we compare performance of the
SysMod project with best practices (as defined by IEEE, PMI, Gartner Research and other similar
sources). Detailed Findings and Recommendations are presented in the following pages.
B. Project C. Vendor D. Requirements E. Communications &

A. Technical Solution Management and Management & Organizational
Performance
Governance Benefits Realization Change
1. Vendor Management Management (OCM)
1. Business Case 1. Scope Definition and
1. Executive Support Strategy Refinement 1. Communication
Planning and
2. Technology 2. Program/Project 2. Vendor Performance 2. Requirements Execution
Infrastructure & Governance Management Management
Process Strategy and 2. OCM Planning and
Planning 3. Risk Mitigation 3. Development 3. Benefits Realization Execution
Strategy and Planning and Planning and
3. Security Planning Management Execution Execution
and Execution
4. Schedule 4. Test Planning and
4. Data Conversion Management Management Risk Level
Planning and
Execution 5. Budget Planning and 5. Training Strategy &
Planning High Risk of Failure
Management
5. Integration/Interface
Significant Risk
Planning and 6. Resource Planning
Implementation and Management Manageable Risk
6. Operational Managed Risk
Transition Planning
High Risk Significant Manageable Managed
of Failure Risk Risk Risk
A. Technical Solution: Business Case
Best Practices Key Findings and Observations
Activities and processes for this Assessment Area are in concert with § The Business Case is part of the Technology Investment Request (TIR)
IEEE STD 1490 and 1058, including: document used by the state of Nevada to review and approve funding for
§ The program/project has a well developed Business Case that has been the SysMod project
reviewed and approved by program/project sponsors and stakeholders § Anticipated financial benefits and other financial metrics were labeled as
§ The Business Case documents: TBD in the “Final Version” of the Business Case reviewed
§ Current challenges faced by the organization with opportunities and § A requirements template is included in the Business Case, linking benefits
specific areas for improvement to requirements. However, requirements are not specific and most span
multiple business benefits
§ Program/project’s goals, benefits, costs, and risks areas
§ A process to measure business benefits was not included in the Business
§ Solution alternatives and recommendations for action
Case
§ Linkage between the tangible business benefits and program/project
§ The alternatives assessment model used to justify selection of a COTS
deliverables
solution is very high-level with no meaningful discussion of any other
§ Business benefits are defined and quantified and will be used to option than the status quo
measure the successful operation and performance of the new system
§ A process is in place to measure the business benefits on an ongoing
basis; also, management is aware of the benefit measurement
process, and sponsors its implementation
Impact
§ Without quantifiable business benefits and a process to continually monitor progress toward achieving these metrics, the DMV lacks the means to assess
and communicate business benefits and ROI for this project
§ Presenting detailed alternative assessment results is important to demonstrate due diligence in evaluating and deciding the best modernization strategy —
and a confirmation that solution selected is the best one
. Recommendations
§ Revise the SysMod Business Case to include specific, detailed metrics that may be used in assessing and reporting progress in attaining anticipated
benefits — as the system is developed, and when it is completed
§ The Business case should also provide an explanation of how benefits will be measure and tracked
A. Technical Solution: Technology Infrastructure & Process Strategy and Planning
Activities and processes for this Focus Area are in concert with Gartner § The decision to require an on-site hosting environment for SysMod was
Best Practices, including: made over 2 years ago, before a cloud-based option might have been
§ Server technologies are appropriate to new solution, storage capable of considered workable. This decision has begun to lock the DMV into a
supporting new solution costly, long-term obligation for maintenance and support of the SysMod
solution
§ Backup/recovery/DR/BC meets likely solution requirements, plans for dc
upgrades § The proposed technical architecture for the SysMod solution has changed
multiple times since the project began, as the vendor has rotated different
§ Plans end user devices & upgrades, communication and collaboration
architects onto the team over the last 1.5 years
services capable of meeting new solution requirements
§ DMV acquired an Uninterruptible Power Supply (UPS) for the equipment;
§ Networks capable or extensible to support new solution, network access
this equipment was damaged but damage was not detected until after
supported in environment
warranty had expired
§ User support capable of supporting new solution and surges in support
§ DMV does not presently have the skills needed to support the proposed
request demand during deployment
SysMod technology infrastructure
Impact
§ Solution architecture changes are not going through a formal change review process. This makes it difficult to assess the value and performance of those
changes. It is uncertain what value this project will provide due to the changing architecture and lack of mapping back to the business benefits
§ Also, DMV is not likely to achieve best value for their investment by going with an on-premise solution given the potential cost savings and efficiencies to be
gained by going to a cloud solution. With the current solution, it is unlikely that DMV can promote this system to the cloud without significant overhaul
.
Recommendations
§ Perform a cost-benefit or return-on-investment analysis, comparing the vendor’s proposed on-site solution for hosting the new project vs. a cloud-based
hosting option
§ Before deciding on a technology architecture for the new project, DMV should consider whether it can acquire and keep technical staff with skills needed to
maintain and support the solution, as well as the total cost of ownership
§ Designate the vendor’s solution architect as key personnel, subject to DMV’s review and approval. All development work should be suspended until an
acceptable solution architect is identified and the solution architecture deliverable is reviewed and approved
A. Technical Solution: Security Planning and Execution
Activities and processes for this Focus Area are in concert with ISO § The vendor is contractually obligated to develop an Information Security
17799 and Gartner Best Practices including: Plan but has not done so to date, and no such plan is identified in its
§ Project security requirements are completely specified and clearly written Master Project Schedule. This plan and a related security architecture
and the vendor has a clear understanding of the security requirements document should be finalized and approved by the DMV before any
development work begins
§ The vendor will implement a secure means for storage and transfer of
sensitive project data § The IFTA module of SysMod was installed absent any DMV-approved
security requirements or security architecture, other than FedRAMP
§ The vendor maintains a written policy which prohibits the transfer of
standards (as recommended by the DMV’s ISO)
sensitive data via unencrypted email or other media
§ The vendor has designated a Security Lead for the SysMod project other
§ The Project vendor addresses all security concerns through a qualified
than the individual originally proposed. It is not clear that current Security
security lead resource
Lead has the appropriate qualifications or if these qualifications have been
§ The vendor designates a security professional to facilitate the resolution of vetted by the DMV PMO
security issues
§ Project systems are compatible with all required agents or other collection
techniques required for intrusion management and the vendor provides
documentation proving this
§ The Project and IV&V vendors monitor sources of vulnerability reporting
and ensure vulnerabilities that arise during development are properly
addressed
Impact
§ DMV does not currently have a detailed understanding of its risk exposure with the current solution, particularly because the solution architecture is not well
documented or understood
§ The IFTA module of SysMod was installed absent any DMV-approved security requirements or security architecture
§ Lack of a security plan and defined solution architecture should be one of the top concerns and priorities for Nevada DMV
Recommendations
§ Require replacement of the vendor’s current Security Lead with an appropriately skilled/experienced resource that meets the DMV’s approval
§ Give the vendor 15 days to produce an Information Security Plan that meets DMV approval
§ Consider termination of the vendor’s contract if it cannot meet this deadline, as well as deadlines for submitting other key deliverables identified in the
vendor’s Scope of Work
.
A. Technical Solution: Data Conversion Planning and Execution
Activities and processes for this Focus Area are in concert with IEEE § DMV assigned dedicated staff to work with MVIT to begin data discovery,
STD 1074 and Gartner Best Practices, including: analysis and quality checks in September 2016
§ Accountability for data conversion is assigned and was planned early in § Very recent focus was data migration to support the IFTA module
the project deployment
§ Processes are in place for validating conversion success and mitigating § A Data Conversion Plan in identified in the vendor’s Scope of Work —
conversion failures however, no such plan is included in its most recent project schedule
§ Validation routines exist to ensure conversion success and conversion § Further data conversion/migration progress has slowed as the DMV team
checklists and resources are defined awaits decisions on an approved solution architecture and approach (big
§ Required vendor support during conversion defined and communicated bang vs. phased implementation)
§ Restart and rollback scenarios in case of conversion failure defined and § Delays by the vendor in developing high level designs and solution
estimated conversion effort defined architecture is compressing time available for conversion activities
§ All legacy system data stores have been identified § The DMV data conversion team has also requested a point of contact with
the vendor to begin exporting data to the target system
§ Extensive and effective data cleansing and standardization is
accomplished before data conversion activities began and automated data
conversion routines have been validated
§ Manual data conversion routines and processes have been validated.
§ Key subject matter experts perform rigorous data accuracy reviews during
each phase of the conversion process
Impact
§ Data conversion/migration progress has slowed as the DMV team awaits decisions on an approved solution architecture and approach (big bang vs.
phased implementation)
§ Typically, Gartner sees clients underestimate the complexity and difficulty of a data conversion/migration effort of this size. This can frequently lead to very
lengthy delays in implementation timeframes if a plan is not put in place early
Recommendations
§ Give the vendor 15 days to develop a Data Conversion Plan that meets DMV approval
§ Give the vendor 15 days to develop a solution architecture that meets DMV approval, so as to allow time for data conversion activities to resume
§ Consider termination of the vendor’s contract if it cannot meet these deadlines, as well as deadlines for submitting other key deliverables identified in the
A. Technical Solution: Integration/Interface Planning
Activities and processes for this Focus Area are in concert with IEEE § The vendor is obligated to develop an Interface Architecture deliverable
STD 12207 and 1028, including: and submit to the DMV for review and approval. It is not clear when, or if
§ System Interface (hardware and software) requirements are fully defined the vendor will submit this document, as this deliverable is not identified in
and documented in the Interface Requirements Specification document, its recent Master Project Schedule
and approved by OHA § System interfaces have been well defined by the DMV in its TIR and RFP
§ The overall system architecture is fully defined and documented (both
hardware and software) in the systems architecture document
§ Critical or new interface technology has been fully tested and key
resources identified to provide needed support during testing and go-live
activities
§ The system testing process has been designed to assess and validate all
interfaces and their performance (operational, security and integration)
Impact
§ Without a clear architectural plan, the vendor has not defined its approach to integrating/interfacing data and services from critical external systems needed
for the SysMod solution to function as desired
Recommendations
§ Give the vendor 15 days to develop Interface Architecture deliverable that meets DMV approval
A. Technical Solution: Operational Transition Planning
Activities and processes for this Focus Area are in concert with IEEE § A “Transition Approach” deliverable is cited in the vendor’s scope of work.
STD 1074, 1219, 1362 and Gartner Best Practices including: However, it’s not clear it has developed this document and/or submitted
A Transition Plan has been prepared and approved, providing a for DMV review and approval
comprehensive approach to transition of services from the vendor to the client § A Transition Plan is critical to ensure the DMV is prepared to assume
or its maintenance and support contractor, including: operational responsibilities for the SysMod solution — either partially
§ The timeframes, key milestones, critical path activities and deliverables (phased implementation) or upon final acceptance of the entire solution
required for ongoing operations and support § This transition issue was evident in the recent IFTA module deployment,
§ Identification of all critical resources (internal and vendor) and a process to as no formal plans were defined for providing operational support for this
ensure they are available to support the transition and a readiness function after go-live
assessment and knowledge transfer process to ensure the client or its
contractor are prepared to provide required support
§ An approved governance structure and communications plan defining the
operational support decision process and go/no-go assessment of events
and criteria to ensure planned operations and support activities are being
performed as defined
§ Service-level agreements which have been developed for all operations
and support
Impact
§ A Transition Plan is critical to ensure that the DMV is prepared to assume operational responsibilities for the SysMod solution — either partially (phased
implementation) or upon final acceptance of the entire solution
§ This transition issue was evident in the recent IFTA module deployment, as no formal plans were defined for providing operational support for this function
after go-live
Recommendations
§ Give the vendor 15 days to submit a Transition Approach deliverable that meets DMV approval
B. Project Management and Governance: Executive Support
Activities and processes for this Assessment Area are in concert with § The Project has strong, visible and active leadership support at the
PMBOK and IEEE STD 1028 and 12207, including: Executive level of the organization
§ The program/project enjoys sustained and visible support from project § Executive level leadership actively participates in project meetings and
sponsors/executive leadership. consistently makes themselves available for consultations with the PMO
§ The most knowledgeable and valuable stakeholders from each affected regarding key project decisions
area (business and IT) have been identified and are assigned to the § Executive leadership was present for the recent IFTA module go-live,
project on a full-time basis demonstrating support for the project’s first (limited) deployment of
§ Senior management make themselves available to the end-user SysMod functionality
organizations to explain the changes driving the new system development
activities and why they are needed
Impact
§ Based on active participation in the project, the Executive Level is well positioned to make the critical decisions necessary at this point in the project
Recommendations
§ Strengthen executive support through improvements in project governance and leadership roles defined in OCM and Communications Plans
B. Project Management and Governance: Program/Project Governance
Activities and processes for this Focus Area are in concert with IEEE § No formal governance framework or RACI has been adopted for the
1490, IEEE STD 1028 and 12207 including: SysMod project
§ A governance framework has been formally established, defining decision § Many key project decisions regarding acceptance of work products and
making rights, responsibilities and processes deliverables are made by the DMV Program Manager, creating a
§ The governance framework is communicated and well understood by all bottleneck in the approval process
project team members and stakeholders § No charter has been prepared outlining the makeup, decision making
§ The framework includes a RACI matrix outlining decision rights in key rights, responsibilities and processes for the Executive Committee
domain areas (e.g., budget/schedule changes, design changes, etc.) for all § The Executive Committee for the SysMod project has not yet met as no
individuals and groups involved in the project decisions have needed to be made
§ An Executive Steering Committee has been set up, ideally comprising § A Steering Committee comprising two DMV business executives, the
between 3–7 key business and technology leaders from the client SysMod Program Manager and two of the vendor’s executives have
organization provided overall project direction to date
§ A Steering Committee Charter exists, formally defining the § Until very recently, minutes of Steering Committee meetings have not
structure/membership, meeting frequency, decision making and been recorded/maintained
documentation processes. Members may rotate over the duration of a long
project — but, otherwise turnover is low
§ Consultants and contractors may be invited to Steering Committee
meetings to provide input, but do not have voting rights
Impact
§ Without a formal Governance framework, the DMV cannot confirm that key decisions are made by the right people — individuals with the appropriate level
of knowledge and authority
§ Having (2) DMV business executives, (2) vendor executives and a contract Program Manager with voting rights introduces conflict of interests and
competing agendas. What is best for the vendors involved will not always be what is best for the DMV
Recommendations
§ Develop a comprehensive governance framework that formally defines guiding principles and processes for project-level decision making
§ Define makeup and roles/responsibilities of governance groups — including the Executive Steering Committee, Project Management Office and ad hoc
groups (e.g., architecture review board, business process review board, policy/standards review board)
§ Define decision making protocols and prepare RACI or RAID matrix to assign decision making authority
§ Prepare Executive Steering Committee Charter to outline group organization, authority/responsibility, decision making protocols, documentation standards,
meeting frequency, etc.
B. Project Management and Governance: Risk Management Strategy and Planning
Activities and processes for this Focus Area are in concert with PMBOK, § The vendor is contractually obligated to prepare a Risk Management Plan
IEEE 1490, IEEE STD 12207 and 1028 including: — but, this plan had not been submitted. Hence, risk management
§ A Risk Management Plan has been developed for the program/project that activities have been handled in an ad hoc fashion since the project began
includes the following elements: 1.5 years ago
§ Roles and responsibilities are defined for identifying, managing and § Risks are reportedly captured in risk logs by each of the 6 project teams
mitigating/addressing project risks — it is not clear how these risks are centrally recorded, analyzed,
mitigated and tracked
§ Timetable/schedule and approach is prescribed for meetings to
identify, manage and document the status of project risks § Risks are identified in the vendor’s status reports — however, there are
no regularly scheduled meetings to discuss and address risks at the
§ Guidance and tools for quantitative and qualitative risk analysis are
program level
defined to determine probability and severity of identified risk
§ Templates/tools are defined for recording the status of risks
§ Processes are defined for monitoring and controlling risks and for
identifying selected risks to be reported to the Project Sponsor and the
Executive Steering Committee
§ An individual is designated to ensure risk management activities are
executed according to plan
Impact
§ Without a clear formal approach for managing risk, the project tends to operate in a reactive mode, rather than proactively anticipating and addressing
problems before they become significant
Recommendations
§ Give the vendor 15 days to submit a Risk Management Plan deliverable that meets DMV approval. This plan shall require:
§ That risks be identified, analyzed and managed at an enterprise project level
§ Regular meetings between the vendor and DMV to assess new risks and track the status of current risks.
§ Risks categorized as significant or major reported to the Executive Steering Committee
B. Project Management and Governance: Schedule Management
Activities and processes for this Focus Area are in concert with PMBOK, § The vendor did not produce a master project schedule (for Phases 1, 2
including: and 3) until nearly 1.5 years after project kickoff; Current schedule has
§ The program/project schedule provides a deliverable-oriented hierarchical been recently approved by the DMV
decomposition of the work to be executed by the project team, by Start § A Schedule Management Plan is identified in the vendor’s Scope of Work
and Finish dates, work durations, dependencies between work activities, — yet it appears this plan was never created. The vendor’s approach for
and specific resource name assignments to those activities tracking and updating the schedule (e.g., percent tasks completed) — and
§ A program level schedule is developed and managed to ensure related DMV’s approach for verifying that these updates are accurate — is not
projects are effectively planned and executed with respect to clear
task/milestone interdependencies, assignments, risks, etc. § Several DMV Project Managers and MVIT staff who reviewed the new
§ The schedule "breaks down" the scope of the Statement of Work into schedule felt time frames for tasks in the new plan were aggressive —
whole work units, delivery dates and time commitments for all project and in fact, the vendor has already started to miss on target dates for
resources. Schedules are developed with inputs from all team leads completing key activities/deliverables, including those for planning and
preparation of high-level design documents (HLD)
§ Schedules are resource loaded and balanced to ensure all team members
understand the timing and duration of their respective assignments and § Several deliverables identified in the vendor’s Statement of Work are not
have sufficient time to complete assigned tasks identified in its project schedule (e.g., Interface Architecture, Data
Conversion Plan, Transition Approach, Information Security Plan)
§ Current schedule is resource loaded and balanced — though we could not
determine if the level of effort required was accurate for all tasks
Impact
§ For 1.5 years, the project essentially functioned without a formal and structured approach for planning and executing work — or for determining actual
resources needed to complete defined tasks. The current approved schedule now provides that approach — however, task time frames and/or resource
assignments appear to be inaccurate, as the vendor is already failing to hit target dates for completing key activities/deliverables
Recommendations
§ Give the vendor 15 days to develop a Schedule Management Plan and revise and re-baseline its Project Schedule to address findings and observations
cited above
§ Consider termination of the vendor’s contract if it cannot meet these deadline, as well as deadlines for submitting other key deliverables identified in the
B. Project Management and Governance: Budget Planning and Management
Activities and processes for this Focus Area are in concert with PMBOK, § No change control process is in place for evaluating proposed budget
including: increases — Change Management Plan, including Change Control
§ Labor costs are based on work assignments defined in a resource-loaded procedures were to have been completed in November 2017 (as per the
and balanced project schedules recently updated project schedule)
§ Actual costs for contract services and equipment are included if known, or § A Cost Management Plan was included in the vendor’s Scope of Work —
are estimated using historical data maintained by the organization from however, this deliverable is not included in its project schedule and does
prior similar work or via other industry sources not appear to have been developed
§ Contingency costs are included § No risk management is in place to actively monitor and mitigate project
risks — notably potential funding/budget shortfalls. DMV had noted in its
§ Baseline budgets, as well as proposed budget increases, are submitted for
TIR that budget/funding risks would be mitigated by managing and
internal review and approval according to defined governance processes
monitoring “to ensure it is within budget, time, and delivery date”
§ DMV has dedicated funding in its project budget for contractors to backfill
positions for SMEs and other DMV resources assigned to the SysMod
project. Project delays by the vendor may require that costs for backfill
contractor personnel be adjusted
Impact
§ By not monitoring planned and change related activities, it is difficult to monitor the burn, therefore it is not possible to know if the appropriate level of funds
are available throughout the life cycle of the project. Since it is fixed cost project there is the potential for funding to dry up prior go-live
Recommendations
§ Give the vendor 15 days to develop a Change Management Plan and Change Control Procedures that meets DMV approval
§ Give the vendor 15 days to develop a Cost Management Plan that meets DMV approval
§ Introduce budget management risk as part of the overall project risk management plan, develop mitigation strategy and regularly track/report status
§ Determine the cost impact of delays to DMV (e.g., backfill personnel and associated costs). Also, assess potential reductions in budget for backfill
resources associated with streamlining the PMO and overall staffing structure
B. Project Management and Governance: Resource Planning and Management
Activities and processes for this Focus Area are in concert with PMBOK, § No resource management plan is in place to provide a clear picture of the
IEEE 1490, IEEE STD 12207 and 1028, and Gartner Best Practices project organization, delineation of roles and responsibilities and
including: procedures for addressing turnover of staff
§ Project team is receiving commitment from management in the form of § DMV business SMEs were invited to apply for roles with the SysMod
additional resources and support when necessary and in active project — consequently, some of the most qualified SME resources may
engagement and resolution of staffing issues not have been assigned to the project and many DMV business analysts
§ A process is in place for addressing project turnover issues, i.e., people assigned to SysMod lack the appropriate skill set for the position
leaving and joining the project (both internal and external resources) § The DMV’s functional team structure may not be the most effective means
§ Planned project staffing levels have been adequate to meet project to organize and staff the project:
requirements and the overall project organization and reporting § Imbalance of workload between teams
relationships are well-defined and stable § Assigned SME’s may not have appropriate expertise in many team
§ Project member roles and responsibilities are clearly defined, documented functions (requires other business SMEs to become engaged when
and are stable and members of the Project Team know and understand needed)
their role in the organization and have the skills appropriate to perform § Project managers and project coordinators could be more effectively
these roles leveraged across the project organization
§ Members of the Project Team know and understand the reporting § Vendor staff has experienced high turnover
relationships for the project and all influences on the Project Team are
§ Many of the vendor’s staff members identified as software SME’s
being managed effectively
reportedly did not have any demonstrated expertise in the software
Impact products.
§ Without a formal, approved resource management plan, reporting structures and the roles/responsibilities are not clear, potentially resulting in inconsistent
and/or slow decision making processes. Without this plan, the project also tends to respond reactively and more slowly in finding competent replacements
for staff that leave the project
§ A high turnover presents challenges to the project due the constant need to continually onboard/train new members along with the potential of losing
competent resources. Competent resources may turn over more frequently due to a higher demand
Recommendations
§ Give the vendor 15 days to develop a Resource Management Plan that meets DMV approval
§ Consider termination of vendor’s contract if it cannot meet this deadline, as well as deadlines for submitting other key deliverables identified in the vendor’s
Scope of Work
§ Restructure and streamline DMV’s PMO and staffing model and leverage only SME’s who have the appropriate skills/expertise required for selected project
activities (e.g., business analysis, design/development, testing)
C. Vendor Performance: Vendor Management Strategy
Activities and processes for this Focus Area are in concert with PMBOK, § The DMV conducted a competitive solicitation process to select a system
including: integration vendor for the SysMod project
§ The organization has conducted an objective risk assessment to identify § The vendor has had limited experience implementing a DMV
specific project activities/tasks that will require external resources, based modernization project for a state of Nevada’s size and complexity.
on needed skills and expertise, budgetary reasons, availability of internal Further, the vendor’s sole prior modernization effort with the New
staff, etc. Hampshire DMV was delivered over a year late and involved
§ All activities/task required to obtain contract services for the project are implementation of a different technology stack
mapped out in advance in the project schedule § The vendor’s scope of work appears to be appropriately detailed and
§ Vendors are evaluated and selected competitively on the basis of a complete and in general conformance with best practices
combination of defined criteria (for example, skills/capabilities, staffing,
prior relevant experience, cost, etc.)
§ Contracts with external vendors specify work to be performed, deliverables
and deliverable acceptance criteria, reporting relationship within the
project organization, performance expectations or service-level
agreements, schedule requirements, pricing assumptions and change
management processes
Impact
§ The vendor’s limited DMV experience and knowledge, as well as its plan to use the proposed solution to meet the Nevada DMV’s business and technical
requirements, could introduce ongoing functional and technical risk to the project
Recommendations
§ If DMV determines it must rebid SysMod system implementation services
§ The Department must more strongly weigh a vendor’s prior experience and performance in similar system modernization projects in its evaluation
model
§ Strong preference should particularly be given to vendors that have successfully executed a comparable project for another state motor vehicle agency
§ DMV should also look at independent research reports, Including Gartner Magic Quadrant results, for additional insight into the capability and vision of
potential vendors
C. Vendor Performance: Vendor Performance Management
Activities and processes for this Focus Area are in concert with PMBOK, § The PMO has not assigned resources to continually monitor the vendor’s
IEEE 1490 and Gartner Best Practices including: compliance with respect to:
The Project Team has a mature and robust vendor management process in § SOW requirements
place to guide, assess, manage and evaluate vendor performance including: § Project schedule
§ Monitor vendor compliance to SOW requirements, schedule and budget § Conformance with established project quality standards and
requirements and quality standards and practices practices
§ Assess vendor deliverables tracking, review and approval process § Deliverable acceptance (over 4000 deliverables identified)
§ Ensure frequent, open, and honest communications between the § There are no consistent periodic review sessions planned with the
vendor and project team vendor to proactively address and update performance risks and issues
§ Establish and track vendor performance to published criteria/metrics § There are no plans to Identify, manage and review vendor performance
and supports vendor contract performance assessments objectives periodically with program sponsors and governance groups
§ Oversee vendor activities regarding contracting, delivery and quality § In many cases, the vendor project team resources lack business analysis
matters and document and report issues found by the project manager skills and expertise in the solutions proposed for SysMod
§ The vendor has not established a partnership which aligns day-to-day
program interactions with overall vendor practices
Impact
§ Without a formal, approved Vendor Performance Plan — and the resources to execute this plan — the DMV will be challenged to continually assess how
well the vendor is performing per the terms of the contract. This could lead to contract/policy violations and/or the vendor possibly not addressing all
requirements accurately and completely
Recommendations
§ If DMV elects to retain the vendor to complete the SysMod project, it should restructure the vendor’s Scope of Work to provide stricter project controls and
performance expectations, including compliance with schedule and quality requirements
§ DMV should also designate an individual within the PMO to monitor the vendor’s performance going forward
§ The DMV should also conduct review sessions with the vendor at least once monthly to address performance issues
C. Vendor Performance: Development and Implementation Planning and Execution
Activities and processes for this Focus Area are in concert with IEEE § Development activities/tasks and assignments are provided in the recently
STD 1233, 1058, 1074, 1490 and Gartner Best Practices, including: approved Master Project Schedule
§ The Project Plan identifies the submittal dates (draft and final) for all § Project teams have been developing high-level design documents (HLD);
development documentation each HLD is time-boxed in the plan — however, project team members
§ A formal process has been documented and is being used to review and state that these designs are not executed/written well and are often late
provide comments on development documentation submitted by the § The vendor has not identified or applied a consistent methodology and
vendor to the client approach to development and implementation; when discussing the
§ The vendor is meeting all quality and delivery schedule standards for software development life cycle (SDLC), some interviewees described it
development documentation as a waterfall effort while others described it as agile
§ System conceptual design has been fully defined, documented and § A Development Management Plan is cited in the vendor’s Scope of Work
approved — however, no such plan is included in its project schedule
§ System detailed design requirements are fully defined, documented and § The SysMod solution architecture has been continuously developed and
approved updated — but, it has not yet been approved by the DMV
§ System functional, interface database design and performance
requirements are fully defined, documented and approved
§ The overall system architecture is fully defined, documented and approved
§ The overall system design does not employ “cutting-edge” technology but
instead leverages industry best practices
Impact
§ Poorly written design documents have either caused delays, incorrect project decisions or in some cases poorly written documentation had to be rewritten
by DMV project resources
§ A standard, documented development approach and methodology helps to ensure consistency and efficiency in development work. Lacking such
consistency and efficiency, development work may be delayed
Recommendations
§ Give the vendor 15 days to develop a Development Management Plan that meets DMV approval
■
§ A standard,
Consider documented
termination development
of the approach
vendor’s contract if it and methodology
cannot helps to ensure
meet this deadline, as wellconsistency
as deadlinesand
forefficiency
submittinginother
development work. Lacking
key deliverables such
identified in such
the
consistency and efficiency,
vendor’s Scope of Work development work may be delayed.
§ DMV shall review the vendor’s Solution Architecture and approve or return with detailed comments for revision
C. Vendor Performance: Test Planning and Management
Activities and processes for this Focus Area are in concert with IEEE § The vendor is contractually obligated to develop and submit a test plan for
STD 12207, 1028, 1044, 829, 730 and 1061, including: DMV’s approval; the test plan was to be completed by November 30,
§ A master test plan has been developed, defining a framework for all 2017 (Gartner did not see a draft or final version of this plan)
procedures relating to testing (ownership, notification, timing, § Testing durations are not known so they can’t measured against best
reconciliation, error handling, restart procedures, regression, etc.) practices
§ Testing is mapped out to include Unit, SIT, UAT, Performance, Security, § Entrance/exit gate criteria can not be confirmed without a plan
and Negative/Break Testing § Appropriate testing sequences (Unit/DEV, SIT, UAT, Security,
§ Criteria for successful completion of each testing phase is clearly defined Performance) listed as tasks in the Master Project Schedule, with the
(e.g., for phase gate entrance/exit to and from each testing phase) exception of negative/break and regression testing
§ Tools and tracking mechanisms are identified with documented § DMV and the vendor resources have been dedicated to test management
procedures on how to track test execution, report defects, escalate defects § A tool has been identified to facilitate test management
to issues, and bring them to resolution
§ Scripts are developed that align with the requirements and target business
processes
§ UAT testers are trained to execute scrips
Impact
§ Without a formal, approved test plan neither the vendor, nor the DMV agree on an approach for confirming that SysMod solution will satisfactorily address
the State’s requirements. Resource requirements are not defined, nor are processes and criteria for demonstrating the solution will function as expected, or
how defects will be corrected if/when discovered
Recommendations
§
■ Give
A the vendor
standard, 15 days to
documented develop a Master
development Test
approach Plan
and that meets helps
methodology DMV approval
to ensure consistency and efficiency in development work. Lacking such such
§ consistency and efficiency, development work may be delayed.
Consider termination of the vendor’s contract if it cannot meet this deadline, as well as deadlines for submitting other key deliverables identified in the
§ DMV should suspend any further development activity until the vendor produces an acceptable Master Test Plan
C. Vendor Performance: Training Strategy and Planning
Activities and processes for this Assessment Area are in concert with § The vendor is contractually obligated to develop a Training Plan;
IEEE STD 12207 and 1220, including: according to its recent Master Project Schedule, this plan will be delivered
§ A detailed training plan is in place and approved by the organization for to the DMV by January 2018
end-user staff (business and technical) training § No stakeholder assessment has been conducted to determine current
§ End-user training programs are developed and established early in the staff skills sets and training needs and expectations of end users
program/project’s life cycle
§ Capability to “pilot” functionality of the system prior to go-live is being
defined
§ A process is defined to verify that users have adequate training in the new
system
§ A verification process is defined to help ensure that end-users follow the
expected process upon go-live
§ The program/project has a formal documented knowledge transfer plan
that has been approved by and that provides the foundation for ongoing
support
Impact
§ Without a stakeholder assessment the state will not be able to accurately assign the appropriate resources to training topics and end users. The state will
also not be able to identify the appropriate prerequisite skill sets needed prior to training and the skill sets post go live
Recommendations
§ Give the vendor 15 days to develop a Training Plan that meets DMV approval
■
§ A standard,
Consider documented
termination development
of the approach
vendor’s contract if it and methodology
cannot helps to ensure
meet this deadline, as wellconsistency
as deadlinesand
forefficiency
submittinginother
development work. Lacking
key deliverables such
identified in such
the
consistency andof
vendor’s Scope efficiency,
Work development work may be delayed.
§ A stakeholder assessment should be performed to determine skill sets and training needs of end users
D. Requirements Management and Benefits Realization: Scope Definition and Refinement
Activities and processes for this Focus Area are in concert with IEEE § The project scope, as defined in a Work Breakdown Structure (WBS) was
STD 1028 and 12207 including: not formally presented until the vendor submitted its first master project
§ A clearly defined project scope is submitted and approved schedule in October 2017
§ Project deliverables are clearly defined and submitted and approved by § Original requirements for SysMod were poorly written and do not conform
the management team and key stakeholders to accepted industry standards (e.g., 830-1984 — IEEE Standards)
§ A structured mechanism and process are in place to assess and § The vendor has not consistently conducted fit gap sessions to determine
implement scope changes and the project team fully understands and alignment of SysMod requirements to the solution elements it has
respects the change methodology/process proposed — and where configuration or customizations are required (and
potential scope changes)
§ Required reports, interfaces, conversions, extensions, forms, etc. are
defined § The skills and competencies of the vendor’s business analysts and
technical SME’s vary and requirements validation has been inconsistent
§ The above scope documents are submitted for review and approval via
across the SysMod team
protocols and authority defined in the program governance framework
§ As no formal change control plan has been adopted, its unclear how
§ Requirements are written per the 830-1984 — IEEE Guide for Software
proposed scope changes are addressed
Requirements Specifications. A Software Requirement should be Correct,
Unambiguous, Complete, Consistent, Ranked for importance and/or § Customizations, extensions, modifications, configurations, and interfaces
stability, Verifiable, Modifiable, Testable, and Traceable are not logged centrally
Impact
§ Without a defined scope, the DMV and the vendor have no agreement on an approach for structuring and staffing work to meet project objectives. Also, the
need for scope changes and the impact of these changes on the project schedule and budget cannot be accurately assessed if the scope was never
baselined
§ Without a comprehensive fit-gap assessment, it is impossible to determine the alignment of the original requirements to the vendor’s proposed solution.
Further, without this detailed fit-gap, the DMV has no basis for determining whether additional costs for customization are truly warranted
Recommendations
§ Requirements should be rewritten so that they meet accepted industry standards (e.g., IEEE) and the vendor should validate these requirements via DMV-
wide Fit/Gap sessions
§■ Give
A the vendor
standard, 15 days to
documented develop a Change
development approachManagement Plan that
and methodology meets
helps DMV approval
to ensure consistency and efficiency in development work. Lacking such such
§ consistency
Consider and efficiency,
termination of the development workifmay
vendor’s contract be delayed.
it cannot meet this deadline, as well as deadlines for submitting other key deliverables identified in the
D. Requirements Management and Benefits Realization: Requirements Management
Activities and processes for this Focus Area are in concert with Gartner § Requirements are being added or revised without any traceability back to
Best Practices including: a centrally managed RTM
§ A requirements development and validation process has been established § Consequently the system could go live with the ability to test a
to ensure that impacted business units are integrated into the overall requirement and without the desired functionality of the requirement
requirements development and validation process in order to ensure § Requirement could be changed or modified without identifying impact
requirements are testable on training, policies and procedures
§ Subject matter experts and points of contact have been identified for each § SysMod could go live violating policies and procedures, which could
business unit to be responsible for requirements validation compel the DMV to update requirements, training material and
§ Business unit subject matter experts and points of contact have been policies during a period where training and testing are occurring at
trained in requirements valuation to ensure requirements are testable the same time
§ Requirements traceability matrix documented and is being updated to § Most teams are not performing requirements validation sessions and
ensure requirements management and traceability throughout the those that are do not follow a consistent approach/methodology
program/project life cycle § Requirements are poorly written and in most cases untestable
§ The program/project is using formal change management and
configuration control process to manage and track system requirements
throughout the programs life cycle
Impact
§ Poorly written requirements will not allow for an appropriate level of testing needed to validate the complete functionality of the system
§ Based on the current condition of the RTM it will be very difficult to manage how scope changes will be logged and how the changes impact the
requirements
§ By not updating the RTM with scope changes it is likely that functionality is not be accurately tested or ever tested at all. This could lead to multiple defects
identified post go live. An inaccurate RTM could lead to voids in training materials as well
Recommendations
§ The vendor must establish processes and tools, subject to DMV review and approval, for centrally managing all requirements for the new project
§■ A standard,
These documented
processes/tools development
must approach and
include governance methodology
provisions helps toupdating
for reviewing, ensure consistency andrequirements
and managing efficiency in from
development
validationwork. Lacking
through such such
development and
consistency
testing and efficiency, development work may be delayed.
§ All ]requirements for the new project must be reviewed and validated with DMV staff to ensure that each requirement is clearly stated and testable
D. Requirements Management and Benefits Realization: Benefits Realization Planning
Activities and processes for this Focus Area are in concert with Gartner § A benefits map was created as part of the TIR approved by the state;
Best Practices, including: however, this map is simply an explanation of what KPI’s will be
§ Benefits realization approach is documented and approved measured for 3 benefit area’s
§ Describes benefits measurement approach § No explanation is provided in the TIR (or in other project documents)
detailing anticipated benefits and how these benefits will be measured
§ Aligned with the most current business case, benefits are still
long term
compelling & realistic
§ No component of the SysMod project team or third-party organization has
§ Reflects quantified and measurable, business and IT benefits
been assigned to formally identify and track benefits
§ Entire organization accountable for realization of benefits, baked into
resource management (e.g., Incentives, financial projections, etc.)
§ Benefits realization is inked to scope, and deployment, communications
and Organizational Change Management Plans
§ A neutral third-party organization is assigned to track benefits
Impact
§ The lack of an ongoing benefits realization program will not allow the project to discuss ongoing successes to state representatives and stakeholders. In
addition, the lack of a business realization program will not allow the state to measure and report performance against the business plan
Recommendations
§ DMV needs to clearly define anticipated benefits and determine how benefits will be measured over the course of the project.
§ A member of the PMO — or third-party organization should be assigned to formally identify and track benefits
E. Communications & OCM: Communications Planning and Execution
Activities and processes for this Assessment Area are in concert with § A Communication Plan has been prepared by the vendor and approved
IEEE STD 1028 and 12207 including: by the DMV PMO
§ A stakeholder analysis has been conducted to better understand the § The Public Information Officer is named as the person responsible for
information needs of stakeholders and end-users SysMod Project Communications, but was not listed as a contributor to
§ The program/project has a documented communications plan in place that the Plan
addresses internal and external program/project stakeholders § Key stakeholders were named in the plan but a stakeholder analysis was
§ The program/project’s leadership team conducts routine meetings to keep not performed to assess information needs and potential channels/media
all stakeholders apprised of the program/project’s progress and issues to be used in communicating with end users
§ The program/project provides routine updates and news related to the § Project updates are not being shared with DMV employees, possibly
program/project either on the web, through email or with a program/project creating uncertainty among staff regarding the status of the project
newsletter § Communication approaches and timelines are specified in the plan but
are not being followed
Impact
§ Inconsistent communication approaches will lead to multiple messages regarding how the new system will impact the various user groups and how the new
system will improve how users perform their daily required business tasks
§ Inconsistent messaging will not allow the state to accurately offer messaging regarding the status of the project and/or any successes realized
Recommendations
§ DMV should conduct a stakeholder assessment and the results used to drive a redesign of the current communication plan, as well as provide better
direction for OCM and training planning and execution
§ Project updates should be regularly updated and disseminated and approaches/timelines should be followed by all stakeholders
§ The DMV PMO should monitor the vendor’s execution of the plan. If the plan is not being executed, DMV should give the vendor 15 days to remedy any
issues noted
E. Communications & OCM: OCM Capabilities and Planning
Activities and processes for this Assessment Area are in concert with § The DMV appears to be responsible for Organizational Change
IEEE STD 1490 and Gartner Best Practices including: Management as this item was not included in the vendor’s scope of work;
§ An Organizational Change Management (OCM) Plan is prepared for the and yet no OCM plan has been developed to date
program/project and includes the following provisions: § The extent of DMV’s internal OCM capabilities/competencies and
§ Identification of key stakeholders in each functional area impacted by expertise in planning and executing organizational change management
the new system that will be part of the overall organizational change for a project of size and complexity of SysMod is not clear
team § Reportedly OCM leads have been identified — but, without a formal,
§ Governance and management support structure needed to address approved plan, approaches to OCM on the SysMod project will be
cross-functional organizational and process issues inconsistent, if performed at all
§ Processes to be used by the program/project team and organization to § No stakeholder assessment has been conducted to help determine
define the as-is and to-be business and operational processes communications, OCM and training needs
§ The development and documentation of success criteria that will be § There has been limited end-user involvement in the project to date — this
used to measure the organizational change realized vs. what was could likely lead to challenges at go live due to inconsistent or unrealistic
envisioned expectations
§ Integration of the organizational change planning activities with other
key elements of the program/project plan (e.g., communications,
staffing, etc.)
Impact
§ Without an OCM plan and skilled resources to lead organizational change activities, stakeholders are typically anxious, uninformed and ill-prepared to use
the new SysMod solution. Lack of a well-orchestrated OCM effort may lead to end-user disenchantment and resistance to using the solution
§ The lack of a plan could also lead to training and testing challenges as the OCM Plan usually provides some of the information needed for:
§ Training Material Development
§ Test Script Development and Test Scenario Identification
Recommendations
§ DMV needs to prepare an OCM plan, based on results from a stakeholder assessment, to fully understand and prepare stakeholders for the business,
technical and cultural impacts of the new system
§ As OCM planning and execution can be very complex, DMV may wish to retain an external firm specializing in this field
§ The OCM plan shall also be prepared to complement Communications and Training Plan
Internal Audit Report Findings and Status of Response Implementation
In the table below, Gartner comments on the current implementation status of the DMV and vendor response to
addressing findings in the Division of Internal Audit’s review of the SysMod project (as of December 2017).
Key Audit Finding DMV/Vendor Response Gartner Comments on Current Response Status
Contractor did not provide Vendor pledged to provide staff with The scope of Gartner’s risk assessment did not include a
proposed personnel appropriate project management, technical review of the capabilities of post-audit additions or changes
and DMV domain expertise to current vendor staff
Contractor did not provide Vendor prepared technical writer style guide Gartner randomly reviewed selected vendor documents
personnel proficient in English and is having technical writers review each posted to Confluence — those documents that we reviewed
project deliverable were reasonably well written
Representations in the RFP Vendor replaced its Program Manager and Vendor is late in preparing many key planning deliverables,
were not fulfilled began addressing core project management which suggests many project management issues have not
issues/shortcomings been fully resolved
Amend contract with approved Vendor and DMV committed to developing a DMV approved the vendor’s Master Project Schedule in
Master Project Plan Master Project Schedule by late September October 2017. Vendor is already late in meeting many key
2017 milestone dates identified in this schedule
Establish protocol for review DMV and vendor renewed agreement to use Draft deliverables are posted to Confluence. It is not clear
and approval of deliverables Deliverable Signoff process and will leverage that all deliverables are posted here — or how review and
Confluence and Jira to facilitate more efficient approval processes are expedited by using this tool
review and approval process
Follow issue agreed resolution DMV and vendor both commit to follow the Project issue log is maintained in MS Excel file. Folders for
procedures Issue Resolution Process defined in RFP issue logs have been created in Confluence — but, its not
going forward clear issues are managed at a program level using this tool
State resume not used for Vendor’s onboarding procedures updated to Vendor onboarding procedures document (dated
contractor staff require use of state resume for new staff 10/10/2017) has been updated to require use of the State
Resume Form
Appendix A:
Project Health Assessment Approach and
Timeline
Gartner’s Project Health Check Approach
Gartner’s Project Health Check approach comprises the set of four interrelated tasks outlined below.
Our risk assessment framework, as well as the activities and deliverables associated with each task
are elaborated in following slides.
Week 1 Week 1–4 Week 4–7 Week 8–9
TASK 1 TASK 4
TASK 2 TASK 3
Develop Health Check Finalize and Present
Assess Project Risks Develop Draft Project
Assessment Plan and Findings and
and Issues Health Check Report
Schedule Recommendations
Refine Findings and

Prepare Executive
Conduct Health Check Conduct Interviews and Compare with Accepted
Summary and Detailed
Kickoff Meeting Group Meetings Best Practices (e.g.,
Final Report
Gartner, IEEE, PMI)
Identify Preliminary
Develop Prioritized Set of
Request and Review Findings and Corroborate Present Final Report to
Recommendations to
Project Documentation Via Additional Interviews DMV Leadership
Address Findings
and Documentation
Schedule Interviews with Validate Findings and

Review and Validate Initial
Project Sponsors, Team Recommendations with
Findings with Project
Members and Project and DMV
Leadership
Stakeholders Leadership
Project Schedule
Project Review Initial Review Draft Submit/Present

Planning and Assessment Health Check Final Health
Kickoff Meeting Report Report Check Report
Week 1 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8 Week 9
Deliverables:
Ongoing Project Management and Status Reporting § Kickoff Meeting Materials
§ Project plan
§ Bi-weekly Status Reports
Conduct Interviews
and Review
Documentation
Deliverables:
Define and Validate Initial § Initial Assessment
Findings Findings
§ Initial Assessment Report
Conduct Detailed Risk Assessment and Deliverables:

Review Draft Health Check Report with DMV § Draft of Project Health
Check Report
Deliverables:
Update and Finalize § Final Project
Health Check Report Health Check
Report
Appendix B:
Risk Category Definitions
Appendix B
Risk Category Definitions — Strategy Phase
Risk Category Definition

Strategy — Program/Project Governance The ability of the organization to place importance through demonstrated leadership commitment
Strategy and governance accountability for scope, benefits, resource, schedule, communications, and
risk/issues management.
Strategy — Business Case The extent to which costs and benefits have been articulated, linked with the proposed solution and
process scope, and vetted with key business and IT stakeholders to ensure input and ownership.
Strategy — Risk Mitigation Strategy The extent to which experienced personnel and proven methodologies have been applied to
identify risks, contingencies, and level of effort required to successfully mitigate the issues that
may impede the success of the program.
Strategy — Executive Support The extent to which business and IT executive leadership is engaged in the program as evidenced
by their understanding of the program scope, business case, and challenges. Also examines the
perspectives of the project leadership team regarding how well they feel executive management is
involved and understands the critical role they should play to ensure program success.
Strategy — Scope Definition The ability of program leadership to fully define and provide the rationale for business process
scope, stakeholder scope, division/operating company scope, application (legacy and new) scope,
and deliverable scope along with assumptions. Also includes a clearly defined change request
process.
Strategy — Organization/Project The project management and oversight capabilities of the organization and the overall approach to
Management Capabilities using those capabilities to support the implementation of the new solution.
Strategy — Technology Infrastructure & The capability of the organization’s data center and distributed computing infrastructure to support
Processes Strategy the new solution, the overall approach the organization is taking to modify the infrastructure, and
the processes in place or being implemented to support the new solution.
Appendix B
Risk Category Definitions — Planning Phase

Planning — Program/Project The extent to which program management processes (e.g., schedule management, issues,
Governance Plan management, resource management, etc.) are defined and governance mechanisms and
structures are documented and in place across the enterprise and project teams. Also assesses
the experience of the program leadership personnel.
Planning — Risk Management Planning The extent to which program leadership has planned for processes and multiple perspectives to
address ongoing overall risk as well as domain-specific risks and issues in a timely and effective
manner. Contingency planning effectiveness is also examined.
Planning — Schedule Planning The extent to which program leadership has estimated, scheduled, communicated, and managed
the critical path activities, key milestones, and the enablement of high quality deliverables.
Planning — Budget Planning Examines the level of risk related to the approved budget, it’s feasibility, and the planned
processes to enable program leadership to track, monitor, and control the budget.
Planning — Scope Refinement The ability of program leadership to review the rationale for business process scope, stakeholder
scope, division/operating company scope, application (legacy and new) scope, and deliverable
scope along with assumptions and to determine/address gray areas in which clarity is lacking.
Planning — Resource Planning The ability of the organization to plan for and create a dedicated internal and external team, of the
required size and skills, backfilled appropriately, to enable focused attention to the program effort.
Planning — Communication Planning The extent to which program leadership has established focused accountability for the
communications effort and that the accountable team has established a network of resources and
a plan to engage those resources to build and execute an effective communications program.
Planning — Organizational Change The extent to which program leadership has established focused accountability for the change
Management Planning management effort and that the accountable team has established a network of resources and a
plan to engage those resources to build and execute an effective change program.
Appendix B
Risk Category Definitions — Planning Phase (Cont)

Planning — Security Planning The extent to which program leadership has established focused accountability for security and
compliance controls design, build, and test activities and that the implementation methodologies
explicitly support integrated, role-based security design.
Planning — Development Planning The extent to which program leadership has articulated a development strategy that prioritizes and
defines development activities and methodologies/tools from design through deployment. Also
assesses the ability to adhere to guiding principles and approaches such a: “minimize
customizations”; “iterative build cycles”; “off-shore vs. on-shore development”; etc.
Planning — Test Planning The extent to which program leadership has developed a test strategy that defines the types of
tests, tools/methods to leverage, the accountability for tests, and considers the schedule and
participation needed to ensure high quality test results when executed.
Planning — Data Conversion Planning The extent to which program leadership has developed a data conversion strategy and plan that
defines the types of conversions, the conversion options, tools/methods to leverage, the
accountability for conversions, the data cleanup approach, and considers the schedule and
participation needed to ensure high quality data conversion results when executed. Also examines
the explicit communication of legacy systems being retired.
Planning — Training Strategy & The extent to which program leadership has developed a training strategy that defines the types of
Planning training, tools/methods to leverage, the accountability for training, and considers the schedule and
participation needed to ensure the end-users are self-sufficient in the operation and the
maintenance of the software after go-live.
Appendix B
Risk Category Definitions — Planning Phase (Cont)

Planning — Integration/Interface The extent to which program leadership has developed an integration/interface plan that will define
Planning the schedule and strategy for inter-process communications and subsystem (i.e., third- party or
legacy bolt-on, shadow system interaction, etc.). The plan should also consider the participation
needed to build/test the interfaces as per proven best practices as well as the ability to create a
stable integration environment.
Planning — Benefit Realization Planning The extent to which program leadership has developed a benefits realization plan that defines the
benefits (e.g., hard or soft), the metrics, the accountability for measuring benefits, and considers
the schedule and participation needed to ensure that both hard and soft benefits are realized.
Planning — Technology Infrastructure & The plans to update the organizations data center and distributed infrastructure to support the new
Processes Planning solution.
Appendix B
Risk Category Definitions — Execution Phase

Execution — Program/Project The extent to which program leadership executes key governance processes (e.g., scope,
Governance Execution schedule, resource, budget, resource, requirements, etc.). Examines the effectiveness of input
rights and decision rights with regards to each of these processes.
Execution — Risk Management Examines how well risk is being managed on an ongoing basis at the program level and at the
individual team level. Looks at processes and multiple perspectives to address ongoing overall risk
as well as the ability for domain-specific risk and issues management in a timely and effective
manner. Also examines how well internal and external risk management and contingency planning
roles are being leveraged.
Execution — Schedule Management Examines how well individual and team time is being proactively estimated, scheduled, maintained,
communicated, and managed to hit critical path milestones with high quality deliverables. Looks at
specific cases for how schedule change requests are managed.
Execution — Budget Management Examines how well the budget is being managed on an ongoing basis at the program level and at
the individual team level. Looks at specific cases, if applicable, for how budget change requests or
contingency budget usage requests are managed.
Execution — Scope Management Examines how well the business process scope, stakeholder scope, division/operation scope,
application (legacy and new) scope, and deliverable scope is being managed on an ongoing basis
at the program level and at the individual team level. Looks at effectiveness of any scope
refinement activities from both a functional and a technical perspective to control complexity
without sacrificing benefits.
Execution — Resource Management Assesses program leadership’s ability to achieve optimal capacity from constrained internal and
external resources through proactive resource loading visibility, effective internal/external team
collaboration, resource changes (when required), training, and knowledge transfer.
Execution — Communication The extent to which program leadership and the accountable team is executing an effective
Management communications program. Examines upward, downward, and cross-team communications as well
as communications to extended team and other external parties required for success.
Appendix B
Risk Category Definitions — Execution Phase (Cont)

Execution — Organizational Change The extent to which program leadership is working with the focused Change Management team to
Management Execution communicate and prepare the organization for specific change impact items that are crucial to
achieving optimal solution usage and business case realization.
Execution — Vendor Performance The extent to which program leadership has aligned itself with highly capable and knowledgeable
Management vendor support personnel during all implementation phases. This also includes examining the
vendor support processes and service-level agreements in place.
Execution — Requirements Management Examines the ability to document and trace requirements through Design, Build/Test, Deploy, and
Post-Implementation phases and to link requirements to scope and business case management
activities. Also assesses how well shadow system requirements are considered in the overall
implementation activities.
Execution — Security Execution The extent to which program leadership is executing the security and compliance controls design,
build, and test activities and ensuring that the implementation methodologies explicitly support
integrated, role-based security design.
Execution — Test Management Examines the team's ability to manage the overall test process, specifically looking at the
movement of modules from development to each successive type of testing, the leveraging of
appropriate resources across test activities and the sharing of testing tools and scripts.
Appendix C:
Interviewees and Documents Reviewed
Gartner Interviewed the following Stakeholders and Subject Matter Experts
§ Aman Sethi — SysMod Program Director

§ Vilas Dravid — SysMod On-site Program Manager
§ Charon Gowda — SysMod Enterprise Architect
§ Rebecca Gallagher — DMV Services Supervisor I/SysMod DMV SME Team 1
§ Charles (Zeb) Gray — DMV Technician III/SysMod DMV SME Team 1
§ Peter Olson- DMV MS&P SME
§ Wayne Bahmiller — DMV Management Analyst III/SysMod DMV SME Team 2
§ Alison Day — DMV Technician III/SysMod DMV SME Team 3
§ Michelle Moorehead — DMV Technician IV/SysMod DMV SME
§ Brandi “Alex” Smith — DMV Public Information Officer
§ Mark Froese — MVT Administrator/SysMod Executive Steering Committee — MVIT
Gartner Interviewed the following Stakeholders and Subject Matter Experts (Cont)
§ Jude Hurin — SysMod DMV Executive Steering Committee — MS&P

§ Louise Roberts — DMV Application Lead/SysMod MVIT Resource
§ Bill Bernard — DMV Network Manager
§ Phani Mallampalli — SysMod Data Migration Lead
§ Frank Maiden — SysMod BPA Team 1
§ Dana Purcell — SysMod BPA Team 3
§ Brittney Finlay — SysMod Project Coordinator Team 4
§ Dionna Perez — SysMod BPA Team 5
§ Amber Galperin — SysMod BPA Team 4
§ Bernadette Nieto — SysMod BPA Team 2/SysMod PMO
§ Lori Poirier — SysMod Testing Lead
§ Praveen Madhusudhanan — DMV Applications Staff/SysMod MVIT Resource
Gartner Interviewed the following Stakeholders and Subject Matter Experts (Cont)
§ Kiran Rasakatla
§ Dianna Arnold — SysMod SME Team 6
§ Lamar — Interfaces Lead
§ Thomas Harlan — DMV Security Officer
§ Elizabeth “Ann” Conlin — PM Team 6
§ Rene Jamie — PM Team 5/PMO
§ Barbara Middleton — PM Team 3
§ Kade Rogers — PM Team 4
§ Arun Kumaran — PM Team 2
§ Dave McTeer — PM Team 1/PMO
§ Ismael “Izzy” Hernandez — SysMod Sr. Program Manager/PMO/DMV Executive Steering
Committee
Gartner’s fact finding activities included a review of the following project artifacts
§ Audit Report (NV Division of Internal Audits) § Correspondence Plan

and DMV Response § Procurement Plan
§ Vendor Scope of Work § Project Management Tools
§ Deliverable Acceptance Criteria Documents § Project Methodology
§ Business Case/Vision Documents § Resource Onboarding Process
§ Project Plan/Schedule
§ Program Management Issues Log
§ Risks Log § Hard-Disk Drive Standards
§ Issue Log § SharePoint Standards
§ Communication Plan § Org Chart
§ Quality Assurance Plan § Org Chart with Names
§ Current/Future/Target Business Processes
§ State Audit Recommendations and Response
§ Requirements
§ Architecture Plan
§ QAT Plan
Gartner requested the following documents, but they were not provided
Document Requested Reason from SysMod PMO For Not Providing
Governance Plan Not Available
Risk Plan Risk Plan — To Be delivered by the vendor over the weekend- this was never provided to the Gartner team
Scope Log Scope Log — Not available

Change Log Change Log — No log available — However, we keep track of all documentation by Change Request
Project Charter Not created, using RFP
OCM Plan To be provided by Jude Hurin — this was never provided to the Gartner team
Security Plan This document to be created prior to go-live as part of the project — No security plan provided for IFTA.5
release
RTM The requirements can be found in the RFP
Configuration Plan This document will be completed as part of the Low-Level Design Document
Testing Plan (Unit, SIT, UAT, and Each testing phase will have its own test plan. Unit, SIT, Performance
Performance)
Interface Plan We have a Detailed Business Requirements (DBR) and High-Level Design for all interfaces; Templates can be
provided
Data Management (Cleansing, We can provide this on Monday — this was never provided to the Gartner team
Controls, Conversion/
Migration/Validation)
Training Plan This document to be created prior to go-live as part of each release
Cutover Plan This document to be created prior to go-live as part of each release
Deployment Plan This document to be created prior to go-live as part of each release
Knowledge Transfer Plan This document to be created prior to go-live as part of each release
Vendor Management Strategy The vendor management is done by the PMO using the RFP as the guideline. We track all program issues in
the Program Management Issues Log
Post Implementation Support Plan This document to be created prior to go-live as part of each release
Appendix D:
Sample Position Descriptions
Sample Position Description
Project Director (Manager)
General Responsibilities
§ Identify project deliverables, milestones and required activities and tasks and ensure they are scheduled and tracked
§ Oversee development of work plans and staffing model for project activities, iterations or phases, and arrange for recruitment or assignment of
project personnel
§ Review project plans to assess/determine time frames, staffing requirements and allotment of available departmental resources to various project
phases
§ Review/approve work plan of external service provider or consultancy contracted to execute project
§ Implement or direct the execution of project communication plan
§ Conduct or direct risk assessment and mitigation in accordance with risk management plans
§ Monitor project activities, ensuring the currency, quality and integrity of the information, while providing consistency in content and "look and feel"
across the enterprise
§ Ensure that project goals are accomplished and in line with business objectives
§ Assign duties, responsibilities and scope of authority to project personnel
§ Direct and coordinate activities of project personnel to ensure project progresses on schedule and within budget
§ Review status reports prepared by project personnel, and modify schedules or plans as required
§ Establish standards and procedures for project reporting and documentation
§ Help establish and execute project governance requirements
Education: Bachelor's degree in business information systems (IS), computer science or related field, or equivalent related IT experience. MBA in IS or
other master's degree preferred. PM training and certification desirable, but not required.
Experience: Seven or more years of progressively responsible IT experience, with sensitivity and commitment to business problem solving; five years
successfully managing complex and risky projects; Five or more years of BU or BU-facing experience, with sensitivity and commitment to business
problem solving; Demonstrated experience in leading diverse teams.
Complexity: This is a senior-level project manager position. The Project Director (Manager) is expected to demonstrate the required competencies at an
advanced proficiency level. These are generally additive to the competencies of a standard (or journeyman) project manager.
Project Director (Manager) (Cont)
Other Knowledge and Skills
§ Distinctive blend of business, IT, financial and communication skills, as this is a highly visible position with substantial impact
§ Knowledge of project management methodology and experience or familiarity with major defined PM approaches (for example, PMI, Prince II or
agile)
§ Significant knowledge of project planning/scheduling tools (for example, Microsoft Project and Open Workbench), with solid track record of practical
application
§ Exceptional verbal and written communication skills; expertise in setting and managing customer expectations
§ Effective influencing and negotiating skills in an environment where this role may not directly control resources
§ Strong knowledge and understanding of business needs, with the ability to establish and maintain a high level of customer trust and confidence
§ Ability to support a tactical resource plan by mapping future demand to forecast project completions in the project portfolio
Business (Process) Analyst
General characteristics
§ Serves as a liaison between the business community and the IT organization to help provide technical solutions to meet user needs
§ Possesses expertise in the business unit(s) they support, as well as, an understanding of the IT organization’s systems and capabilities
§ Analyzes business partner’s operations to understand their strengths and weaknesses to determine opportunities to automate processes and
functions
§ Assists in the business process redesign and documentation as needed for new technology
§ Translates high level business requirements into functional specifications for the IT organization and manages changes to such specifications
§ Educates the IT organization on the direction of the business
§ Negotiates agreements and commitments by facilitating communication between business unit(s) and IT from initial requirements to final
implementation
§ Possesses an understanding of technological trends and uses this knowledge to bring solutions to business units supported to enhance the
enterprise’s competitive edge
§ May make recommendations for buy versus build decision
Education: Bachelor’s Degree in Computer Science, Information Systems, Business Administration, or other related field. Or equivalent work
experience.
Experience: Typically has 3 to 5 years of relevant technical or business work experience.
Complexity: Intermediate professional level role. Works with business partners within one business function to align technology solutions with
business strategies. Demonstrates an informed knowledge of a business area to resolve problems on an ongoing business. Supports several
moderately complex business processes. Works on multiple projects as a project team member, occasionally as a project leader. May coach more
junior staff.
Business (Process) Analyst
Other Knowledge and Skills
§ Exceptional analytical and conceptual thinking skills
§ Ability to influence stakeholders and work closely with them to determine acceptable solutions
§ Excellent communication and documentation skills
§ Fundamental analytical and conceptual thinking skills
§ Experience creating detailed reports and analyses
§ Competency in Microsoft applications including Word, Excel, and Outlook
§ Excellent planning, organizational, and time management skills
Gartner Contacts
Mark Lennon Bryan Groden Sara McDermott

Managing Partner Senior Director Consultant
Gartner Consulting Gartner Consulting Gartner Consulting
+1 916 832 1494 +1 919 696 3774 +1 925 752 8890
Mark.Lennon@gartner.com Bryan.Groden@gartner.com Sara.McDermott@gartner.com

Gartner Final Health Check Report and Recommendations

Hochgeladen von

Dokumentinformationen

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Verfügbare Formate

Gartner Final Health Check Report and Recommendations

Hochgeladen von

Copyright:

Verfügbare Formate

Independent Health Check of

DMV System Modernization

CONFIDENTIAL AND PROPRIETARY

Based on Gartner’s experience, projects with these types of attributes typically

A graphic depiction of the current Detailed findings mapped against

Options for Moving Forward on the Final Report & Detailed

Recommendations to Address the Findings Managed Risk

1. Technical Solution and Scope

6. Operational Transition Planning

B. Project Management and

Recommendations to Address the Findings Managed Risk

3. Vendor Performance Management

4. Requirements Management and Benefits Realization

Recommendations to Address the Findings Managed Risk

5. Communications and Organizational Change Management

Implement these Reassess & Renegotiate.

February 2018 March 2018 April 2018 May 2018

W1 W2 W3 W4 W5 W6 W7 W8 W9 W10 W11 W12 W13 W14 W15 W16

1. Restructure DMV PMO and Project Staffing Model

2. Establish Clear Project

3. Revise and Validate Requirements

Foundational Activities Conduct Fit-Gap to Assess Define Staffing, Contract &

Begin Preparing RFP for New System Integrator and Solution

1.0 PMO Statement of Purpose .................................................................................. 2

Project Processes (e.g., risk/issue management, change

Manage Vehicle Operator Manage Regulation and

Manage IRP/IFTA Collect/Maintain

Manage Vehicle Records Provide Customer Service

Maintain Manage/ Receive and

Change or Issue Salvage Develop Issue License

Most Costly Least Costly

Financial Impacts — the degree to which the option is likely to 4 3 2 1

Most Difficult Least Difficult

Most Time Least Time

that the DMV’s selected vendor can successfully deliver a 4 3 2 1

Financial Risk RISK/IMPACT CONSIDERATIONS

Financial Risk RISK/IMPACT CONSIDERATIONS

Financial Risk RISK/IMPACT CONSIDERATIONS

Vendor Track Record 4 2 2

Weighted Risk Score 2.70 2.75 1.40

B. Project C. Vendor D. Requirements E. Communications &

Week 1 Week 1–4 Week 4–7 Week 8–9

Refine Findings and

Schedule Interviews with Validate Findings and

Project Review Initial Review Draft Submit/Present

Week 1 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8 Week 9

Conduct Detailed Risk Assessment and Deliverables:

Risk Category Definition

Risk Category Definition

Risk Category Definition

Risk Category Definition

Risk Category Definition

Risk Category Definition

§ Aman Sethi — SysMod Program Director

§ Jude Hurin — SysMod DMV Executive Steering Committee — MS&P

§ Audit Report (NV Division of Internal Audits) § Correspondence Plan

Scope Log Scope Log — Not available

Experience: Typically has 3 to 5 years of relevant technical or business work experience.

Mark Lennon Bryan Groden Sara McDermott

Das könnte Ihnen auch gefallen