Uniting Three Families of Risk Management--

Complexity of Implementation x 3
Dr. Thomas H. Holzer
National Geospatial-Intelligence Agency
12310 Sunrise Valley Dr.; Mail Stop: P-59
Reston, VA 20191
Thomas.H.Holzer@nga.mil
NGA asserts no copyright claim under Title 17 of the United States Code with respect to any copyrightable material compiled in this paper, nor
does NGA require any compensation for its use. Published and used by INCOSE with permission.

Abstract. This case study presents how a government agency overcame difficult challenges to
establish a solid foundation for the management of its three families of risk- program, enterprise,
and joint enterprise. Establishing an efficient and effective risk management process in a single
development program is very challenging- culture resistance to change and unwillingness to
share information viewed as negative prevail. There is additional complexity convincing people
to adopt a process that is part of the bigger organization and sharing information regarding their
ability to achieve program objectives. Now convince two different organizations to share
information with each other. The outcomes show how the process can be scaled, adapted,
integrated and adopted across the complex systems and diverse organizations and enable
program teams to identify, assess, and mitigate complex risks. For an organization aspiring to
establish an internal or joint enterprise risk management program, these findings are a head start.

Implementing a Risk Management Program

Performing Risk Management (RM) is an essential element of managing any program regardless
of its size or complexity. Practicing risk management1 on any activity is one means of
improving the likelihood the activity will successfully meets its objectives with as few surprises
and ‘show-stopping’ events as possible. The rigor and breadth to which RM is performed can be
scaled and tailored to the particular effort based on the activity’s size and complexity and in a
manner not affecting the RM process integrity or activity’s outcome. With today’s trend to
develop highly complex and integrated systems to form enterprises (i.e., system-of systems,
family-of-systems, or whatever your favorite term is), RM implementation needs to be
considered at the enterprise and not strictly individual system level (i.e., family). Finally on joint
programs, organizational boundaries should not be a barrier to implementing RM process. As
discussed at the end of the paper, there are benefits to implementing RM at all levels.

The Case Study. This paper focuses on actual challenges faced, resolutions applied and results
achieved. Also provided are the processes, practices and tools applied by the author and RM
teams, at the US Department of Defense National Geospatial-Intelligence Agency (NGA) and a
collaborating organization known as a mission partner (MP). NGA’s initial process was

1
While ‘risks’ and ‘issues’ are separate entities, this paper’s discussion of risk includes issues. Risk is a measure of
the potential inability to achieve overall work effort objectives within defined cost, schedule, and technical
constraints. An issue is a problem or event that has already occurred or that has a 100% probability of occurring.
(NGA 2004)
implemented in September 2000 and appraised at Capability Maturity Model Level 32 in October
2003 through an external appraisal of NGA’s acquisition processes based on the Federal
Aviation Administration Integrated Capability Maturity Model (FAA-iCMM) (Ibrahim 1997).
NGA and the MP implemented a joint enterprise3 risk process in March 2002. The challenges
and resolutions presented (and recommended for consideration by others in similar
circumstances) are based on the lessons really learned by NGA. Overcoming the
4
implementation and institutionalization challenges are enabling program management teams,
joint technical working groups and their risk managers to better focus on the real and potential
mission impacting events. Thus, they will improve systems engineering management
performance and the quality of delivered capabilities.

NGA’s Risk Management Process. NGA’s Enterprise Risk Management Process (Figure 1)
was tailored from the FAA-iCMM Version 2 (Ibrahim 1997) and integrated with other NGA
acquisition management processes. The establishment and implementation of the risk
management process was consistent with recommendations provided by (DoD 2003) and best
practices cited by (Conrow 2005). Using established processes and practices provided credibility
not to mention a better chance at being successfully repeated.

Enterprise Risk Management (ERM) Process Overview

Entry Criteria
• Project Kick-Off
Initiate Execute and Control
Prim ary Inputs
• Any potential risk, issue, or
opportunity event introduced ERM 01
at any time during the w ork Develop ERM 02 ERM 03 ERM 04 ERM 05 ERM 06
effort’s life cycle by (but not ERM Identify Analyze Plan Monitor Control
limited to) the follow ing: Strategy
• Work effort WBS
• Work effort requirements
• Security Assessments
• Joint risks, issues, and
opportunities w /Mission
Partners
• Readiness liens
• IT investment data Enterprise Enterprise
Project Configuration
• Alternatives analyses Needs and Security Readiness
Management Manage ment
Requirements Engineering (ER)
Prim ary Outputs (PM) (CM)
(ENR) (ESE)
• Risk, Issue, Opportunity
Profiles
• Risk Manage ment Plans
Quality Technology Pre-Aw ard and
• Action Plans Alternatives Measurement
Assurance IT Investments Insertion Contract
• Status Reports Analysis (AA) Process (ME)
(QA) (TI) (PC)
• Measurements
Exit Criteria
• Work effort termination Primary NGA Process Interfaces ERM Process Activ ities Future Process and/or Other Interfaces TBD

Figure 1. Enterprise Risk Management Process.

Nature of implementation challenges and their resolutions. Challenges faced stemmed from
a general lack of understanding of the purpose of and benefits from performing RM. The
challenges increased moving from system to enterprise to joint enterprise implementation. Since

2
Capability Level 3: A defined process is a managed, planned and tracked (capability level 2) process that is
tailored from the organization’s set of standard processes according to the organization’s tailoring guidelines; has a
maintained process description; and contributes work products, measures, and other process improvement
information to the organization’s process assets. (Ibrahim 2001)
3
The integration of two or more independent enterprises.
4
Making the practice of Risk Management a second nature day-to-day activity across many systems
NGA’s primary mission is not acquisition management, most of those responsible for program
management and systems engineering did not have the experience with RM processes and
therefore an appreciation of RM. The basic nature of challenges faced did not vary greatly when
going from individual system to enterprise to joint enterprise risk process implementation
however the difficulty in resolving the challenges increased due to the larger number of people
and different cultures involved (i.e., internal program, NGA, MP). The challenges faced
centered on peoples’ unwillingness to change the way they were doing things and realization that
identifying shortfalls through risks and issues was a sign of good leadership and management.
NGA’s initial challenges were in convincing system level development programs that
performing RM was in their best interests. Managers needed to be convinced that making risk
known would be positively recognized and at times ‘rewarded’ with an allocation of risk
mitigation funds. This value had to be recognized by the program team. The second set of
challenges faced was in convincing programs whose systems were being integrated to
collaboratively and cooperatively manage risks evolving from the integrated relationships. In
this enterprise risk management family, there were more risks to identify (outside the individual
systems) and more people to bring together to assess and mitigate risks. Again, the added value
of more work had to be recognized by both systems’ team. The third further compounding and
greatest set of challenges was faced when joining two enterprises managed by distinctly different
organizations to create a ‘joint enterprise’. There were more diverse leaderships, objectives,
motivations, and other cultural views (and ways of doing RM) to meld.

Challenges faced at lower levels were encountered and more difficult to overcome at the next
level due to the greater diversity of players and paradigms. NGA’s RM team was not only trying
to overcome internal implementation challenges, it was facing many of the same but more
challenging challenges with its MP. To lessen the time, effort and frustrations of overcoming
resistance, solutions applied to a lower family were tailored to be equally useful and successful
when applied in other families. Success was achieved by overcoming cultural differences,
accepting the ideas of others and learning that the identification and sharing of risk information
on ones system was a good thing.

Paper’s Objective. The objective of this paper is to educate and better prepare those planning
or trying to implement different ‘families’ or levels of RM- on a single development system
program, on a single enterprise (i.e., system-of-system) or across enterprises spanning multiple
organizational boundaries. While one focus of the paper could be seen as the need to instill
discipline in project management, systems engineering and risk management, the real focus is on
the ‘How to’ and ‘Why to’ overcome the challenges. The compounding challenges faced and
how they were overcome to achieve a pattern of success while moving from single system to
joint enterprise RM process implementation are identified. For those having problems
overcoming resistance at the various implementation levels, this case study provides a starting
point for anticipating, reducing, and avoiding problems or failures. They should also consider the
presented lessons observed and resolutions found then decide if they too can learn from them.

NGA Risk Management Team Evolution. The initial risk management team was formed in
August 2000 to develop, implement and manage a rigorous systematic approach to managing
cost, schedule and performance risks and issues within NGA’s major systems acquisition
programs. The team’s scope quickly evolved from a within individual development systems
programs perspective to application across the systems (i.e., enterprise). In February 2001, the
Enterprise Risk Management (ERM) Team was chartered and today still includes government
and contractor personnel experienced in RM. In March 2002, their scope increased further to
include risks and issues related to the NGA-MP Joint Enterprise. As NGA’s facilitator of RM at
the system, enterprise and joint enterprise levels, the ERM Team continues to be on the front line
dealing with managing the process and its product. They provide the necessary foundation and
day-to-day oversight for meeting and resolving the challenges across all families of risk.

Challenges and Resolutions

The progression of challenges encountered will be discussed in succession of the system level,
enterprise level and joint enterprise level.

System Level Risk Management Challenges

Risk Management Program Implementation. NGA’s Enterprise Risk Management Process
was originally instantiated within individual acquisition programs. Each program team focused
just on their program’s risks. The most immediate and difficult challenge was getting the
program managers and their team to buy into the necessity of a formal risk management
activity5. RM was first perceived as a lot of extra work and oversight created by highlighting
their problems. Senior leadership sponsorship and education provided the resolutions. The buy-
in created by senior leaders, starting with the NGA Director, repeatedly voicing their support for
RM and holding their peers and subordinates accountable was the single greatest resolution to
many programs’ implementation barriers. People began to see risks RM as a benefit not a
hindrance. In reality, many programs were already performing informal risk management but
did not realize it because they were not familiar with the process. Through in-class, one-on-one
and computer based training, as the teams realized the additional work was less than expected
they were more receptive to adopting the formal practices. Making risks and issues a formal
agenda item in program management reviews further promoted accountability and increased
system level process implementation. By doing, documenting and presenting risk assessments
and mitigation planning, program managers saw they were able to get ahead of potential impacts
and in many cases receive additional funds to mitigate their risks and issues. While the
formalized process brought an increased ability to efficiently and effectively identify and
mitigate risks, some programs still resisted. Program Managers who fully understood the
benefits were then called upon to convince their reluctant peers.

Increasing institutionalization momentum. Institutionalization was the next, albeit more

manageable, challenge. The Risk Management Team increased their outreach, training and user
support initiatives to meet with more new groups needing to implement RM and helping others
solve process application difficulties. Today, the consideration of risk in strategic planning,
decision making, and program start up has become more prevalent. New opportunities to
integrate RM with other processes continue to be pursued.

5
Formal risk management activity considers formal risk analysis, documenting mitigation planning and monitoring,
lessons are learned, cross-organization communication to manage expectations, communicating successes,
Process and product oversight. Being able to independently ensure the risks and issues were
being properly managed became the next challenge as program teams’ ability to manage system-
level risks and issues matured. Resolution has been through the consistent and persistent
oversight by the Risk and Opportunity Management Board (ROMB). Established by the ERM
team, the ROMB facilitates the communication and validation of risk, issue, and opportunity
information at the enterprise-level across NGA programs. In addition to pre-coordinating the
presentation of all risks and issues planned for review at the monthly Risk Management Core
Team (RMCT) Meetings, they review risks and issues identified by many sources for enterprise
and joint enterprise level impacts. The sources include system risk managers, and as discussed
later, enterprise-level program offices, joint enterprise working groups and the Joint Risk
Working Group (JRWG). The government led RMCT is the risk management authority
comprised of government and contractor representatives from NGA’s organizations and
development programs. The RMCT assesses process performance and mitigation plan
completeness and viability; validates impact findings; and reviews status of critical risks and
issues. Gaining momentum was achieved in part by making the RMCT a government led forum.
Doing so put them in charge and gave explicit government ownership furthering their buy-in of
the risk process. While contractors are generally the ones implementing mitigations, the
government sees itself as the ultimate risk owner and mitigator. Government involvement (while
it still needs to be greater) has been strong and a source of the process’s success.

Cataloging and tracking risks and issues. Efficient access and maintenance of the risk records
was another early challenge whose complexities grew as risks in all families were identified. The
resolution, developed by the ERM Team, evolved from four years of experience- a single risk
entry and repository- the Risk, Opportunity and Issue Tool (RIOT). This SQL-based tool
captures a wide range of essential information to include: risk description; cost, schedule and
performance exposure; risk owner and manager; change history; detailed risk mitigation plan and
mitigation status. Read-access to risks in the RIOT is available to anyone with a log-on, but
write-privileges are limited to program risk managers, risk owners and the ERM Team.

Institutionalizing the resolution is still having implementation challenges. While the ERM
Process and ERM Team has called for the exclusive use of the RIOT to capture all risks and
issues, some program offices and/or their development contractors already have an established
risk management process and database they do not want to give up. Some do not want to
undergo the effort to re-host their files, and others are concerned about having their risks visible
to outsiders. Future RIOT upgrades will provide limited view access, improved automated
report generation, and further incentives to migrate to the RIOT. Some exceptions have been
made in order to facilitate the acceptance of the process. For example, one program has a
separate instance of the RIOT database because they do not have electronic connectivity to the
RIOT. Every Friday they provide a copy of their database updates for uploading to the RIOT.
NGA’s operations and support contractor and some NGA organizations also have separate
databases but enter their risks in the RIOT if they meet enterprise level risk criteria (discussed
later). These program offices remain responsible for reviewing their risks and entering those that
may affect more than their system (i.e., NGA Enterprise Risk) in the RIOT.

System-level Risk Evaluation Criteria. An early challenge was to provide an equitable risk
consequence evaluation criterion applicable to the range of the programs’ cost, schedule and
performance impacts. A lasting resolution was established by the ERM Team after the enterprise
level risk consequence evaluation criteria was developed (Figure 2). Until then, programs were
performing informal risk management using internally developed evaluation criteria that differed
across programs. The enterprise criteria provides a uniform basis all programs can tailor to
establish criteria that make sense for their system level risks and is less varied across programs.
Using the enterprise level range thresholds also enables system level risk exposures to be ranked
with enterprise risks.

Category Range Cost Schedule Performance

No cost
None 0 No schedule impact. No performance impact.
impact.

< 1 Month slip to System or No impact to capability

Negligible 1 < $100K
Capability-based Effectivity. performance or KPP/CKPP.

1 to 3 Month slip to System

$100K - Effectivity. No impact to KPP/CKPP,
Marginal 2
$500K No impact to Major User degraded capability performance.
Capability

$500K - 3 – 9 Month slip to System or Degraded Enterprise capability

Significant 3
$1M Capability-based Effectivity. performance.

$1M - 9 – 12 Month slip to System or Degraded capability performance

Critical 4
$50M Capability-based Effectivity. with impacts to KPP.

> 12 Month slip to System or Degraded capability performance

Catastrophic 5 > $50M
Capability-based Effectivity. impact to KPP/CKPP

Figure 2. System/Enterprise/Joint Enterprise Risk Consequence Evaluation

Criteria

Enterprise Level Risk Management Challenges

NGA’s various systems were originally developed for integration into an enterprise architecture
called the National System for Geospatial-Intelligence (NSG). Performing NSG enterprise risk
management was a quick and natural evolution from the system level implementation.
Resolutions to system level challenges needed to work while doing enterprise risk management.
Reusing system level resolutions on related enterprise level challenges was a goal. As with the
risk consequence evaluation criteria, an enterprise resolution solved a system level challenge.
Still, along the way unique enterprise challenges were still encountered and resolved.

What makes a risk “Enterprise”? Agreeing on what characterized an NSG enterprise level
risk has been very politically and technically challenging. The resolution was through
continuous negotiations, compromise and ‘word-smithing’. “NSG Enterprise” had to be defined
and boundaries established through agency level discussions. Then attention was put on
technical, cost and performance interfaces within the enterprise and between the enterprise and
external entities. Further, senior leadership needed to be assured that risks and issues on matters
that “kept them awake at night” would be captured. After lengthy discussions, the following set
of enterprise risk identification criteria (NGA 2005) was established to identify risks and issues
that could keep the NSG enterprise from meeting its cost, schedule and performance objectives
and lead to insomnia.
 • Affects NGA Mission capabilities or
Key Performance Parameters
• Affects the capability to comply with
oversight mandates
• Impacts Mission Partner priorities
and/or milestones
• Crosses multiple agencies or contracts or
impacts more than one project, system,
or capability releases
• Affects NGA Master Schedule Level 1
or 2 milestones
• Affects the NGA budget submissions
• Impacts the NGA Strategic Intent, site
consolidation or infrastructure
• Affects the availability, dissemination,
or reliability of NGA information and
data
• Impacts the configuration management
of NGA networks
• Impacts the ability to meet NSG
Enterprise capabilities, requirements or
specifications

Think ‘enterprise’ not just ‘system’. System risk managers were very challenged in
expanding their risk assessments from just considering the system view to the more
comprehensive enterprise view. Considerable education, discussions and monitoring by the
ERM and RMCT has helped resolve the shortcomings. System risk managers were coached to
think ‘enterprise’ not just ‘system’ and assess risks for impacts to one or more other systems in
the enterprise. They learned how their system affected interfaces with other systems within the
enterprise and how to evaluate system changes for enterprise impacts. Otherwise, an enterprise
risk related to the interfaces could be missed and issues occur. Enterprise risks are identified
from a system level risk that expresses or implies incompatibilities between systems when
viewed from the enterprise perspective. A system risk with known or potential enterprise impact
would be flagged in the RIOT and passed to the ROMB for enterprise impact validation. The
additional challenge of determining primary risk ownership was resolved through the ROMB’s
independent analysis of the risk. Typically, the ROMB assigned the program whose action (or
inaction) created the risk as the risk owner and required them to collaborate the risk mitigation
with other impacted systems’ program office and related stakeholders. The ROMB and RMCT
then monitors the mitigation progress.

Finding all the enterprise level risks and issues. The ERM and ROMB had difficulties making
a comprehensive look across system level risks and identifying enterprise risks (not found by a
program office) since not all the systems’ risks were in the RIOT for the reasons noted earlier.
The challenge was met by senior program managers and the ERM meeting with the system risk
managers and emphasizing the risk to their system and the NSG enterprise by not posting all
risks in RIOT. Many program managers understood and decided to take advantage of the value-
added risk analysis service provided by the ERM and ROMB. Others improved their system risk
assessments with more comprehensive enterprise impact analyses but still held their risks in their
database. Additional visibility and the opportunity for more risk stakeholders and impacts to be
identified were gained by making “Risk and Issues” a standing agenda item at all system and
enterprise reviews.

Enterprise-level Risk Evaluation Criteria. The ERM Team was challenged for two years in
developing a single Enterprise risk consequence evaluation criteria to compare enterprise risks
across a range of complexity in development programs (e.g., software tools, workstations, and
large-scale distributed data archives). The resolution (Figure 2) championed by the ERM Team
focuses on common elements between the programs and normalizing across the large range of
cost, schedule and technical performance values. Schedule impacts are rated with respect to
delays in capability or system delivery. Cost impacts are tied to a dollar range needed to mitigate
the risk and potential need for additional funding. Performance impacts were rated on the
severity of impact to system performance (e.g., not achieving a Key Performance Parameter
versus a lower level requirement), availability of alternatives, and need to modify requirements,
schedule or funds to mitigate the performance risk. The specific thresholds within the brackets
evolved through negotiations with the MP to establish criteria applicable in all three risk
families. Adjustments have not been required for over a year.

Enterprise risk prioritization criteria. Given an evaluation criteria, the ERM Team was then
challenged with ranking enterprise risks consistent with senior leadership expectations. The risk
exposure equation enabled ordinal ranking of risks within and across programs by risk exposure
value but did not completely address the risks’ real impact on the enterprise. While senior
leadership was satisfied with the identified risks, they disagreed with where the risks that most
“kept them awake at night” ranked in ‘the big picture’ and on the “Top 10” list. The ranking
challenge was compounded by the seniors having different sensitivities and urgencies towards
the various risks and issues based on their area or responsibility. To translate their qualitative
gut feelings into objective consequences (e.g., impacts to capability and performance delivery)
and probability ranges, the ERM Team met with senior leadership. Interviews captured their
individual and collective views on priority. The result was: 1) a revised risk evaluation
consequence criteria reflecting specific impacts with numeric value ranges tied to the descriptive
consequence levels (e.g., Negligible, Significant, Catastrophic); and 2) provision for unique
prioritization consideration of critical but non-enterprise related activities. Risks are now
prioritized by what the seniors collectively feel have a greater enterprise impact, need more
attention and the ability to apply attention to risks in their particular areas. As the leaders
change, the criteria is readdressed and altered as appropriate.

Categorizing and reporting enterprise risks. How to logically categorize and report on the
risks has been an on-going challenge. Two resolutions were tried with mixed results. Risk
analysis showed that many system and enterprise risks had common themes (e.g., schedule
achievability, test data availability, bandwidth availability) and relationships at the enterprise
level. Umbrella Risks were grouped by common themes, had constituent risks allocated to them
and provided easier identification of the nature and number of risks in a themed area. This
enabled program and technical managers and senior leadership to identify and focus on problems
with broad impacts in the enterprise. While the constituent risks were still managed individually,
their status was aggregated and reported at reviews under the umbrella risk. Using umbrellas or
leaving risk ungrouped with pointers to related risks was debated. First, neither the RIOT nor
MP risk databases were sophisticated enough to trace the relationship, and the manual labor
involved in trying to do that was very time intensive. Second, people did not understand that
umbrellas were not discrete risks and wanted to brief umbrellas while they should be briefing
discrete risks that could be tracked and closed. These inefficiencies led to the concept being
dropped after about 18 months, but the debate continues.

Overcoming challenges with the “4 C’s”. Achieving good Communication, Coordination,

Collaboration and Cooperation became a greater challenge as enterprise risk management
increased. Program office risk boards and owners continually needed to work closer with one
another and the ERM Team to assure thorough risk analyses, complete mitigation plans and
integrity of the ERM process. The RM Senior Process Owner/RMCT Chair has continually
emphasized the 4 C’s between programs as the means to work through problems impeding the
process and reducing system and enterprise risk exposure. One notable example was in aligning
NGA’s RM process with the risk process being instituted by a new program. While the
program’s overall process was the same, the new program used a separate risk tool, evaluation
criteria. matrix (i.e., 5x5 versus 4x4), mitigation terminology (e.g., monitor, manage, resolve)
and enterprise risk identification criteria. The differences made uniform assessment of system
and enterprise level risks and transfer of risks to the RIOT difficult. The program also wanted to
maintain a higher degree of autonomy than other programs yet wanted to work with the ERM
Team, ROMB and RMCT. Months of negotiations achieved a balanced set of compromises on
tools, governance, criteria, and matrices. A major derived benefit was the modification of the
RIOT and program’s risk tool/database to enable the weekly transfer of risks to the RIOT.
Changes to the new program’s evaluation criteria enable reporting of their risks and exposure at
equivalent degrees with other programs and further enable them to be a member of the ROMB
and RMCT contributing to improved enterprise risk management and reduced risk exposure.

Joint Enterprise Level Risk Management

The NSG integrates with a separate enterprise comprised of several systems developed by a
Mission Partner. The highly complex technical integration and interdependencies drove the need
in 2001 for a strong joint enterprise risk management program to assess and mitigate the
associated risks. During the same time period, the MP was also beginning a risk management
program and facing many of the same challenges as NGA. Establishing a Joint Enterprise Risk
Management Program presented unique as well as similar challenges to those faced within NGA
(and the MP) but even more difficult to resolve due to the broader scope of the joint program.

Executing Joint Enterprise Risk Management. The first challenge was to form a joint
enterprise risk management process that would effectively integrate the two risk management
processes. Since NGA’s enterprise risk process was a little more mature, it offered, the resolution
leveraged NGA lessons learned. The joint process foundation is based on NGA’s ERM process
model, expanded for joint application, and applied consistently within NGA and the MP. Joint
risks can also be identified by either partner about the other. When a system risk or issue is
identified during their internal risk analysis, the risk is further assessed for meeting the joint
enterprise identification criteria (discussed below). An examination of the joint enterprise
architecture is also made to identify further impacts across the joint enterprise interface. Two
general examples of joint enterprise risks are: 1) a partner is unable to provide specified data
across an interface to support an operation performed by other, and 2) one partner’s schedule
delay impacts the other’s dependent activity. Since NGA and the MP are performing joint
systems engineering6, the risk identification, analysis and mitigation plan is often identified by
both partners in one of the joint systems engineering functional working groups. Ideally, the
risks are jointly coordinated then simultaneously passed back to the ROMB and the MP’s
internal risk team for a detailed analysis. The completed risk analysis findings are passed to the

6
Joint Systems Engineering is performing systems engineering in an integrated and collaborative manner between
NGA and its mission partner. Several joint working groups exist to focus on particular systems engineering process
areas and engineering management tasks.
JRWG for final validation and activation. While responsibility to mitigate the risks remains with
the joint risk owners, the JRWG will periodically status the risks. From the degree of joint
enterprise risk exposure and potential consequences, the JRWG will status a risk at a senior level
engineering review board. Before the JRWG meeting, the partners’ risk management teams
meet to assess and prepare disposition recommendations on new joint risks, status existing risks,
assess process execution, review and assign actions and set the JRWG agenda.

What makes a risk “Joint Enterprise”? Agreeing on what characterized a joint enterprise
level risk was an expected challenge. Joint technical, programmatic and the sleepless leadership
points of view needed to be considered. NGA’s enterprise risk identification criteria were the
start as it reflected concerns that readily extended to the joint enterprise. The partners’ senior
leaders were interviewed to identify the common concerns, and the following criteria were
developed. After three years of use, they have proven to capture the situations arising by virtue
of the technical and programmatic interfaces between the enterprises that could impact joint
mission success. (NGA 2005)

• End-to-End Systems Engineering • Strategic planning implications to

affecting both organizations
• Integration of new requirements that
span the scope of both organizations
• Integration with Mission Partners
• Long lead technology insertion
shared by both partners
imagery architectures
• Acquisition and operations Joint
Architecture policy
• Identified by insight to performance,
cost, or schedule variants (or trends)
affecting dependencies between both
organizations

Establishing a common joint RM tool/database. One of the greatest challenges in the joint
enterprise process was establishing a joint enterprise risk database readily usable by both
partners. The resolution was based on the lessons learned with the RIOT. The partners needed a
one-stop-shop that provided a holistic picture of the joint enterprise risk situation and mitigated
concerns over duplicative storage and maintenance work and database synchronization. Options
for the tool/database were: 1) a single new tool/database used by both partners, 2) each partner
use their own tool and an integrated database, and 3) each partner use their own tool and manage
their own instantiation of the joint enterprise risk database. In the spirit of cooperation, a fourth
option was selected. NGA and the MP would use their internal enterprise risk management
tool/database, and NGA would manually transfer its joint enterprise risk candidates to the MP’s
Enterprise Risk Database now to have a joint enterprise scope. The next challenge to resolve is
automating the transfer of joint enterprise risks from the RIOT.

Joint senior leadership support. Gaining joint senior leadership support was not a challenge.
The need to perform life-cycle joint systems engineering was already agreed to by the respective
organization and technical leaderships. This agreement was the mandate and basis for forming
the joint enterprise risk management and other processes. Senior systems engineering managers
and leaders know the merits of performing risk management, frequently make their views
known, and continue to be ardent promoters of the process. While they challenge the JRWG to
improve process performance and risk owners’ responsiveness to active risks, senior leadership
support continues to overcome resistance in the joint enterprise domain.
Joint enterprise risk evaluation criteria. Achieving a joint enterprise risk scoring criteria
(Figure 2) was very difficult but resolved through considerable time, negotiations and
willingness to compromise for the greater good of the joint enterprise. NGA and the MP had
their own enterprise risk evaluation criteria that differed in the number of consequence levels,
consequence factors and normalized impact ranges. Considering each partner’s original effort to
renegotiate any change with their respective programs and achieve their enterprise evaluation
criteria, neither was initially willing to change. After more careful examination, NGA concluded
that using the MP’s criteria would not be a significant impact if the MP would make some minor
adjustments in the schedule factor ranges. NGA and the MP performance impacts were already
tied to the joint enterprise key performance parameters and cost evaluation ranges were nearly
the same. The revised criteria continue to satisfy most cases and are providing good results.

Who’s the leader? Determining whether either organization needed to be designated ‘the lead’
and which would be was a political challenge. Common sense and the need for two equal
organizations to collaborate provided the resolution- joint leadership. In order to mitigate a joint
risk, neither partner wanted to relinquish control over their ability to manage and mitigate risks
within their respective enterprise. Joint risks had to be analyzed, mitigated and managed jointly
to achieve a mitigation that was mutually affordable, feasible, and satisfied the joint enterprises’
mission objectives. The objective of performing as a true partnership with sharing the successes
and failures of the joint enterprise remains the driving force behind risk mitigation and
management decisions. The JRWG composition is an equal mix of the partners’ risk
management experts. The co-chairs are each partner’s Senior Engineer who both understand this
philosophy and lead the activities accordingly. Resolving differences is approached objectively
and from the point of view of what makes sense for the joint enterprise; ultimately the end user is
the focus. The joint leadership is continually challenged to maintain equitable joint participation
in risk related meetings and assure equitable outcomes. When there is an imbalance, objectivity
and making the right decisions falls squarely on the co-chairs.

Synchronizing risk management meeting schedules. Getting joint enterprise risks to JRWG
meetings in a timely and coordinated fashion to assure the process functions efficiently was a
logistical challenge. This challenge was overcome by establishing fixed dates for the succession
of risk review starting with the JRWG meeting (e.g., second week of the month). The frequency
and time during the month of NGA and MP risk adjudication meetings (i.e., ROMB and RMCT)
originally were not aligned and inhibited a smooth flow of joint risks to the JRWG meetings.
Based on the co-chairs availability and when other joint forums that may generate risks were
meeting, the JRWG meeting date was set then the predecessor risk review meeting dates backed
off from then. This established a sequence pattern that was replicated through the year. The
meetings’ dates for the rest of the year were then placed on a Joint Enterprise Master Calendar
and provided the other joint forums. This scheduling enables new risks to reach the JRWG
between 7 to 30 days from identification- faster if the situation warrants. In no case, would joint
enterprise risk mitigation actions be held up awaiting a formal JRWG meeting.

Getting and keeping people motivated. Getting technical teams within both organizations
motivated to do joint enterprise level risk management is an ongoing challenge. Standard culture
change enablers identified by (Holzer 2005) are being applied. The greatest enabler is each
agency’s senior engineers and leaders providing unambiguous support to the Joint Enterprise
Risk Management Process. A ‘carrot’ (provision of risk funding) versus ‘stick’ (admonishment
for not addressing risks) approach is also adopted. Decisions to fund requirements are being
made with clear consideration of the risk reduction (identified through the risk assessment)
achievable thru applied funding. As this direct benefit from risk management is seen, more and
more engineers and managers are becoming motivated to perform RM.

The Benefits
To date, the benefits from overcoming the challenges of System, Enterprise and Joint Enterprise
Risk Management are qualitative and anecdotal with quantitative implications. Determining
benefits in terms such as the Return-on-Investment, cost savings, cost avoidance, and time saved
(schedule) with any accuracy, is a challenge still to solve. The overall program and individual
constituent projects have improved because cost, schedule and performance impacts have been
reduced with risk and issues are being identified and managed earlier. Addressing risk and
issues in a structured, repeatable, well documented, and visible manner has greatly increased the
agency’s consciousness of risks and issues. The consideration of potential (or actual) risks and
issues by program managers, systems engineers and their teams is becoming second nature.
Having “Risks and Issues” as a regular item in program reviews and a monthly topic and the
Joint Engineering Review Board has made program teams accountable for risk management,
kept senior leadership more aware of their programs’ risk exposure and actions on their part to
mitigate the risks and issues. This increased visibility is enabling focused attention to be applied
to assessment and mitigation, awareness of the problem to be elevated and basis for and
provision of ‘risk funding’ or non-material solutions to be applied for mitigation. Repeatedly,
early identification, assessment and mitigation of risks has reduced the number of issues dealt
with and lowered program cost, schedule, performance and political impacts. Implementing risk
management particularly in the Joint and Joint Enterprise areas has been a forcing function
towards improved collaboration between formerly disparate technical teams. The joint technical
working groups formed have established new relationships dealing with risk and issue
management and enabling complex problems to be resolved early or avoided completely.
Technical and programmatic changes in many specific instances have been introduced
minimizing or eliminating additional development costs that can escalate on joint programs. The
deliberate focus on identifying joint enterprise risks is now driving risk assessment in areas that
would not have been addressed as early in the life cycle and provided more time for pre-emptive
planning or mitigation. Solutions are being reached more effectively and jointly in relatively less
time than before. The risk assessments are more thoroughly performed given the use of joint
participation. The bottom line is that through improved communication, coordination,
collaboration and cooperation, overcoming the once insurmountable challenges of system,
enterprise and joint enterprise risk management is being achieved.

Summary
Performing formal risk management is an essential element to managing an activity of any size
or complexity. Implementing and institutionalizing efficient and effective RM at any level
requires overcoming numerous challenges. The basic nature of challenges faced may not vary
greatly going from system to enterprise to joint enterprise risk process implementation, but
difficulties in resolving the challenges can increase due to the larger number of people and
different cultures involved. Regardless of the level of the implementation, people’s willingness
to change how they are or are not doing RM will play a direct role in the integrity of the process
outcome. The lessons learned implementing at lower levels need to be leveraged to improve RM
at higher levels. In the end with all implemented families, overcoming the challenges will enable
the characterization, evaluation and prioritization of risks and issues within the subject system,
enterprise and joint enterprise and enable them to be efficiently and effectively managed.

References
Conrow, Edmund, “Risk Management for Systems of Systems©”, CrossTalk, Volume 18, No. 2,
February 2005, Hill AFB, Hill AFB, UT, pp 8-12.
Department of Defense, Risk Management Guide to DoD Acquisition, 5th Ed. Version 2, Defense
Acquisition University, June 2003.
Holzer, Thomas, Learning form Lessons Observed- Mitigating Resistance to SE Process Change,
Proceedings of INCOSE, 2005.
Ibrahim, Linda, et al., The Federal Aviation Administration Integrated Capability Maturity
Model, v1.0. Federal Aviation Administration, November 1997.
Ibrahim, Linda, et al., The Federal Aviation Administration Integrated Capability Maturity
Mode, v2.0. Federal Aviation Administration, September 2001.
NGA, Process Improvement Policy and Processes (PIPP)- Tier 1, Revision B. National
Geospatial Intelligence Agency, June 2004.
NGA, Enterprise Risk Management(ERM)- Tier 2, Version 3.0, National Geospatial Intelligence
Agency, June 7, 2005.

Bibliography
Dr. Thomas H. Holzer is the Deputy Director, Infrastructure Engineering Office in the National
Geospatial-Intelligence Agency Enterprise Operations Directorate and Deputy Program Manager
for Information Technology, New Campus East Program Management Office. He has over 30
years experience in systems engineering, process improvement and leading technical
development programs. He is responsible for assuring the integrity of the systems engineering
performed and developing a proficient systems engineering workforce. He chairs the NGA
Enterprise Process Group and Risk Management Core Team and is co-chair of the Joint Risk
Working Group. He is an instructor in the Department of Engineering Management and Systems
Engineering, George Washington University, Washington, DC. Dr. Holzer has Doctor of
Science and Master of Science degrees in Engineering Management from George Washington
and a Bachelor of Science degree in Mechanical Engineering from The University of Cincinnati.