Engineering Management Journal

Engineering Management Journal

EMJ, 2010 12(34): 567-890 Online ISSN: 1234-5678

Risk Quantification for the Design of Boeing 777 Engine Nacelles

Steven Broussard a, Josh Goldschmid b, William Harkness b, Damon Slaughter c
The Boeing Company St. Louis, MO P.O.Box 516, St. Louis, MO 63166, Steven.R.Broussard@boeing.com b The Boeing Company Everett, WA 9801 27th Ave W, Everett, WA 98208-4316, William.a.harkness@boeing.com, josh.h.goldschmid@boeing.com c United States Army-- Fort Leonard, MO 1805 Cooley Avenue, Fort Leonard Wood, MO, 65473, damon.slaughter@us.army.mil
a

Abstract Identifying and mitigating project risks are crucial steps in managing successful complex projects. This article proposes the application of a Risk Mitigation Quantification methodology quantify, analyze, and mitigate project risks. Popularity and Success Rubric charts were developed organize, grade, and communicate our expectation of the product evaluated. The rubrics essentially help determine how successful the product will meet the criteria. The payoff for management and engineers will be a process that implements historical risk mitigation steps with known success. The risk mitigation quantization methodology proposed enables effective planning to mitigate any consequences from occurring and thus providing a safe and reliable product that complies to high-level government regulations as well as minimizing unnecessary work, reducing overall program cost, and keeping programs on schedule. Index Terms Risk Mitigation Quantification, Project Management, Risk Severity, Engine Nacelles, Design, Risk Rubric.

I. INTRODUCTION
Identifying and mitigating project risks are crucial steps in managing successful complex projects. This article proposes the application of a Risk Mitigation Quantification and Mitigation

methodology quantify, analyze, and mitigate project risks. Imagine the risk of developing a new product, taking a leap of faith into the design world and hoping the risk pays off in the end. How would that risk be handled? In the past, one would find the smartest person in the room and ask them what they think the risks of the new product would be. Unfortunately, not one will have a perfect answer as complex design escapes human abilities. Collaborative efforts across engineering disciplines such as propulsion, structures, materials, project management, aerodynamics, and safety are necessary actions to identify and quantify risk and risk mitigation items. There are strides being made in risk and the end-product will help remove some of the ambiguity from the risk world. Grantham Lough, Stone, and Tumer have developed a process called The Risk in Early Design (RED)1 which uses historical data to identify risk as early in the design phase a possible and assign a likelihood and consequence to the risk. But, how would one mitigate the risk? Risk Mitigation Quantification is feeding off a similar idea. At this point a process will not be developed, however the first step is gathering the data needed to create a process. Risk Mitigation techniques are used in every program, so the idea is to gather as many risk mitigation techniques as possible, document when the technique was used, how it was used, how often it

Engineering Management Journal

was used. Who assigned the technique? Who is responsible for the technique? Most important though, was the technique successful for reducing risk, and by how much? Management will benefit from this method the most. With detailed risk data and mitigation items will provide a manager the capacity to make intelligent decisions. As a secondary outcome of using this strategy management funding will potentially go further. Earned Value Management has two key objectives that need to be addressed. Encourage contractors to use effective internal cost and schedule management control system Provide the customer timely data produced by those systems for determining contract status2. Earned value is a metric that is used to report to the customer on progress. Technical performance measures (TPM) and the risk are always key program tracking points that must be reported to the customer. By using this method, management can quantify what was done in the past. Estimations for tracking the risk on the program and meeting mitigation goals will have more merit and historical data to back up the approach. The first step is to gather the historical data. The intent of this paper is to take the risk data and provide a means of showing how much each mitigation step has reduced the overall risks, using the example of the design of a Boeing 777 engine nacelle. The function of the nacelle is to provide housing for the airplane engine, fuel lines, and electronic equipment. In simple terms, the engine nacelle is analogous to the hood of a car. Cylindrically shaped, the nacelle forms the outer, aerodynamically smooth covering for the engine (see figure 1). The design of the Boeing 777 nacelles is extremely complex and takes approximately eight (8) months to complete. The nacelle design complexity often surpasses human ability in many aspects- the calculation, modeling, and simulation of considerable amount of design and analytical data provides many opportunities for considerable consequences. Thus, an understanding of identifying and quantifying risk and risk mitigation items is a prerequisite for making intelligent decisions.

Figure 1: Boeing 777 Engine Nacelle

II. RISK MITIGATION QUANTIFICATION

Risk quantification is a process of the independent quantification of probability and severity associated with a particular product, service or scenario. The product of the probability and severity provides a risk value needed to make decisions as to what should be done about them. Risk mitigation quantification, however, involves the quantification of reduction in the probability and the reduction in severity. Before we could quantify risk mitigation, we identified the risk elements associated with the design of the Boeing 777 engine nacelles, determined the likelihood and severity values for each risk, and determined mitigation strategies. A. Identification and quantification of risk elements of the 777 nacelle design Several tools have been used to identify ten (10) risky scenarios shown in Figure 2 (larger pictures in Appendix A): Failure Modes and Effects Analysis (FMEA) Physical inspection of components and systems Logic models (event trees and fault trees)

Engineering and statistical tools and analyses have been used to estimate the likelihood and consequences in the event that the risk is materialized. The tools used for quantifying probabilities/likelihood were: Failure analysis Failure rate data and probability analysis

Engineering Management Journal

Subject Matter Expert (SME) opinion and subjective probability analysis Bayesian probability analysis Human error analysis

For each risk, the relative likelihood and severity of consequence lead to the calculation of the risk factor product below:
R=PxS

(eq. 1)

Where R is the risk score (chance of an identified consequence occurring), S is severity of harm (consequence), P is the likelihood of the occurrence of that harm. According to Figure 2, our top four biggest concerns in the design of the nacelles are: 1. Excess weight 2. Excess production cost 3. Non-compliance to regulations 4. Safety requirements not met Consequences are large if these risk items occur. Fortunately, there are mitigation strategies (Figures 3, 4, and 6) to reduce the probability and severity of the consequences. The mitigation steps are allocated to the risk items to support the goal of a safe nacelle design.

assigned to each mitigation step; the risk waterfall chart for a specific risk mitigation item to reduce the likelihood of the consequences. Activities are introduced to progressively reduce risk by increasing knowledge and confidence in the re-design of the 777 nacelles. Assuming each activity is successful, risk is shown to decrease from high to low as each activity is completed. Unfortunately, these quantitative estimates of risk reduction are subjective basing on SMEss estimates; however with the props of the cross-functional teams engineering and manufacturing immeasurable experience and sophisticated skill, the waterfall chart communicates a well thought out plan with each risk assigned to a responsible engineer or manager. If an activity fails, the chart would be modified to show increasing risk and additional activities.
Risk Title: 777 Nacelle Design Change
0 1 2 3 4 H i g h 5 6 7

Percent Likelihood that the 777 Nacelles Do Not Fail Over the Life-Cycle

25

9 M o d e r a t e 10 11 12 13

50

15 16 17 75 18 L o w

19 20 21

100 Q1 Q2 Q3

Figure 3: Nacelle Risk Management Plan

Figure 2: 10 Risk Elements and Risk Score B. Waterfall chart and risk mitigation of the 777 nacelle design

For each essential safety planning elements, the SMEs were asked to use their experience and judgement to identify risk mitigation strategies and to rate what they believe the average reduction in probability and severity of a disaster. Shown in Figure 3 and 4 is a risk management plan (the full management plan can be seen in the appendix) with job roles

Figure 4: Risk Identifier and job role assignment for the waterfall chart (Figure 3) C. Risk Plot of the 777 nacelle design This plot below shows the 10 identified risk values plotted with potential severity of

Engineering Management Journal

consequences on the x-axis and relative likelihood of consequences on the y-axis. severity of these consequences. This particular role goes to regulatory administrators delegated by the Federal Aviation Administration and the Boeing Certification Engineers (Figure 4). Other important risk mitigation strategies include performing careful materials selection, having supplier management controls in place, and performing conformity and safety tests, performed by Design and Certification Engineers and Regulatory Administrators.

Figure 5 : Nacelle Risk Plot D. Assessing the 777 nacelle design By pulling out the data from the nacelle design one can begin the process of quantifying the risk mitigation effort. Figure 6 shows some of the key points from the risk mitigation effort. The figure can also be viewed in the appendix.
Mitigation Effort.
Project proposal for re-design Integrated, multi-discipline, Integrated Product/Process (IPP) team formed Develop detailed project management aspects Develop detailed work statement Initial studies and cross-functional research including economics Develop plan for concept design Early involvement with leadership and regulatory authority Identify applicable regulations and requirements Define plan to meet regulatory and safety requirements Validate certification process Validate design process Design the nacelle per the requirements Perform careful materials selection Validate build process Develop prototype for testing Supplier management controls in place Develop conformity and safety test plans Perform conformity and safety tests (flammability, aerodynamics, and appreciable effects of design) Identify safety and certification issuesperform failure analyses Perform technical metrics and audit Conduct production readiness review

What does the mitigation effort describe? Propose design Form team Develop plan Develop strategy Conduct research Develop plan Assess progress Identify limits Define plan Validate process Validate process Design product Select material Validate process Develop prototype Implement controls Develop Plan Conduct test Identify concerns Conduct checks Conduct review

Does it reduce likelihood? If so, by how much? Yes, 2 Yes, 2 Yes, 2 Yes, 4 Yes, 4 Yes, 2 Yes, 10 Yes, 6 Yes, 6 Yes, 4 Yes, 6 Yes, 2 Yes, 4 Yes, 4 Yes, 2 Yes, 6 Yes, 8 Yes, 4 Yes, 4 Yes, 4 Yes, 4

Does it reduce consequence? If so, by how much? No No No No No No Yes, 10 Yes, 6 Yes, 6 No No No Yes, 4 No Yes, 2 Yes, 6 No Yes, 4 No No No

Figure 7: Risk Mitigation Allocation to Reduce Consequences Figure 7 shows the allocation of risk mitigation items from Figure 4 to the risk items to reduce the severity of the consequences. For instance, developing a nacelle prototype will ensure design and safety requirements are met as well as supporting the understanding of production to reduce production costs and time.

Figure 6: Nacelle Risk Mitigation Data Figure 6 details how each risk mitigation step affected a particular part of the nacelle design. The figure also shows the impact to the overall risk on the redesign. These parts will be important to determine which mitigation steps are critical to the success of the risk mitigation effort. For example one of the mitigation efforts listed is forming an integrated product process team (IPPT). Just by forming the team there is a positive impact on reducing the likelihood a failure event might occur in the nacelle design, which is by two (2) percent. The consequence of a failure is not affected by this step. This IPPT is made up of a core group of engineers and managers that can make critical and intelligent decisions on the nacelle design. Early involvement with leadership and regulatory authorities and identifying safety and regulatory requirements has the greatest impact in reducing the likelihood of the failure consequences as well as reducing the

III. POPULARITY AND SUCCESS RUBRIC FOR RISK QUANTIFICATION

A. Popularity Rubric The popularity Rubric chart in Figure 8. is modeled after Dr. Granthams work with slight modifications to meet the generic needs of risk quantification. See Appendix for clearer picture of the Rubric Chart. The Rubric organizes, grades, and communicates our expectation of the product evaluated.

Engineering Management Journal

Figure 8. Popularity Rubric Figure 9. Success Rubric With popularity, we used the measurement of good, moderate, and poor which will be converted numerically into a 1-3 scale (3 being best, and 1 worst). We can evaluate the success of each criteria to see whether the risk mitigation met the requirement outlined by the engineers. It has the advantage of being a top level review of a product before we move into lower level review, which is covered by the next type of rubric, Success Rubric. B. Success Rubric Success Rubric Chart (See Figure 9) is nearly similar to the Popularity Rubric Chart with the only exception; it actually evaluates the success of the product that has gone through the Popularity Rubric Evaluation initially. Success Rubric helps determines how successful the product itself was able to meet the criteria outlined at a much lower level. It takes into consideration Dr. Granthams RED methods as a crucial aspect of the design phase for risk mitigation. See Appendix for a clearer picture.

IV. IMPLICATIONS TO THE ENGINEERING MANAGER

When we talk about risk quantification, Bernstein (1998)3 stated that without numbers, risk is wholly a matter of gut. To minimize arbitrary decisions with absolute subjectivity, we need to collectively form an integrated team with sufficient level of skill and experience to identify and quantify all possible associated risks with a product design. Risks have a major impact on business decisions that often reflect on the business case net present value (NPV) or internal rate of return (IRR). We make business decisions on the basis of limited data and likelihood of consequences occurring and thus, in the face of probabilities, risk management and quantification plays a crucial role in decision-making. A corporate objective is to maximize value for the shareholders and itself. The role of engineering manager is to provide value to the company, shareholders, and the customer in the form of a functional product. The goal of the engineering manager is to provide a product that is reliable and safe. The risk mitigation quantization methodology proposed enables effective planning to mitigate any consequences from occurring and thus providing a safe and reliable product that complies to high-level government regulations.

Engineering Management Journal IV. CONCLUSION

This method of quantifying risk mitigation can redefine how risk is managed. There are important strategies that need to be taken away from this process: Collect the data from risk mitigation plan Use the Success Rubric to ensure all necessary data is was recorded in the risk mitigation efforts. Use the data for future risk mitigation efforts Many people will benefit from the risk mitigation process in this paper. At the engineering level, historical data will finally be used instead of subjectively guessing at an answer. From a managers perspective effective planning for risk mitigation will save schedule and cost on the overall program while having the confidence corners are not being cut when providing a reliable product to the customer. When a process is created from these initial steps it will save time and money for all of industry who is looking for innovative ways for risk mitigation.
About the Authors

Josh Goldschmid received his M.S. in Engineering Management from Drexel University and a B.S. In Biological Systems Engineering from UC Davis. He is a very cool guy with 20 years of experience in skiing and welcomes everyone for free lessons. Steve Broussard received his B.S. in Mechanical Engineering from The Pennsylvania State University and an A.S. in Engineering Science from SUNY Morrisville. In 8 years I have worked for NAVAIR, Lockheed Martin and currently work for The Boeing Co. I hate moving and I personally hate to ski and am not a cool guy. William Harkness received his M.S. in Engineering Management from Drexel University and a B.S. In Mechanical Engineering from California State University, Chico. He is a very laid back guy with 0 years of experience in skiing and doesnt understand why anybody would want to go up, down, up, down, up, and down a frozen snow covered hill only to break their leg or collar bone in the process of doing so. Damon Slaughter received his B.S. in Engineering Management from Park University. He has never tried snow skiing but would like to at some point as long as he does not break his leg or collar bone in the process.

ACKNOWLEDGMENT
Dr. Granthams Rubric Chart.

REFERENCES
[1] Gratham Lough, Stone, Tumer (2008) The Risk in Early Design (RED) Method [2] NAVSO P-3686 (1998) Top Eleven Ways to Manage Technical Risk. Office of the Assistant Secretary of the Navy (RD&A) Acquisition and Business Management. [3] Bernstein, P. (1998) Against The Gods The Remarkable Storey of Risk. Published by John Wiley & Sons, Inc. New York, USA [4] Project Management Institute, A Guide to the Project Management Body of Knowledge (PMBOK Guide), Project Management Institute (2000). [5] H. Kumamoto, E. Henley (1996) Probabilistic Risk Assessment and Management for Engineers and Scientists. Published by IEEE Press.

Engineering Management Journal

APPENDIX A
Figure 2: 10 Risk elements and Risk Score

Engineering Management Journal

Figure 3 : Nacelle Risk Management Plan

Engineering Management Journal Figure 4: Risk identifier and assigned job roles for the waterfall chart (Figure 3)

Engineering Management Journal

Figure 5 : Nacelle Risk Plot

Figure 6 : Nacelle Risk Mitigation Data

Mitigation Effort.
Project proposal for re-design Integrated, multi-discipline, Integrated Product/Process (IPP) team formed Develop detailed project management aspects Develop detailed work statement Initial studies and cross-functional research including economics Develop plan for concept design Early involvement with leadership and regulatory authority Identify applicable regulations and requirements Define plan to meet regulatory and safety requirements Validate certification process Validate design process Design the nacelle per the requirements Perform careful materials selection Validate build process Develop prototype for testing Supplier management controls in place Develop conformity and safety test plans Perform conformity and safety tests (flammability, aerodynamics, and appreciable effects of design) Identify safety and certification issuesperform failure analyses Perform technical metrics and audit Conduct production readiness review

What does the mitigation effort describe? Propose design Form team Develop plan Develop strategy Conduct research Develop plan Assess progress Identify limits Define plan Validate process Validate process Design product Select material Validate process Develop prototype Implement controls Develop Plan Conduct test Identify concerns Conduct checks Conduct review

Does it reduce likelihood? If so, by how much? Yes, 2 Yes, 2 Yes, 2 Yes, 4 Yes, 4 Yes, 2 Yes, 10 Yes, 6 Yes, 6 Yes, 4 Yes, 6 Yes, 2 Yes, 4 Yes, 4 Yes, 2 Yes, 6 Yes, 8 Yes, 4 Yes, 4 Yes, 4 Yes, 4

Does it reduce consequence? If so, by how much? No No No No No No Yes, 10 Yes, 6 Yes, 6 No No No Yes, 4 No Yes, 2 Yes, 6 No Yes, 4 No No No

Engineering Management Journal

Figure 7 : Risk Mitigation Allocation to Reduce Consequences

Engineering Management Journal

Figure 8. Popularity Rubric:

[modeled after Dr. Granthams Rubric Charts]

Engineering Management Journal

Figure 9: SUCCESS CHART: [modeled after Dr. Granthams Rubric Charts]