Using Modified FMEA Logic to Drive Risk Based Testing -- Talk notes Presented by Charlie Taylor CQE (ASQ)

charlie.taylor@eds.com (FMEA outline Originally Based on http://www.fmeca.com/ffmethod/methodol.htm now says "under construction".) Manufacturing -- Penlight Example Problem: Design> Manufacture> Exposure> Excess Moisture> Corrosion> Poor Contact> Insufficient Current> Bulb Dim> Dissatisfied Customer Analysis: Casing Function Prevent exposure to Moisture Failure Mode Allows Excess Moisture Cause Abraded o-ring (What is the root cause?) Effect Corrosion, etc. Bulb Function Provide bright light Failure Mode Bulb Dim Cause Insufficient current (Root Cause is Corrosion) Effect Dissatisfied Customer Traditional FMEA Steps FMEA Preparation Assemble cross functional team Identify Object for analysis Process, Product, Module, or etc. Identify Functions Identify Failure Modes for each Function Consider 5 failure Categories: Complete, Partial, Intermittent, over time, over-perform E.g. Bulb is: No light, Dim, Blinking, Gradual Dim, Too Bright FMEA Analysis for each failure mode S - Severity - Highest of all the effects Use customer contact, experience, and industry standards Brainstorm possible consequences then Arrange into Effects Clarify original Failure modes Add additional Failure Modes Loop until team is satisfied, but adjust later as needed. Assign Severity to each Effect ( 1 10 Low to High ) Assign the highest Severity to the Failure Mode O - Occurrence - Highest of all the causes Brainstorm possible Causes of Failure Modes Identify Root Causes ( 5 Whys) Assign an Occurrence to each Cause ( 1 10 Low to High ) Assign the highest Occurrence to the Failure Mode D - Detection - Highest risk for each test/inspection Identify Current Controls Prevent or Reduce rate of Cause or Failure Mode Detect a Cause Detect a Failure Mode before it goes to a Customer Assign a Detection ( 1 10 High to Low ) Assign Lowest Detection (highest number) to Failure Mode

FMEA Form example: http://www.colorado.edu/ContinuingEducation/sixsigma/downloads/FMEAForm.xls FMEA Calculations Populate FMEA form Calculate Criticality = S * O Calculate RPN (Risk Priority Number) = S * O * D Use FMEA to Reduce Risk Identify RPNs that exceed a threshold Work on highest RPN first Change Controls, Process, or Design to reduce S, O, or D Recalculate RPN and repeat until it is under the threshold. Modified FMEA Steps Or How to Use modified FMEA logic to Drive Risk Based Testing in limited time situations Test Architecture - Build a set of test case scripts to define single functions - manual or automated - Should not preclude exploratory testing but: - Exploratory testing time is not included. - Scripting gives a basic structure to "hang" exploration on. - Build Suites of test cases to model functionality, use cases, and etc. - Associated test case scripts to the test cases i.e. a script can be used in many suites, or multiple times in one suite, each instance of running the script is a unique test case. The FMEA engine - For each test case script, estimate the time to execute the test. - Assign S, O, and D for each test case script - For this method, re-define D as: "the likelihood that the defect will be found if this test is not run". - You can do this step by step in each script and assign the highest step RPN to the script. - For each test suite, calculate the total time to execute the test cases to be executed. - You can choose entire suites to run/exclude e.g. If a suite RPN is less than the threshold, eliminate it. - Or, you can eliminate low RPN steps from the suites - Be careful not to disturb the logic of the suite, so if you must run certain steps, just include them. Use the FMEA Tool - For the test cycle, identify the acceptable RPN threshold. - Sum the execution time for all suites with RPN greater than the acceptable RPN. - If the total test time > the available time, adjust the threshold - Inform the project team that there is not enough time to reach the original RPN threshold - Inform the project team what RPN threshold can be reached with the available time Software FMEA test management spreadsheet example (Charlie Taylor): FMEA Test Tracking.xls Resources Common Software failure modes: http://www.softrel.com/fmea.htm FMEA Maturity Stages: http://www.qualitydigest.com/june99/html/body_fmea.html Software FMEA Literature (Mike Signor): http://users.compaqnet.be/fmea/Links/Literature_references.htm ( Mike Signor -- http://www.linkedin.com/ppl/webprofile?action=vmi&trk=ppro_viewmore&id=1436037 ) Sample Traditional FMEA Form: http://www.colorado.edu/ContinuingEducation/sixsigma/downloads/FMEAForm.xls