KC-135 Simulator Systems Engineering Case Study

KC-135 SIMULATOR
SYSTEMS ENGINEERING
CASE STUDY

EXECUTIVE SUMMARY

By: Don Chislaghi, Richard Dyer,

& Jay Free, MacAulay-Brown, Inc.

Air Force Center for Systems Engineering

2900 Hobson Way, Wright-Patterson AFB, OH 45433-7765

Executive Summary
ID 8845
 KC-135 Simulator Systems Engineering Case Study

FOREWORD
At the direction of the Secretary of the Air Force, Dr. James G. Roche, the Air Force Institute of
Technology (AFIT) in 2002 established an Air Force Center for Systems Engineering (AFCSE) at its
Wright-Patterson AFB, Ohio, campus. With academic oversight by a subcommittee on Systems
Engineering (SE), chaired by Air Force Chief Scientist Dr. Alex Levis, the AFCSE was tasked to develop
case studies of SE implementation during concept definition, acquisition, and sustainment. The committee
drafted an initial case-study outline and learning objectives, and suggested the use of the Friedman-Sage
Framework to guide overall analysis.
The Department of Defense (DoD) is exponentially increasing the acquisition of joint complex systems
and Systems of Systems (SoS) that deliver needed capabilities demanded by our warfighter. SE is the
technical and technical management process that focuses explicitly on delivering and sustaining robust,
high-quality, affordable solutions. The Air Force leadership has collectively stated the need to mature a
sound SE process throughout the Air Force. Gaining an understanding of the past and distilling learning
principles that are then shared with others through our formal education and practitioner support, are
critical to achieving continuous improvement.
The AFCSE has published eight case studies thus far, including the C-5A, F-111, Hubble Telescope,
Theater Battle Management Core System, B-2, Joint Air-to-Surface Standoff Missile, A-10, Global
Positioning System and Peacekeeper ICBM. All case studies are available on the AFCSE web site
[http://www.afit.edu/cse]. These case studies support academic instruction on SE within military service
academies, civilian and military graduate schools, industry continuing education programs, and those
practicing SE in the field. Each of the case studies comprises elements of success as well as examples of
SE decisions that, in hindsight, were not optimal. Both types of examples are useful for learning.
Along with discovering historical facts, we have conducted key interviews with program managers and
chief engineers, both within the government and those working for the various prime and subcontractors.
From this information, we have concluded that the discipline needed to implement SE and the political
and acquisition environment surrounding programs continue to challenge our ability to provide balanced
technical solutions. We look forward to your comments on this KC-135 Flight Simulator case study and
our other AFCSE published case studies.

GEORGE E. MOONEY, SES

Director, AF Center for Systems Engineering
Air Force Institute of Technology
http://www.afit.edu/cse

Approved for Public Release; Distribution Unlimited

The views expressed in this Case Study are those of the author(s) and do not reflect the
official policy or position of the United States Air Force, the Department of Defense, or the
United Stated Government.

Executive Summary Page ii

ID 8845
 KC-135 Simulator Systems Engineering Case Study

TABLE OF CONTENTS
EXECUTIVE SUMMARY .......................................................................................................... 1
1.1 KC-135 SIMULATOR CASE STUDY LEARNING PRINCIPLES .............................................. 3
1.2 LEARNING PRINCIPLE-1: SYSTEMS ENGINEERING MUST TRANSLATE PROGRAM GOALS
AND OBJECTIVES INTO CLEARLY DEFINED AND VERIFIABLE SYSTEM REQUIREMENTS, FOCUSING
ON THE ENTIRE LIFE CYCLE. ......................................................................................................... 4
1.3 LEARNING PRINCIPLE-2 THE SYSTEMS ENGINEERING PROCESS MUST BE STRUCTURED TO
PROPERLY MITIGATE CHALLENGES GENERATED BY THIRD-PARTY MODIFICATION
CONTRACTORS. ............................................................................................................................ 5
1.4 LEARNING PRINCIPLE-3: SYSTEMS ENGINEERS MUST BE RESPONSIBLE FOR ENSURING
THAT ALL STAKEHOLDERS ARE INVOLVED DURING KEY DECISION TECHNICAL PLANNING AND
EXECUTION PROCESS REVIEWS. .................................................................................................... 6
1.5 LEARNING PRINCIPLE-4: INTEGRATED LOGISTICS/MAINTAINABILITY SUPPORT
STRUCTURE AVOIDS PARTS OBSOLESCENCE AND DIMINISHING MANUFACTURER SUPPLY ISSUES. . 7
1.6 LEARNING PRINCIPLE-5: SIMULATOR MODELING DATA/MODIFICATION REQUIRES
VERIFICATION AND VALIDATION TO ENSURE AIRCRAFT-LIKE FLYING QUALITIES. ........................ 8

LIST OF FIGURES
Figure 1. KC-135 OFT WITH SIX DEGREE OF FREEDOM MOTION BASE. ................................................... 1
Figure 2. FRIEDMAN-SAGE FRAMEWORK KC-135 SIMULATOR CASE STUDY LEARNING PRINCIPLES. ..... 4

Executive Summary Page iii

ID 8845
 KC-135 Simulator Systems Engineering Case Study

EXECUTIVE SUMMARY
From its beginning in the early 1960s as mobile KC-135 simulators housed in railroad cars, the
KC-135 Aircrew Training System (ATS) has evolved into an effective ground based aircrew
training system while simultaneously achieving a high level of customer acceptance and
approval. In order to realize the goal of increased usage of the ATS for crew training, several key
elements needed to be in place: a cultural change was made within the Air Force tanker
community in that training is achievable by effective use of ground based trainers. In addition,
money was allocated to effectively operate, maintain, and upgrade ATS capabilities.
Corresponding improvements to the hardware and software were realized in a cost effective
manner and good relationships between the Government and Contractors were established and
maintained. And finally, common goals associated with the efficient development and effective
operation of the ATS, were established. The AF KC-135 Aircrew Training System (ATS) is
comprised of 19 KC-135 model Operational Flight Trainers (OFT), 27 Global Air Traffic
Management Interactive Hand Controller Part Task Trainers, a number of Cockpit
Familiarization Trainers, 258 Computer Based Training (CBT) Workstations, one Cargo Loading
Trainer, 40 Air Force Mission Support System Computers, and 102 CBT laptop computers. This
equipment is distributed across 13 worldwide bases with the main schoolhouse located at Altus
AFB. The KC-135 ATS provides initial qualification, re-qualification, upgrade training,
difference training, conversion training, the central flight instructor course, and selected
continuation training through academics and simulator applications.
The KC-135 Simulator Case Study reflects the KC-135 ATS teams SE processes as they
matured during a phase of major modifications/upgrades made to the KC-135 ATS during the
period 1992 2007. This timeframe illustrates a wide range of issues encountered/associated
with the delivery of realistic and concurrent modifications required to maintain a modern air
vehicle training system. The integration of these upgrades is equally complex as those associated
with major aircraft weapon systems. Not only must they replicate the aircrafts physical
configuration and performance they must also replicate the environment in which the aircraft
operates. The current KC-135 Operational Flight Trainer, see Figure 1, is a fully replicated and
functional cockpit trainer with a visual system capable of meeting FAA level C certification and
equipped with a full six-Degree of Freedom motion system.

Figure 1. KC-135 OFT with Six Degree of Freedom Motion Base.

Executive Summary Page 1

ID 8845
 KC-135 Simulator Systems Engineering Case Study

During this period Air Mobility Command (AMC) established two key program goals that
formed the foundation of the KC-135 ATS upgrade strategy. The first addressed the need for
concurrency which is to ensure the OFT is upgraded and ready for training prior to the aircraft
with its modifications being fielded. The second addressed the Commands goal to upgrade
operational flight simulator training effectiveness.
The KC-135 ATS program took advantage of advancements in computing and visual technology
and the use of stimulated actual aircraft equipment to achieve significant improvements in
providing cost-effective training to AMC crews. Stimulated equipment uses the actual aircraft
line replacement units with appropriate software changes necessary to have the unit function in
the simulator as it would in the aircraft. These improvements have allowed the OFT to achieve
FAA AC-120-40B Level C simulator certification and demonstrate 99 percent availability
ensuring KC-135 air crew training is delivered when and where needed.
Major modifications to the OFT included the addition of a six-degree motion system, a new
visual system, aerodynamic model enhancements, incorporation of the KC-135 Pacer Compass
Radar and Global Positioning System (CRAG) avionics modernization upgrade, and
incorporation of Communications, Navigation, and Surveillance/Air Traffic Management
(CNS/ATM) Block 40 capabilities. Implementation of these improvements to the 19 OFTs to
maintain commonality and aircraft concurrency have resulted in a dynamic engineering
coordination effort between the Air Logistics Center (ALC) aircraft office, Air Education and
Training Command (AETC), AMC, the acquisition community at Hill AFB, support service
contractor FlightSafety, and third-party subsystem upgrade contractors.
What drove the requirement for a structured SE process was the need for the contractor to ensure
that a timely, consistent level of training is provided to the Air Force. Specifically, ensuring all
courseware, documentation, hardware, and software (development and integration) is sufficient
to provide training value to the user. Some of the reasons given for the teams success include
the fact that the prime contractor understands the users training needs because they are the
trainers. The products they develop are used by themselves. Therefore, they have good insight
into what the products must do. Furthermore, the ATS team consists of a small, highly skilled
group of government and contractor personnel who have established, over a period of 17 years, a
working relationship based on mutual trust and respect that facilitates their ability to successfully
achieve a high level of system performance and customer acceptance.
One of the key processes employed by the team is a rigorous design review cycle with
participation by all training system stakeholders including participating in KC-135 weapon
system design reviews to ensure the training community is working in parallel with the aircraft
community. One of the characteristics of the team is their ability to be flexible and react quickly
to customer needs. Ultimately the teams biggest triumph can be illustrated by the fact that the
simulators are typically completed and ready for training before the fielding of the aircraft with
the applicable modifications installed. Today, 3,900 aircrew members receive effective training
on the KC-135 ATS every year at bases in the United States, United Kingdom, and Japan.
One of the challenges facing the Government in the future is to foster the advantages associated
with long term support contracts (i.e., workforce continuity, knowledgeable support personnel,
program stability, sense of ownership, incentives for process improvements, incentive for long-
range planning) while meeting the Governments requirements for increased competition and
shorter term contracts. The success of the current systems engineering process is due partly to the

Executive Summary Page 2

ID 8845
 KC-135 Simulator Systems Engineering Case Study

strength of the highly knowledgeable government and contractor (prime and subs) team that has
been assembled.
This case study examines the continuing maturation of the applied systems engineering processes
and the interactions of government organizations and contractors necessary to deliver and
maintain first rate training equipment. Numerous interviews were conducted with individuals
from the using command, acquisition community and support contractor to generate this story.
1.1 KC-135 Simulator Case Study Learning Principles
The Friedman-Sage matrix was used to examine the systems engineering and systems
management concepts encountered in the development of the KC-135 case study. This
examination resulted in the identification of the learning principles depicted in Figure 2. As
noted in the case study, the systems engineering processes and management have a significant
place in the successful execution of an evolutionary development and support program like the
KC-135 simulator.
Concept Domain Responsibility Domain
1. Contractor 2. Shared 3. Government
Responsibility Responsibility Responsibility
A. Requirements Definition and LP1 - Systems
Management Engineering must translate
program goals and
objectives into clearly
defined and verifiable
system requirements,
focusing on the entire life
cycle.

B. Systems Architecting and LP4- Integrated LP3 - Systems engineers

Conceptual Design logistics/maintainability must be responsible for
support structure ensuring that all
avoids parts stakeholders are involved
obsolescence and during key decision
diminishing technical planning and
manufacturer supply execution process reviews.
issues.
C. System and Subsystem Detailed LP3 - Systems LP1 - Systems engineering
Design and Implementation engineers must be must translate program
responsible for goals and objectives into
ensuring that all clearly defined and
stakeholders are verifiable system
involved during key requirements, focusing on
decision technical the entire life cycle.
planning and
execution process
reviews.
D. Systems and Interface Integration
E. Validation and Verification LP5 Simulator modeling
data/modification requires
verification/validation to
ensure aircraft like flying
qualities.

F. Deployment and Post Deployment

Executive Summary Page 3

ID 8845
 KC-135 Simulator Systems Engineering Case Study

Concept Domain Responsibility Domain

G. Life Cycle Support LP4- Integrated
logistics/maintainability
support structure
avoids parts
obsolescence and
diminishing
manufacturer supply
issues.
H. Risk Assessment and Management
I. System and Program Management LP2 - Systems
engineering process must
be structured to properly
mitigate challenges
generated by third party
Developers.

Figure 2. Friedman-Sage Framework KC-135 Simulator Case Study Learning Principles.

1.2 Learning Principle-1: Systems Engineering must translate program goals
and objectives into clearly defined and verifiable system requirements,
focusing on the entire life cycle.
Systems engineering needs to be cognizant at all times of the over-arching program goals (i.e.,
concurrency and Level C certification) to ensure that the subsequent modification requirements
lead to and achieve the top-level program goals.
A key element of this process is the upfront reviews that focus on ensuring that the KC-135 ATS
team has established a level of mutual expectations and understanding of the system
requirements and that the proposed preliminary designs will satisfy those requirements. From
these reviews, the contractor can then draw a box around what specific capabilities are to be
delivered. The entire ATS is reviewed at quarterly System Review Boards (SRBs) while student
critiques are reviewed during monthly Training System Configuration Working Group
(TSCWG) meetings at Altus. The entire KC-135 training program is reviewed annually in group
forum via the Realistic Training Review Board.
Requirements analysis is a critical tenet of systems engineering and must remain focused not
only on the development phase but also on the systems entire life cycle. As a part of the Pacer
CRAG Block 40 program, a FlightSafety systems engineer recognized a potential issue
associated with the future operation and maintenance of Block 40 configured OFTs at multiple
KC-135 bases and as a result initiated a program requirement for system reconfiguration. Since
modifying aircraft from a Block 30 configuration to a Block 40 configuration involved major
changes to the aircraft and ATS, systems engineerings assessment was that a scheduling
nightmare might occur given the uncertainty of which bases and which batch of aircraft would
get the Block 40 modification. Their derived requirement for system convertibility or
reconfiguration, which was flowed down to their subcontractors, was to be able to convert a
Block 30 simulator to a Block 40 configuration in eight hours by two people in order to maintain
the required training schedule and student throughput. The actual conversion takes less than four
hours today and is a main contributor to the OFTs availability for training.
The systems engineer must be cognizant of all elements that make up the system and ensure the
proper identification and allocation of requirements flow down to those elements. As an
example, for a ground-based simulator one element is the facility the device is housed in. One

Executive Summary Page 4

ID 8845
 KC-135 Simulator Systems Engineering Case Study

such consequence of bringing motion to the KC-135 OFTs related to the excursion envelope
the motion system required for the trainer within the facility. In some cases, the facility was a
few inches too small. In other cases, a new facility was required. Early interaction with local
units and the detailed engineering studies of the new motion system requirements allowed local
MILCON budgets to be modified and/or initiated so that the facilities would be ready for the
motion system. The requirements allocation steps must also address maturity of the technology.
During the development of the Visual Upgrade Program there were several technology risks that
played a larger role in the modification than the government realized at the time. The technology
for cross-window coherent, wide field-of-view visual systems existed, but had never been scaled
to the dimensions required to support air refueling and engine out requirements. The contractor
was confident this could be done quite easily and convinced the government team. The SE
requirements allocation steps failed to identify this as a risk and assumptions were made about
the capacity of the motion platforms for both the KC-135 as well as the KC-10 simulators.
Other key elements of requirements allocation include early involvement by the support
contractor with the aircraft systems prime contractor(s) to ensure simulator specific requirements
are addressed. Because aircraft upgrades are identified by the KC-135 Program Office at Tinker
AFB there are roadmap meetings held at Tinker where upcoming modifications to the KC-135
aircraft are discussed. The KC-135 ATS O&M contractor and the Training System Program
Office at Ogden now are present to assess those modifications and ensure the ATS requirements
are included in the early planning process. During the KC-135 Block 40 upgrade that was
developed by Rockwell-Collins, FlightSafety sent personnel to Rockwell-Collins to obtain
specific Block 40 Type 1 Training and to attend all of their design reviews to ensure they
(FlightSafety personnel) understood fully how the system was intended to function on the
aircraft. By working closely with Rockwell-Collins, this early involvement also provided the
ATS community with an opportunity to begin the planning and coordinating process for
incorporating training system requirements into the aircraft program as needed thereby reducing
cost and schedule risk to the ATS upgrade.
For example, Rockwell-Collins modified the aircraft software to incorporate software hooks
that were needed to facilitate training system development which helped to ensure the fielding of
a Block 40 configured training device before aircraft arriving on the ramp. Thus, the Rockwell-
Collinss design solution met the user requirements, but also in this case, facilitated increased
trainer efficiency and availability.
Since the Government and Contractor personnel have worked together for over a period of 17
years, an open working relationship has been developed that readily enables FlightSafety to
transform Government (KC-135 ATS Program Office and the KC-135 Program Office) program
objectives and goals into training systems engineering modifications that meets Government
needs. These open communication lines help to ensure that Government expectations are
achieved by the FlightSafety proposed modification.
1.3 Learning Principle-2 The Systems Engineering process must be structured
to properly mitigate challenges generated by third-party Modification
Contractors.
The KC-135 O&M contract states the prime contractor, FlightSafety, is responsible for meeting
the overall performance requirements of the training system including trained students that meet
Government standards. Competitive contracting for early OFT upgrades did not formally involve

Executive Summary Page 5

ID 8845
 KC-135 Simulator Systems Engineering Case Study

buy-in from the O&M contractor who retains ultimate responsibility for providing a
guaranteed student to the Government. This was recognized and remedied in future contracts
by involving the support contractor in a more disciplined manner to ensure early involvement in
the development effort. For example, the requirement for a Performance Work Statement (PWS)
has evolved that identifies specific tasks to be accomplished by the KC-135 ATS prime in order
to ensure training system requirements are identified at the ATS level and properly allocated to
the various subsystems under development. Initially, in the contract phase, Associate Contractor
Agreements (ACAs) were used to define the relationship between the aircraft subsystem prime
contractor and the KC-135 ATS prime contractor. These ACAs typically varied in scope, and as
a result, problems occurred such as modifications to the ATS not being thoroughly tested,
documentation not in a usable format, interfaces not well-defined, ATPs needing to be properly
structured in order to validate performance of the system under development, and in some cases,
insufficient spares being bought. In addition, maintaining configuration control of the ATS was
an issue since the ATS prime contractor did not have much of a hammer to ensure
documentation/drawings provided by third-party contractors were correct and of good quality.
Infusing the tenets of a structured systems engineering process early in the development phase
mitigate these unacceptable third-party consequences.
1.4 Learning Principle-3: Systems engineers must be responsible for ensuring
that all stakeholders are involved during key decision technical planning
and execution process reviews.
The KC-135 program requires a tailored set of formal reviews be held during the development
phase that is based on the size and complexity of the modification program. These reviews
ensure that the entire KC-135 ATS team is working to the same requirements, designing and
developing the correct modifications, adequately testing the modifications and generating the
appropriate courseware changes. The reviews employed throughout the modification
development and verification efforts may include a Systems Requirements Review (SRR),
Preliminary Design Review (PDR), Critical Design Review (CDR), Test Readiness Review
(TRR), Required Assets Available Review (RAAR) and In Process Reviews (IPR). KC-135
simulator reviews are structured to ensure that the KC-135 Simulator team has mutual
expectations and understanding of requirements and that the contractors proposed preliminary
designs and program plan satisfies the development specification. In addition to internal reviews
pertaining specifically to ATS development efforts, it is critical for the ATS systems engineer(s)
to remain focused and cognizant of planned KC-135 aircraft modifications to ensure appropriate
changes affecting training system concurrency and training effectiveness are addressed early in
the development process.
For example, to ensure the proper emphasis is placed on concurrency, KC-135 ATS systems
engineers both within the Government and the ATS support contractor review every
modification to the aircraft to determine if the modification will affect the OFT and aircrew
training programs. For the Pacer CRAG Block 40 upgrade, during the early phase of
development the ATS team ensured simulator specific needs were addressed in the aircraft
design by working closely with Rockwell-Collins, the aircraft prime contractor. The result of this
close coordination was that Rockwell-Collins modified the aircraft software to incorporate
software hooks that were needed to facilitate training system development ensuring the
fielding of a Block 40 configured training device before aircraft arriving on the ramp. In the
flight simulator arena, the application of hooks into the aircrafts operational flight software

Executive Summary Page 6

ID 8845
 KC-135 Simulator Systems Engineering Case Study

allows the training system to easily incorporate such simulator unique functions as freeze frame
or halting the simulation and return to a previous state of simulation such as repositioning the
aircraft on final to repeat a landing sequence.
1.5 Learning Principle-4: Integrated logistics/maintainability support structure
avoids parts obsolescence and diminishing manufacturer supply issues.
As with aircraft systems, training systems must also have detailed technical plans that integrate a
logistics/maintainability support structure to ensure continued future operation. During each
technical planning activity, systems engineering must be concerned with the total support of the
system to assure its economic and effective operation throughout its life cycle. Logistics
objectives for the program need to be included in these technical plans to ensure achieving stated
readiness objectives such as system availability, programmed flying training throughput,
establishment of Reliability and Maintainability performance requirements needed to support
readiness objectives, and emphasizing logistics support considerations in all design trade studies.
During the development phase the systems engineer must incorporate requirements identified by
support organizations in order to properly reflect those in the applicable system
specification/specification change notices, ensure all design trade studies address these
sustainment requirements, and that key design reviews are structured to ensure these issues are
adequately and timely addressed. In addition, it is important for the systems engineer to maintain
awareness of potential life cycle issues, such as diminishing manufacturing sources (DMS) or
parts obsolescence (PO), to ensure there is adequate planning early in the design phase that
address the impacts such issues may cause once the design is fielded.
As an example, the KC-135 ATS program has a requirement for Critical Single Point Failure
Items to be spared at site level unless Mean Time Between Failure (MTBF) data indicates cost is
sufficiently high and failure rate low or item failure will not result in loss of training mission.
Critical items will then be stocked at depot level and supplied to the applicable site with a delay
of less than 48 hours. These requirements are the responsibility of and managed by the Training
System Support Center (TSSC) which has direct responsibility for the continued operation and
maintenance of the ATS and is charged with addressing repair, parts obsolescence (PO), and
Diminishing Manufacturing Sources (DMS). Systems engineering is charged with ensuring
issues and risk mitigation planning associated in these areas are briefed at the quarterly SRBs.
This also facilitates the development and maintenance of a list of priority modifications for AMC
that can be implemented as fall-out money becomes available. For example, FlightSafety
personnel advised AMC of the need for new power supplies. As a result of identifying a potential
problem early, power supplies were procured from a third party thereby ensuring critical student
throughput requirements continued to be met.
A further example of systems engineering proactively addressing potential DMS/PO issues was
the requirement to include a state-of-the-art chip design for many of the subsystems comprising
the VUE. The design approach specified required the system to be designed around a family of
chips that were essentially backward and forward compatible. This commonality of design aided
the support system contractor in maintaining system concurrency and minimizing future
sustainment issues.

Executive Summary Page 7

ID 8845
 KC-135 Simulator Systems Engineering Case Study

1.6 Learning Principle-5: Simulator modeling data/modification requires

verification and validation to ensure aircraft-like flying qualities.
The Systems Engineering process requires both quantitative and qualitative (training value)
verification and validation of the modifications to the Operational Flight Trainer (OFT).
Quantitative verification ensures that component changes to the OFT are validated via testing to
ensure that the modification works in the ATS environment as intended. Low-level, detailed
testing by subcontractors and/or third-party contractors is performed to verify the subsystem
performance. Functional mission tests are Government-conducted tests of the prototype
modification that use Government-defined scenarios to evaluate the operational characteristics of
the systems within the context of conducting the mission. Revisions to courseware products,
such as, classroom lecture, computer based training (CBT), training device, and aircraft lessons
are verified and validated via the formative evaluation processes that include subject matter
expert (SME) review, and individual tryouts (ITOs) and small group tryouts (SGTOs) where
applicable.
Ultimately, for an OFT to be effective as a training device, the aero models, visual system
models, aircraft/cockpit sounds, etc., must provide the student with sufficient cues that are
realistic enough to provide for realistic training. For ground-based training devices, this is really
a qualitative assessment about the realism of the simulatormeaning its a judgment call by the
test crews and Air Force instructors about the systems training value. To the ATS systems
engineer, this issue of subjective testing has been an ongoing dilemma. It has proven extremely
difficult for systems engineers to quantitatively specify this training value. No matter how much
experience a team has quantifying and measuring simulator performance, in reality it remains a
qualitative assessment about the realism of the simulator. The challenge for the systems engineer
in a training program is to not only develop performance requirements that can be measured and
verified but also develop the process by which training value can be qualitatively assessed and
validated while protecting against personality-driven assessments that can change with
Government personnel turnover.
Therefore, the verification/validation process employed by the KC-135 ATS team relies on a
combination of qualitative and quantitative test procedures including the development of
Acceptance Test Procedures (ATPs) that are conducted to verify compliance of the modification
with the requirements as specified in the Prime Item Development Specifications. The KC-135
simulator program incorporates user validation of system performance, through the simulator
certification (SIMCERT) process, to ensure that the OFT flies like the aircraft and the graduate
meets AMC training standards. Formal SIMCERTs, conducted approximately every six months
by the Air Force, certifies that the ATS continues to meet system specification requirements for
the hardware and software. Additionally feedback from students is received and analyzed to
ensure that the training is provided in an effective manner.
AMC and FlightSafety personnel have also developed a test and evaluation process that
promotes confidence that KC-135 simulator modifications fly like the KC-135 aircraft. Even
though the systems integrating contractor and the Government quantitatively specify many
requirements, the final evaluation of the training realism is still subjectively validated. In the end,
the trainer model must be correct enough to allow training, which means its a judgment call by
the test crews and Air Force instructors about the systems training value. The KC-135 ATS
team has implemented an approach, which utilizes no more than one or two contractor instructor
(FlightSafety) pilots and one Air Force instructor pilot to minimize extended test periods and

Executive Summary Page 8

ID 8845
 KC-135 Simulator Systems Engineering Case Study

facilitate reaching consensus on a modifications training value. Ultimately, both Air Force and
FlightSafety pilots validate via testing that the modification fulfills its intended use when placed
in its ATS environment.
Finally, the KC-135 ATS team relies on course ending surveys/comments prepared by students,
which includes a rating of the training value received (scale of 1 to 5), consistent monitoring of
students performance and progression, and, as a final proof, a Government-conducted check
ride to ensure this requirement is met.

Executive Summary Page 9

ID 8845