Using Cloud Computing to Enhance Automatic Test

Equipment Testing and Maintenance Capabilities

Dale D. Reitze
Systems Engineer
Northrop Grumman Corporation
Electronic Systems Defensive Systems Division
Rolling Meadows, Il 60008
dale.reitze@ngc.com

Abstract The purpose of this paper is to present a conceptual

approach and to make practical recommendations on how to
improve the current Automatic Test Equipment (ATE) testing and
maintenance capabilities by utilizing the existing cloud computing
model to build a globally linked ATE maintenance system. The
basic tenet of the ATE community is to support a multi-tiered
maintenance concept which, in general, is a three tiered system that
is composed of organizational maintenance (O-level), intermediate
maintenance (I-level), and depot maintenance (D-level)
organizations. The goal of the ATE is to (1) quickly and
accurately detect and isolate each fault, (2) provide software tools
for analyzing historical data, and (3) gather, manage, and distribute
accurate and reliable maintenance information for the failed Unit
Under Test (UUT). The ATE system should provide services that
will (1) maintain a repository of information that will improve fault
detection and isolation, allow for off-platform assessments,
document failures, and help quantify corrective actions, (2) reduce
false UUT pulls, and (3) reduce repair time by prompting repair
procedures.
Furthermore, the ATE system should provide
additional services that will help optimize the time to diagnose
problems by using collected failure information and by
recommending entry points into the Test Program Set (TPS)
software. It should also present information to the ATE maintainer
to aid in informed repair decisions which could be in the form of
pilot debrief results, platform Built In Test (BIT) results, O-level
test outcomes and corrective actions, and maintenance and usage
history of the platform and UUT. So, based on this definition of
ATE maintenance the use of cloud computing can be used to
provide services to improve the overall ATE testing throughput
which will result in bottom line improvements to ATE life cycle
costs. By using cloud computing, which is defined to be a model
for enabling ubiquitous, convenient, on-demand network access to
a shared pool of configurable computing resources (e.g., networks,
servers, storage, applications, and services) that can be rapidly
provisioned and released with minimal management effort or
service provider interaction, users can develop cloud computing
models that will provide access to application software and
databases that can be used to build a globally linked ATE
maintenance system. This paper will discuss the essential
characteristics of the cloud computing models and define the
various flavors of cloud offerings available to designers today.
This paper will also analyze the cloud computing model to arrive at
a conceptual approach that can be used to enhance the current ATE
Testing and Maintenance capabilities. Practical recommendations
will be discussed on how to transform the current ATE Testing and

978-1-4673-5683-1/13/$31.00 2013 IEEE

Maintenance capabilities into the specific cloud computing model

offerings in order to help configure a globally linked ATE
maintenance system.
Keywords-ATE,
maintenance

Cloud

I.

Computing,

architecture,

INTRODUCTION

The definition of cloud computing as taken from work by

the National Institute of Standards and Technology (NIST) is
a model for enabling convenient, on-demand network
access to a shared pool of configurable computing resources
(for example, networks, servers, storage, applications, and
services) that can be rapidly provisioned and released with
minimal management effort or service provider
interaction[1].
NIST defines four cloud deployment models [2]:

Public clouds (cloud infrastructure made

available to the general public or a large
industry group)

Private clouds (cloud infrastructure operated for

an organization)

Community clouds (cloud infrastructure shared

by several organizations)

Hybrid clouds (cloud infrastructure

combines two or more clouds)

that

With each of these deployment models come three

specific service models that are available to the cloud
consumers: cloud software as a service (SaaS), cloud
platform as a service (PaaS), and cloud infrastructure as a
service (IaaS). Finally, the NIST definition also provides a
unifying view of five essential characteristics that all cloud
services exhibit: on-demand self-service, broad network
access, resource pooling, rapid elasticity, and measured
service [3].
The main emphasis on the cloud computing model is a
more economic method of providing higher quality and
faster services at a lower cost to the consumers. The cloud
computing model will help enable ATE manufacturers to

focus their attention on creating and delivering innovative

services to the end user. In order to deliver high quality
products, the ATE community needs to ensure the reliability
in the delivery of products and processes. By ensuring and
adhering to defined standards for cloud computing in
security, data portability and service interoperability, the
ATE manufacturers will be able to move their applications
into the cloud.
The NIST computing architecture focuses on the
requirements of what cloud services provide and not on
how to design and implement a cloud based solution. This
paper is intended to be used as a tool for describing,
discussing, and developing a system specific cloud
architecture using a common reference point as a starting
point that can be utilized to enhance the current ATE Testing
and Maintenance capabilities.
II.

OVERVIEW OF THE CLOUD COMPUTING REFERENCE

ARCHITECTURE

The NIST conceptual reference model is composed of

five major actors: cloud consumer, cloud provider, cloud
carrier, cloud auditor, and cloud broker (refer to figure 1 for
an overview of the cloud reference model) [3]. The actors in
this model define an entity (person or organization) that takes
part in a transaction or a process and/or performs tasks in the
cloud computing environment. Each of the actors and their
respective definitions will be discussed in the successive
paragraphs of this section.
The cloud consumer is the principal stakeholder for the
cloud computing service. The cloud consumer is a person or
organization that establishes a relationship with the cloud
provider who provides them with services [3]. The cloud
services provided by the cloud provider are varied and range
from Database services to Development & Testing services
to Computing services. The consumers of SaaS can be
organizations that provide members with access to software
applications, end users who directly use software
applications, or software application administrators who
configure applications for end users [4]. The consumers of
PaaS can be application developers who design and
implement application software, application testers who run
and test applications in cloud based environments,
application deployers who publish applications into the
cloud, and application administrators who configure and
monitor application platform on a platform [4]. The
consumers of IaaS can be system developers, system
administrators and IT managers who are interested in
creating, installing, managing and monitoring services for IT
operations [4]. The potential type of cloud services available
are endless and can be created by the cloud provider as
required by the cloud consumer.
The cloud provider is a person or organization that makes
services available to interested parties [3].
The cloud
provider is responsible for purchasing and managing the
computing infrastructure required for providing the services,
manages the cloud software that provides the services, and
arranges to have the cloud services delivered to the cloud
consumer through a computing network.
The cloud
providers activities are defined in the following areas:

service deployment, service orchestration, cloud service

management, security, and privacy. The details of these five
activities will be discussed in section III.
The cloud auditor is a party that performs an independent
examination of the cloud service controls [3]. The audits are
performed to verify conformance to prescribed standards
through review of objective evidence. The cloud auditor can
review the services provided by the cloud provider with
respect to security controls, performance benchmarks,
privacy concerns, etc.
The cloud broker is an organization that manages the use,
performance and delivery of cloud services and acts as a
negotiations liaison between cloud consumers and cloud
providers [3].
The cloud carrier acts as an intermediary that provides
connectivity and transport of cloud services between cloud
consumers and cloud providers [3]. The cloud carrier
provides access to consumers through network,
telecommunications and other access devices.
A certain scope of control is defined between the cloud
provider and cloud consumer because the cloud provider and
cloud consumer share the control of resources in a cloud
system. Different service models (IaaS, PaaS, and SaaS)
affect an organizations control over the computational
resources (application layer, middleware layer, and operating
system layer) and thus what can be done in a cloud system
[3].

The application layer includes software

applications targeted at the end user of the
programs. The applications are used by SaaS
consumers, or installed/managed/maintained by
PaaS consumers, IaaS consumers, and SaaS
providers.The middleware layer provides
software building blocks such as libraries,
database, and virtual machines for developing
application software in the cloud.

The middleware is used by PaaS consumers,

installed/managed/maintained
by
IaaS
consumers or PaaS providers, and hidden from
SaaS consumers.

The Operating System layer includes operating

system and drivers which are hidden from SaaS
consumers and PaaS consumers. An IaaS cloud
allows one or multiple guest Operating Systems
to run virtualized on a single physical host.
Generally, consumers have broad freedom to
choose which Operating System to be hosted
among the Operating Systems that could be
supported by the cloud provider. The IaaS
consumer assumes full responsibility for the
guest Operating System, while the IaaS provider
controls the host Operating System.

Service orchestration is comp

posed of a service layer, a
resource abstraction and contrrol layer, and a physical
resource layer [3].

III.

The service layer orr the top layer is where the

cloud provider defin
nes the interfaces for cloud
consumers to access the computing services [3].
The service modelss (SaaS, PaaS, and IaaS)
interfaces are provideed within this layer. A user
of IaaS is operatin
ng at the lowest level of
granularity available and with the least amount
of prepackaged functtionality so basically this is
an environment for building
b
native applications.
A user of PaaS is op
perating at the middle level
and requires less inteeraction with the bare metal
of the system so basically this is an environment
for building a man
naged application with an
Integrated Developm
ment Environment with a
rich class library th
hat executes in a specified
runtime environmen
nt. A user of SaaS is
operating at the high
hest level so basically this is
an environment fo
or using some form of
packaged software applications.
a
It is possible
that SaaS applications can be built on top of
PaaS components an
nd PaaS components can be
built on top of IaaS components.

The middle layer in the model is the resource

abstraction and conttrol layer [3]. This layer is
composed of the system components that cloud
providers use to provide and manage access to
the physical computting resources via software
abstraction. Examples of resource abstraction
components include virtual
v
machines and virtual
data storage. The ressource abstraction needs to
make sure that efficient, secure, and reliable
usage of the phy
ysical resources is being
maintained. The co
ontrol aspect refers to the
software componentts that are responsible for
resource allocation, access control, and usage
monitoring. This is the software fabric that
binds together the un
nderlying physical resources
and the software abstractions that enables
resource pooling, dynamic allocation and
measured service.

The lowest layer in this model is the physical

resource layer wh
hich includes all of the
computing resourcess [3]. This layer includes
hardware resourcess, such as computers,
networks, storage components and other
physical computing elements.
e

ARCHITECTURAL COMPONENTS OF TTHE CLOUD

COMPUTING REFERENCE ARCHITECT
TURE

This section will discuss the details of thhe architectural

components defined in the cloud reference moddel.
The first architectural component is service deployment
which defines how the cloud infrastructuree is operated.
Therefore, a cloud configuration may be operrated using one
of the following deployment models: public cloud, private
cloud, community cloud, or hybrid cloud. T
The differences
are based on the type of computing resources m
made available
to the cloud consumer.
A public cloud is one in which the cloudd configuration
and computing resources are made available to the general
public over a public network [3]. A public clooud is managed
by an organization selling cloud services.
A private cloud gives the individual clouud consumers
organization exclusive access to and usagee of the cloud
configuration and computational resources [3]]. The private
cloud may be managed by the cloud consumeer organization
or by a third party.
A community cloud serves a group of clooud consumers
which have shared concerns such as similar projects the
community cloud can serve multiple orgganizations as
opposed to a private cloud which serves a singgle organization
[3]. A community cloud may be managedd by the cloud
consumer organization or outsourced to a thirdd party.
A hybrid cloud combines both private andd public clouds
that still remain as distinct entities but are tied together by
standardized technology that enables data aand application
sharing [3].
The second architectural componentt is service
orchestration which refers to the design oof the system
components that are used to support the clooud providers
activities in arrangement, coordination and m
management of
the computing resources in order to provide clooud services to
cloud consumers.

The service layer is dependen

nt on the middle layer and
lower layer to function. The resource abstraction and
control layer exposes virtual clo
oud resources on top of the
physical resource layer and it supports the service layer
where cloud interfaces are expossed to the cloud consumers.
The cloud consumers dont have ready
r
access to the physical
resources.
mponent is cloud service
The third architectural com
management which includes all
a of the service related

functions that are required for the management and operation

of services required by or proposed to cloud consumers [3].
Cloud service management can be decomposed into three
segments from the perspective of business support,
provisioning and configuration, and portability and
interoperability requirements.

Business support entails the set of business

related services dealing with clients and
supporting processes.
It includes the
components necessary to run a business
operation, such as customer management,
contract management, inventory management,
accounting and billing, reporting and auditing,
and pricing and rating.

Provisioning
and
Configuration
entails
providing the set of services to support rapid
provisioning, resource changing, monitoring and
reporting, metering, and service level
agreements management of the cloud to the
cloud consumers.

For portability, prospective customers are

interested in whether they can move their data
or applications across multiple cloud
environments at a low cost with minimal
disruption.
From an interoperability
perspective, users are concerned about the
capability to communicate between or among
multiple clouds.
Cloud providers should
provide the required mechanisms to support data
portability, service interoperability, and system
portability. Data portability allows the cloud
consumers to copy data objects into or out of a
cloud. Service interoperability allows the cloud
consumer to use their data and services across
multiple cloud providers with a common
management interface.
System portability
allows the migration of virtual machine
instances or a machine image from one provided
to another provider, or migrate applications and
services and their contents from one service
provider to another.

The fourth architectural component is security of the

cloud system so it is critical to recognize that this is a crosscutting aspect of the architecture that spans across all layers
of the reference model, ranging from physical security to
application security [3]. Security is not only the concern of
the cloud provider but also of the cloud consumer and the
other relevant actors. Cloud systems need to address security
requirements such as authentication, authorization,
availability, confidentiality, identity management, integrity,
audit, security monitoring, incident response, and security
policy management. The cloud system needs to ensure that
the three service models (SaaS, PaaS, and IaaS) are designed
with security measures in place so that security attacks can
be monitored and blocked.
Additionally, the cloud
deployment models (public cloud, private cloud, community
cloud, and hybrid cloud) need to be designed with security
measures as well.
Basically, security is a shared

responsibility between the cloud provider and the cloud

consumer, and the measures used to provide protection need
to be analyzed to determine which party is in a better
position to implement the required security measurements.
This analysis needs to include considerations from a service
model perspective, where different service models imply
different degrees of control between cloud providers and
cloud consumers.
The fifth architectural component is privacy. Cloud
providers need to be able to protect the disposition of
personal information while this information is contained in
the cloud [3].

IV.

ATE BASICS

The ATE environment is typically composed of one or

more similar test stations configured in a standalone mode of
operation. Furthermore, test stations may be located in
multiple locations throughout the world and are typically
operated in a non-networked environment. ATE systems can
be broken down into two major system categories of
hardware and software. The hardware is a set of physical
devices that make up the ATE. The ATE hardware is
available in widely different shapes, sizes, and capabilities,
but most ATE shares some common components such as the
host computer, instrumentation busses, and some form of an
interface test adapter. The ATE software is the programming
instruction code that drives the different components
contained in an ATE environment. Although ATE varies
widely in software implementations, generally the following
software categories can be found in ATE systems such as,
the Operating System, the Run Time System, Instrument
Drivers, Resource Management, Test Programs, Diagnostic
Programs, and a Development Environment. The hardware
category may be difficult to replicate and manage in a cloud
computing model, but some of the software categories would
be perfect candidates for migrating into the cloud.
V.

ATE SOFTWARE AND CLOUD COMPUTING

Some, if not all, of the ATE software components defined

in section four could be migrated to the cloud to form an
ATE cloud service that could be used by consumers to build
an automatic testing environment. Various cloud model
services can be configured to meet the specific needs of the
ATE consumer. Commercial ATE consumers may want to
use a public or community cloud when architecting their
ATE environment while military ATE consumers may want
to use a private cloud when architecting their ATE
environment. The different types of cloud deployment
models will come at a cost with the private cloud being the
most expensive and the public cloud being the least
expensive to build and maintain due to economies of scale
[4]. As to the ATE software component services that could
be made available to the cloud consumers it is quite possible
that more than one cloud service providers could ultimately
design, build, and manage the operating system, the run time
system, instrument drivers, resource managers, test
programs, and diagnostic programs. The cloud service
providers could even configure and maintain the software

development environment which would enablee them to build,

deliver, and maintain multiple configurationns of the ATE
system, which would become an important serrvice to provide
as station upgrades are made to the hardwaree and software
during the lifecycle of the ATE program. If changes are
made to any of the ATE software componentss, as controlled
by one or more cloud providers, then those uppdates could be
delivered to the ATE environment as requesteed by the ATE
consumer.
A very high level ATE public cloud
configuration is shown in figure 2.
Furthermore, the ATE organization needs ddetermine what
type of service model to use as part of their arcchitecture - will
they be SaaS, PaaS, or IaaS consum
mers of the
cloud. Basically, at what level do they want to request and
receive services from the various cloud providders of software
services. The consumers of SaaS typically use packaged
software applications such as office productivvity tools. The
consumers of PaaS typically need an ennvironment for
building a managed application with an integrated
development environment with a rich classs library that
executes in a runtime environment, such ass an operating
system or an ATE run time system. The conssumers of IaaS
typically need an environment for building specific native
applications, such as instrument drivers, resource
management programs, and test/diagnostic pprograms. So,
based on this information the ATE archiitects need to
determine which cloud providers can provide tthe best service
models to help them build and architect their A
ATE system.

VI.

MS
NETWORKING ATE SYSTEM

It is becoming possible to use cloud netwoorking to build

networks that address rapidly evolving business and
engineering challenges without the cost, complexity,
and constraints of traditional networks. In the future it may
be possible for similar ATE systems to shaare testing and
diagnostic data across the cloud network. T
This data could
then be used by the various ATE facilities tto help resolve
problems by providing services that can (11) provide test
program entry point direction, i.e., directed ttesting, (2) use

automated reasoners to accuratelly isolate and repair faults,

and (3) maintain a detection an
nd fault isolation heuristic
database that can help docum
ment failures and quantify
corrective actions. These types of cloud services could be
nsumer over the cloud
provided to the cloud con
network. The overall goal of netw
working ATE systems is to
improve test times (or Unit Und
der Test throughput) and to
help reduce unnecessary failure reepair actions.
VII.

CONCLLUSION

A cloud computing model was

w derived based on the
NIST definition which lays out an
a architecture that defines
four cloud deployment models (p
public cloud, private cloud,
community cloud, and a hybrid cloud) and three deployment
models (SaaS, PaaS, and IaaS) that are available to cloud
consumers such as the ATE com
mmunity. NIST also defines
the five essential characteristicss (on-demand, self-service,
broad network access, resource pooling, rapid elasticity, and
measured service) that all cloud services need to exhibit in
order to provide a common standaard for the cloud computing
model. By adhering to these standards,
s
the builders and
consumers of cloud services willl be guaranteed that a solid
foundation is being used to dev
velop new services and to
enhance existing services. The ATE community has a
chance to take advantage of clo
oud computing by building
ATE systems which adhere to mo
odels set forth in this paper.
The overall goal is to use on
ne or more of the cloud
computing models to provide serv
vices to improve the overall
ATE testing throughput time. Increased throughput will
result in bottom line improvemen
nts to ATE life cycle costs.
Some of the services that could bee built by one or more cloud
providers are data reasoners, fault isolation databases,
instrument drivers, test set and diagnostic programs, and
various run time systems and opeerating systems. The cloud
consumer could request that thesse services be delivered to
their ATE system either piecemeaal or as a configurable item
to be run on their ATE hardwarre. The services could be
delivered to one ATE system or to
t multiple ATE systems if
they are tied into the ATE netwo
ork. The resulting services,
once downloaded and configured,, would then be executed on
the ATE hardware with the expreess goal of being to improve
our overall testing capabilities, either through throughput
improvements or through en
nhanced debugging and
diagnostic capabilities that willl help in isolating UUT
failures. The future is here and the
t resources exist to begin
to architect ATE systems that caan take advantage of cloud
computing services to enhance our testing and maintenance
capabilities.

REFEREN
NCES
[1]
[2]

Peter Mell, Timothy Grance, T

The NIST Definition of Cloud
Computing, NIST Special Publicattion 800-145, September 2011.
DMTF Informational, Interoperablle Clouds A White Paper from
an Open Cloud Standards Incubaator, Document Number DSPISO101, November 2009.

[3]

[4]

Fang Liu, Jin Tong, Jian Mao, Robert Bohn, John Messina, Lee
Badger and Dawn Leaf, NIST Cloud Computing Reference
Architecture, NIST Special Publication 500-292, September 2011.
Jothy Rosenberg, Arthur Mateos, The Cloud at Your Service The
when, how, and why of enterprise cloud computing, Manning
Publications Company, 2011.