Sie sind auf Seite 1von 89

Gartner Data Center, Infrastructure &

Operations Management Conference


The Venetian
December 5 – 8, 2016

Premier Sponsors’
White Papers
The sponsor white papers contained in this booklet are authored by premier sponsors of this Gartner event, who are solely
responsible for their content. Gartner, Inc. in no way endorses the content of the enclosed white papers or the vendors who
produced them, and disclaims all warranties as to the accuracy, completeness or adequacy of the information contained therein.
WHITE PAPER

Six Important Considerations


When Choosing a
Colocation Provider
David Meredith SVP and Global GM, CenturyLink Business

1 White Paper Hybrid IT Colocation


WHITE PAPER

Six Important Considerations When


Choosing a Colocation Provider
Six Important Considerations Planning, Coordination and
Management
When Choosing a Colocation Provider Operational Staffing and Organization

Practices Training
Colocation services may seem like nothing more than a Operating Conditions
commodity; a data center is a data center. However, what Maintenance
appear as just minor differences between providers can have a
major impact on the overall performance of your business.

Proximity
With applications supporting your key customer interactions and Data Center
Global Footprint
business functions, all aspects of your IT infrastructure, including Location
Available Capacity
your colocation provider, need to be scrutinized. Whether it is
an improved customer experience, 100 percent uptime for your
critical applications or better alignment of IT with your business
priorities, careful consideration of your colocation services Tier 1 Internet Access
provider will impact your ability to achieve these goals. Private Data Networking
Connectivity &
Carrier Diversity
Service Options
From physical location to network integration, there are Access to Managed &
Cloud Services
important elements to consider when placing your hardware
with a colocation company. Asking the right questions can
ensure an optimal deployment. Any latencies or points of failure
need to be eliminated or minimized to secure the performance Physical Security

of your business applications. Redundant systems help protect Security & Network Security

your business and keep it operating to serve your customers, no Compliance Layered Security Services
Compliance
matter what unforeseen events may arise. Also, having a secure
environment protects your business from intrusions that can
have a devastating impact on your business.

Remote Hands
Each business’s needs are slightly different, and you should Support Migration
bear in mind the operational dynamics that make your business Services Design & Installation
unique. That said, the six considerations discussed in this paper Structured Cabling
are relevant to all colocation environments that clients deploy.
They are the essential building blocks of a colocation program,
and it is critical to select a provider that can deliver on all these
attributes.
Power & Reliability of Power Systems
Cooling Cooling Technologies

2 White Paper Six Important Considerations When Choosing a Colocation Provider


Does My Colocation Provider Have the
Necessary Practices and Methods to
Assure Maximum Uptime?
The Uptime Institute reports that 70 percent of all data center Together these behaviors assess the operational practices of a
outages are the result of human error. No matter how resilient data center with focus on the areas most often cited as the root
a data center’s design and construction, you can still have cause of the human errors that adversely impact data center
outages if the operations team does not manage and operate operation. The M&O Stamp of Approval validates the rigor and
the facility with an eye toward operational excellence. One way effectiveness of the facility management and operations and
of accessing the operational practices of a data center is to look gives stakeholders the assurance they need that effective risk
for the Uptime Institute Management and Operations Stamp mitigation is in place.
of Approval. The Uptime Institute M&O Stamp of Approval
provides a means to conduct risk analysis by examining five key
behaviors:

Planning, Coordination and Management Operating Conditions


Site Policies Load Management
Financial Process Operating Set Points
Reference Library
Capacity Management Maintenance
PM Program
Staffing and Organization Housekeeping
Staffing MMS
Qualifications Vendor Support
Organization Deferred Maintenance
Predictive Maintenance
Training Life-Cycle Planning
Data Center Staff Failure Analysis Program
Vendors

How Close is My Colocation Provider’s


Data Center?
A data center that is close to your company’s offices is a small transactions — continue to become widespread, any
common requirement for companies shopping for colocation latency issues become exacerbated and can severely impact
services. However, the advantages go well beyond providing performance. Proximity can help reduce the effect these
easy access for your staff. Proximity to your location can help latencies have on your applications.
improve the performance of your IT infrastructure. If you
are sending large volumes of data from your primary site to In addition to having a center close by, the ability to tap into a
your colocation environment, distance matters. A site that is network of other centers can help make your IT more resilient.
physically close reduces data replication issues. Minimizing Look for a provider that has a data center near you and also has
latency delays has always been important for application multiple options to provide locations that are as far away as
performance. As chatty applications — meaning those that required from your primary site. When you can deploy colocation
wait for server acknowledgment or perform a number of environments across geographies to back up data, run additional

3 White Paper Six Important Considerations When Choosing a Colocation Provider


instances of applications and circumvent local weather or power out your environment. Having the option to expand your space
disruptions, you have a solid foundation for business continuity in the future can save you significant time and costs compared
planning. If you need to deploy colocation environments in other with deploying with a different provider or moving to another
locations, a provider with a global footprint will more likely be data center. Look for providers that are well funded to invest in
present in the major markets you need to serve. additional build-outs for the data center. Providers that do their
builds on a modular basis keep a tighter control on costs but are
When it comes to the availability of physical space, make sure ready to expand to meet client needs.
the provider has additional capacity in case you need to build

How Can I Connect My IT Infrastructure


to Users, Business Partners and Cloud or
Managed Service Providers?
Colocation is more than just racking and stacking your equipment colocation is only one part of a bigger IT picture — you shouldn’t
in a data center and adding a network connection. Without highly think of it in isolation. Consider your selection of a colocation
reliable and redundant network connectivity, your IT performance provider in the context of how your entire IT infrastructure
will suffer. A provider with a full range of connectivity options impacts your business. In addition to colocation services, many
can ensure all your locations, customers and business partners companies require capabilities like managed hosting and are
get the access to the applications and resources they need in developing cloud solutions. A partner that can offer access to all
your colocation environment. these services, one with expertise across the entire portfolio,
can help you achieve a flexible, better-performing infrastructure.
It also is critical that your provider has carrier diversity at
their data centers if you are running applications that have Cloud services will continue to grow as part of the IT solution
this requirement to ensure reliable and redundant network set, but many organizations will continue to have a need for
connectivity. Make sure your colocation provider can offer colocation as part of their infrastructure. The future will likely
you advanced networking capabilities so you can meet your see many side-by-side colocation and cloud environments. Your
application performance and uptime requirements. colocation provider should understand the capabilities of a cloud-
Connectivity increasingly goes beyond networking connections based system and be able to help you work across multiple
to the user community and to connectivity between IT platforms, environments.
including leading cloud providers. For most companies,

How Does My Colocation Provider Ensure


Security and Compliance?
As you design your environment to be secure, don’t forget the an important extra level of protection.
importance of physical security measures in the data center.
Your provider should use current technologies like biometric Beyond the physical security of the center, look for providers
scanners, card readers and video monitors as well as mantraps than can augment your security with services like DDoS
to prevent unauthorized access. Installing your own cameras protection, network security and threat detection. Professional
with the ability to remotely monitor the activity in your colocation services that can test, evaluate and remediate security
cage not only enhances security but also is an efficient way to vulnerabilities can help you increase your ability to withstand
better control your environment — and that should be something intrusions.
your provider can enable. People add a layer of security, too;
providers that staff their centers 24/7 with trained security Your provider also should be compliant with any regulatory
personnel instead of relying solely on automated systems offer requirements for your industry and know the ins and outs in a

4 White Paper Six Important Considerations When Choosing a Colocation Provider


continually shifting regulatory environment. The data centers If you need colocation services in a geographic location that
should be compliant with SSAE 16 service controls. Publically has high risk factors, make sure your provider has addressed
traded companies should seek providers that meet SOX, and them. For example, if you are locating resources in an area prone
any company providing ecommerce needs to meet PCI security to earthquake activity, the data center you select should use
standards. If you are in an industry that requires specific seismic-compliant construction techniques.
compliance credentials such as FISMA in financial services or
HIPAA for healthcare providers, make sure your provider can
meet these stringent requirements.

What Kind of Support Resources


are Available?
Providers that simply give you space and an electrical hook-up If you are a mid-sized or large company, you know that “one
may seem, at first glance, to provide a lower cost structure, size fits all” does not apply to your IT needs. Find a provider that
but they will require you to staff up or pay for services that full understands the complex relationships of your specific business
colocation service providers include in their offering. Make sure processes, IT governance model, application stack and security
you examine the Total Cost of Ownership (TCO) of each of needs. Your provider should operate in a way that is consistent
your alternatives. In a true colocation services environment, the with your internal controls, the processes you utilize for change
provider should address security, provide power and cooling, management and how you respond to incidents. How closely
perform facility management and be able to deliver these these processes are aligned can make a big difference in how
services with a Service Level Agreement (SLA). An SLA imposes well a provider will meet your colocation needs on a day-to-day
financial penalties on the provider if they fail to meet agreed- basis.
upon service metrics.
The experience of the on-site data center staff will have a
On-site support that is available 24/7 provides quick response for significant impact on the level of support you receive when
emergency re-boots or other issues when your staff is not in the you need it. Data center management is a complex task that
center and can help prevent minor incidents from escalating into requires specialized skills across a wide number of domains such
larger issues. The availability of such services can help you meet as security, power distribution, networking and hardware and
your response time requirements while effectively managing software management. Make sure your provider has the skilled
your IT budget. staff for all the key data center capabilities, and they do not
simply manage vendors that come in to service the data center
Many, if not most, companies find it takes time to stand up their systems. Engineers and technicians who are highly trained and
environment and migrate applications. Providers with modular certified with extensive experience in data center management
growth plans give you the flexibility you need to grow your will provide you with dependable service that improves your
environment as you migrate. Look for a contract that gives you uptime. Providers that offer managed hosting and cloud
the option to fill space over a period of time. This can reduce services are often better suited to help with your colocation
costs and allow you to adapt to changing business conditions so needs as their data center staff is trained and experienced to
you can increase or decrease the speed at which you deploy into support complex environments. Make sure your provider has
the center. standard, documented processes and practices in place for all
their activities like maintenance and change management. Such
Providers that offer services to help you move in and can assist standardization drives better performance and uptime.
with hardware racking and stacking or cabling can help make
your deployment go smoothly and reduce migration time. A provider with a portal that supports trouble ticketing and
Structured cabling programs provide installations that meet reporting can help you better manage your environment. Remote
standards for design and testing, ensure better performance management increases your IT staff’s efficiency and enables
of your infrastructure and enable future changes to be made you to be proactive and resolve incidents before they affect the
quickly. performance and availability of your applications.

5 White Paper Six Important Considerations When Choosing a Colocation Provider


Setting up a new environment is a complex undertaking, and driven by green IT principles such as improving server density
missteps can reverberate for years. Providers that have the during the design phase of your environment.
expertise to assist you in the design of your new environment
and can customize an installation based on your needs ease the Given that your staff may be spending significant time at the
installation process and simplify future maintenance. Details data center, look for a provider that offers workspace, lounges
matter: having a cooling expert involved in the initial design can and conference rooms where they can connect laptops or take
greatly improve cooling efficiency by using optimal rack design a conference call. These amenities provide a comfortable work
and other techniques. A provider can help you reduce power environment and help keep your staff productive.
consumption by suggesting ways to move to configurations

What are the Power and Cooling


Capabilities of the Center that will House
My Equipment?
Power is a critical data center element, so look for a provider Server power demands are increasing and so is the power
that has a 100 percent uptime SLA for power and redundant consumption in today’s high-density server environments. It is
power systems. Providers that can offer this assurance have important that the center includes efficient cooling systems, and
engineered their N+1 or better systems with concurrently the provider continues to invest in new cooling technologies
maintainable power resources, so they provide uninterrupted to protect your hardware. Look for cooling techniques like hot
power during both routine maintenance and when any power and cold aisles that provide a front-to-back cooling profile with
source is brought offline. The provider should be able to not blanking plates. Proper air containment designs and cooling
only deliver the power you need today but also meet your future walls promote cooling and can help avoid hot-spots in your cage.
needs. Providers that cannot adequately power their space will Improved cooling also can extend the life of your equipment
limit your options as you seek to grow your colocation installation and improve performance by reducing hardware failures.
or migrate to new servers with higher power requirements. Make sure the center meets your required standard ranges for
temperature and humidity. A provider that can participate with
Your provider should offer transparency and detail in its billing. you to integrate cooling techniques into the design, operation or
Providers that offer options for billing your power based upon upgrading of your environment can help you promote the green
distribution of power circuits, or metered power, can help initiatives of your company.
reduce your costs depending on your configuration and power
requirements.

Summary
As described in each of the six considerations, there are CenturyLink has been providing colocation services to large
significant differences in colocation service providers — from clients for 20 years and has deep expertise and a strong track
staffing to connectivity to geography. With the availability and record of delivering high performance IT environments. As a
performance of your business applications riding on the selection full-service provider, CenturyLink brings the flexibility required to
of a provider, make sure you select one that can help you meet deliver the solutions you need today and the vision to help get
your goal of 100 percent uptime. Colocation is a long-term you where you need to be in the future.
commitment, and the cost and business disruption that comes
from moving installations makes the right selection critical.

6 White Paper Six Important Considerations When Choosing a Colocation Provider


Our assets include over 60 global data centers, a global Operational Excellence can give you the peace of mind that your
network that delivers high levels of security and availability IT infrastructure is up and running so you can focus on your
and a seasoned team of professionals who deliver IT solutions business.
that improve business performance. Our commitment to

About CenturyLink Business


CenturyLink, Inc. is the third largest telecommunications
company in the United States. Headquartered in Monroe, LA,
CenturyLink is an S&P 500 company and is included among the
Fortune 500 list of America’s largest corporations. CenturyLink
Business delivers innovative private and public networking and
managed services for global businesses on virtual, dedicated
and colocation platforms. It is a global leader in data and voice
networks, cloud infrastructure and hosted IT solutions for
enterprise business customers.

For more information visit www.centurylink.com/enterprise.

Global Headquarters EMEA Headquarters Asia Pacific Headquarters Canada Headquarters


Monroe, LA United Kingdom Singapore Toronto, ON
(800) 728-8471 +44 (0)118 322 6000 +65 6591 8824 1-877-387-3764

©2016 CenturyLink. All Rights Reserved. The CenturyLink mark, pathways logo and certain CenturyLink product names are the property of CenturyLink.
All other marks are the property of their respective owners. Services not available everywhere. Business customers only. CenturyLink may change or
cancel services or substitute similar services at its sole discretion without notice. 399010114
The Case for
Orchestration
of Cloud
Infrastructure
How Intelligent and Automated Resource
Provisioning Enables Business Agility
DATA CENTER AS INNOVATION ENGINE
All CEOs share an imperative: Grow the business. Today
that means making the right investments in technology and
infrastructure that can drive business velocity.

More than ever, the C-suite is turning to technology to help


them get a leg up on their competitors. Businesses are searching
for ways to thrive in a world where cloud, data analytics, and
the Internet of Things are offering new opportunities to drive
revenue and evolve their business models.

What does it take for businesses to be able to capitalize on


these opportunities and bring viable ideas to market rapidly,
while fending off competitive threats?

Modern Infrastructure Drives


Business Agility
An agile business demands an agile infrastructure.
Unfortunately, the vast majority of enterprise data centers
are still running on static, purpose-built systems that limit
flexibility and responsiveness. IT continues to be asked to
do more with less and hasn’t been able to truly deliver self-
service agility to their developers and business users.

What’s missing is the capability to optimize operations


across the enterprise data center for compute, storage, and
networking to increase efficiency and a cloud architecture
54% OF LINE-OF-BUSINESS
that provides the flexibility that businesses need to innovate.
EXECUTIVES (NON-IT
DEPARTMENTS) BELIEVE THAT
THE IT GROUP IS AN OBSTACLE
TO THEIR MISSION.¹

of workloads. In an SDI virtualized environment.


Orchestration: environment, orchestration Most importantly, the
• Track and report on
infrastructure and application
Moving to drives a high level of orchestration platform performance and availability.
intelligence across your data can accumulate learnings
Software-Defined center with a policy engine that can help make better
• Monitor system and
application health.
Infrastructure that uses platform and decisions in the future. • Monitor security threats and
application telemetry data adherence to security policies.
Orchestration is a major to improve infrastructure As data centers increasingly • Take effective actions.
evolutionary step forward utilization, accelerate serve a world of dynamic • Predict potential issues early.
in data centers moving application performance, applications, it is critical
to software-defined and provide central data to manage and deliver the
infrastructure (SDI), where center oversight. right levels of performance
applications and the and infrastructure capability
physical hardware they Orchestration platforms to different classes of
run on are separated and available in-market service offerings and
entirely controlled by today enable policy- associated service level
software. SDI delivers the driven management and agreements (SLAs).
full promise of automated, movement of cloud-native
scalable, and self-serve and legacy workloads With orchestration you can:
infrastructure control across resource pools, 64% OF THOSE INDICATING
within your data center, aligning service requests •C  onnect and automate THAT THEY WERE “EXTREMELY
enabling agile delivery of with available resources workflows for specific services. SUCCESSFUL” WITH CLOUD
cloud applications while and monitoring the • Manage configuration, capacity, WERE CURRENTLY USING
maximizing efficiency health of the physical and metering, and chargeback. CLOUD ORCHESTRATION.²

THE CASE FOR ORCHESTRATION OF CLOUD INFRASTRUCTURE 2


THIS WAY TO AGILITY AND GROWTH
Orchestration can reduce costs through intelligent resource
allocation across your compute, storage, and network
infrastructure. Because you are using software to automate
provisioning, management, and coordination of services
dynamically, you can often deploy services faster with fewer
human touch points.

Resources can be precisely provisioned—for example, with the


right capabilities in the right geographic areas. Orchestration
enabled across your software-defined infrastructure allows
you to set global policies that permit applications to engage
the right type of storage depending on access requirements,
provision the right networking levels to ensure network quality
of service needs, and ensure that CPU and memory resources
are allocated as needed based on application requirements.

Orchestration ultimately enables you to better manage


capacity by establishing the pathway to a cloud architecture
where you can control multiple workloads and institute policies
with required levels of security, performance, and governance
dynamically across on- and off-premises cloud resource pools.

30% ARE USING CLOUD


ORCHESTRATION AND 51%
PLAN TO WITHIN THE NEXT
12 MONTHS.²

lines of business as well as


Intel® Technology enterprise IT. Intel based
capabilities to manage
shared platform resources
run on trusted servers and
virtual machines whose
Enables platforms expose telemetry (such as L3 cache and system configurations have not been
data so automation tools
Orchestration and application software
memory) dynamically across
compute, networking,
altered. Integrity is verified
remotely using Intel TXT and
can make better decisions and storage. Trusted Platform Module
Intel’s silicon and software
about provisioning, (TPM) technology on Intel
innovation are the
resource allocation, service • I ntel® Trusted Execution Xeon processors.
foundation of a growing
tiering, and quality of Technology (Intel® TXT)
portfolio of solutions that are
service levels to drive data (available in the Intel Xeon •S
 nap is an open-source
helping data centers move
center efficiency. processors E5 and E7 families) platform telemetry
to SDI and dynamic, policy-
measures and verifies that framework that improves
based, on-demand services.
Key telemetry-related virtual servers boot into intelligent use of data center
technologies include: “known good states,” enabling infrastructure in cloud
Beyond workload-optimized
security automation and environments by enabling
silicon for SDI-based data
• I ntel® Resource Director compliance monitoring. exceptional data center
centers, we also offer key
Technology (Intel® RDT) scheduling and workload
platform technologies
(available in the Intel Xeon® • I ntel Cloud Integrity management through access
and software capabilities
processor E5-2600 v4 Technology (Intel CIT) works to underlying telemetry
that enhance the value
product family) provides with the OpenStack platform data and platform metrics.
of Intel® architecture to
orchestration and automation to ensure cloud applications

THE CASE FOR ORCHESTRATION OF CLOUD INFRASTRUCTURE 3


THE RIGHT ORCHESTRATION PLATFORM FOR YOUR ENTERPRISE
Orchestration changes the way you manage your data center infrastructure. IT staff familiar with managing purpose-built
systems must shift their mind-set to running a broader software-defined domain of multiple nodes and virtual environments.
This ultimately provides a self-service application development environment that delivers line-of-business benefit from
new, cloud-native applications. If you are just beginning to map out your SDI strategy as part of your broader enterprise
cloud initiatives, you have a growing landscape of solution approaches and choices.

Software Platform Options Deployment Options—


Do-It-Yourself or Integrated
Orchestration platforms are available today that run
on a broad array of hardware platforms. Intel has deep SDI Solutions
collaborations across hardware and software vendors,
ensuring optimization of a wide range of enterprise- Once you’ve chosen your cloud platform,
ready solutions. solution deployment presents two approaches
for moving forward.

If your IT environment supports VMware* or •D


 o-it-yourself (DIY) approach.
Microsoft* platforms, consider building your software- Build your SDI with discrete solutions available from
defined infrastructure with their orchestration the portfolio of various hardware and software
solutions. Your staff already has domain expertise in vendors, including open source. This gives you the
these platforms, which helps you mitigate risks by flexibility that comes with an environment that is not
implementing from a familiar source. defined by a single vendor.

If you’ve integrated open source into your data • Converged and hyperconverged solutions.
center strategy, the OpenStack* platform offers Leading vendors and new entrants in the market
rapidly maturing solutions to control data centers today offer a simplified approach compared to DIY
via open APIs and open interfaces, making it another solutions. With these solutions, you have the choice
compelling option for building SDI solutions. Key of all the software-defined components—compute,
vendors such as Red Hat* and Mirantis* provide storage, networking, and orchestration software—
OpenStack distributions optimized for Intel in a single box. Integration is preconfigured and
architecture features and enterprise capabilities. prevalidated, providing an “easy button” road to SDI
deployment today.
To get insight on deploying a software-defined
infrastructure and take advantage of orchestration
capabilities, Intel Builders offers solution briefs, proof-
of-concept publications, and reference architectures.
Intel Builders highlights the landscape of vendors and
Intel partners to help you move forward.

THE CASE FOR ORCHESTRATION OF CLOUD INFRASTRUCTURE 4


GETTING STARTED WITH SDI: A CHECKLIST FOR ITDMS
If you’ve decided to move to a software-defined infrastructure to enable cloud deployments, here’s a high-level checklist
to help you get started.

Understand the business problem. Identify a pilot project.


Work with your business users. What is the business Consider asking your DevOps group for a project they
problem that SDI can help solve? Are there specific want to run on an on-premises cloud that solves a key
workloads that support key business initiatives that business problem. Start small, build on key learnings,
need to be moved to your cloud infrastructure once and then expand your scope of cloud projects.
you have it up and running? Are there new projects
that would benefit from the agility of self-service
Choose your platform.
infrastructure?
Do you want to run your SDI on vendor-specific
solutions or open source? Understand what you will
Assess which specific infrastructure subcategory
gain and trade off with both those approaches before
within your data center will benefit most from SDI.
choosing your solutions.
Is your biggest IT need around lowering storage
costs? Or increasing efficiencies of legacy
Consider your deployment options.
applications? Or improving network security at its
core? Identifying your biggest pain point will help Do you want to build your SDI from a portfolio of
you decide where to implement SDI principles for discrete solutions from different vendors so you
maximum business impact. can customize to your needs, or opt for speed of
deployment via converged and hyperconverged
solutions?

Get educated.
• Read white papers, research studies, and analyst reports.
•F
 ind out what other companies learned on their
SDI journey—build on best practices and avoid
common pitfalls.
•L
 ook for reference architectures of proven SDI
solutions from Intel Builders members.

THE CASE FOR ORCHESTRATION OF CLOUD INFRASTRUCTURE 5


Take the first steps with Intel based
platforms for software-defined
infrastructure.
Learn more about building your own
enterprise cloud at intel.com/cloud.
Find solution briefs, proof-of-concept
publications, reference architectures, and
vendor connections at Intel Builders.

Share with Colleagues

¹ 2015 State of CIO Survey. CIO (January 5, 2015).


cio.com/article/2862760/cio-role/2015-state-of-the-cio.html#slide9
² Open Cloud Management and Orchestration 2015: Adoption and Experiences, Enterprise Management Associates (June 2015).
researchandmarkets.com/research/7pkh48/open_cloud

Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software,
or service activation. Performance varies depending on system configuration. No computer system can be absolutely
secure. Check with your system manufacturer or retailer, or learn more at intel.com.

Copyright © 2016 Intel Corporation. All rights reserved. Intel, the Intel logo, the Intel. Experience What’s Inside logo,
Intel. Experience What’s Inside, and Xeon are trademarks of Intel Corporation in the U.S. and/or other countries.
*Other names and brands may be claimed as the property of others.
0216/AS/MRM/PDF-USA 333994-001
Lenovo ThinkAgile CX Series For Your Next-Gen Data Center
Infrastructure
Lenovo ThinkAgile CX Series For Your Next-Gen Data Center
Infrastructure
Lenovo ThinkAgile CX Series For Your Next-Gen Data Center
Infrastructure




Lenovo ThinkAgile CX Series For Your Next-Gen Data Center
Infrastructure
Lenovo ThinkAgile CX Series For Your Next-Gen Data Center
Infrastructure


Lenovo ThinkAgile CX Series For Your Next-Gen Data Center
Infrastructure





Lenovo ThinkAgile CX Series For Your Next-Gen Data Center
Infrastructure



Lenovo ThinkAgile CX Series For Your Next-Gen Data Center
Infrastructure


Lenovo ThinkAgile CX Series For Your Next-Gen Data Center
Infrastructure
Lenovo ThinkAgile CX Series For Your Next-Gen Data Center
Infrastructure
Lenovo ThinkAgile CX Series For Your Next-Gen Data Center
Infrastructure
Taking the cloud to
your datacenter
Microsoft Azure Stack

Version 1.0

1/29/2016
CONTENTS
Cloud is a paradigm, not a place ....................................................................................................................................2
Cloud computing on your terms ....................................................................................................................................3
Microsoft Azure Stack vision .........................................................................................................................................4
Reinventing datacenter agility through cloud service delivery .....................................................................................4
Cloud service delivery with Azure Stack ........................................................................................................................5
Developer and IT Professional Experiences ..............................................................................................................5
Unified Application Model ........................................................................................................................................6
Extensible Service Framework ..................................................................................................................................6
Core Services .........................................................................................................................................................7
Foundational Services ...........................................................................................................................................7
Additional Services ................................................................................................................................................7
Cloud Infrastructure ..................................................................................................................................................8
Delivering Continuous Innovation .................................................................................................................................8
Taking the first steps with Azure in your datacenter.....................................................................................................9
Conclusion .....................................................................................................................................................................9
The Challenge The Goal The Solution
Many of the competitive Enable the power of the cloud A hybrid cloud platform that is truly
differentiators within organizations computing paradigm across your consistent with a major public cloud
(e.g. supply chain, distribution model, corporate assets. Unlock the agility provider unlocks this enterprise
customer data, operations, R&D, etc.) that comes from application value. This solution provides a
have been optimized by IT modernization, regardless of where continuous stream of innovation to
investments over many years. But that application runs. Do this all in a enables developers and IT to quickly
these assets sit behind a corporate way that doesn’t risk lack of address business needs without
firewall, may be regulated, and are functionality or stagnation from a limiting them by location.
very proprietary. For many of these solution ‘brokered’ between clouds
assets, participation in the cloud and doesn’t requires investment of
computing revolution remains limited resources in customizing the This is the promise of Microsoft Azure
elusive. All the while, new start-ups deployment rather than adding Stack.
without these “assets” accelerate business value.
their businesses quickly.

CLOUD IS A PARADIGM, NOT A PLACE


As you think of the IT investments throughout your company, you can see that over the course of time they have
become inextricably interwoven into the very foundation of your business. Almost every asset in your company,
from brick and mortar, to supply chains, to distribution networks, all the way through your personnel and
processes, have organically grown with your IT investments. That organic growth pattern has led to tremendous
value in the enterprise – and companies that do it well enjoy a strategic advantage over their competition. Most of
these important assets are kept behind the corporate firewall, where companies feel they can best control
performance, customization, and compliance.

As the realization of the disruption of cloud has spread from the server room to the board room, businesses and
technologists alike are looking for how new innovation born of the cloud can be used to modernize existing
investments so that they can continue to provide a strategic advantage to their company. Microsoft believes that a
well formed modernization strategy begins with a modern application platform. The modern application platform
today is being created in hyper-scale clouds. These vendors are setting the pace when it comes to innovation
which has resulted in most organizations considering a public cloud as they evaluate modernizing their application
portfolio.

Given its strategic significance, application modernization must be seen at an organizational level, not an
application by application level. But what of existing investments, assets and business realities? One-off cloud
solutions may lead to a modern application, but enterprise wide application modernization requires the coherent
and continuously evolving platform similar to those created by hyper-scale cloud providers. And this is
fundamentally the problem: existing business assets that cannot currently transition to public cloud risk being
alienated from agility and innovation promised by the cloud computing era.

We believe this is not the way things must be and that the next evolution of application modernization for
enterprise is enabled by leveraging the value of Microsoft Azure not just in our datacenters, but also in enterprise
and service provider datacenters. In this way, you can modernize your portfolio in a way aligned with your existing
business realities while not missing out on the best the hyper-scale cloud can offer – knowing that your
investments can take advantage of the power of a true hybrid cloud platform.
CLOUD COMPUTING ON YOUR TERMS
The disparate nature of public clouds from one another and from each unique private cloud means that for most
organizations, full participation in the cloud paradigm has not been possible. The discrepancies between
environments ultimately compromise the promise of innovation and agility that is fundamental to cloud
computing. As we work with our customers, we recognize three key areas that led us to see the need for a
consistent hybrid cloud platform.

There are important business and technical considerations for organizations that
should all be factored when developing an overall cloud strategy. Adoption of
public cloud services is constrained in regulated industries. Data sovereignty
requirements may prevent information from leaving national borders. Latency
between regions can be an issue when serving customers in remote locations.
Finally, there can simply be custom requirements that may be considered niche
by global public cloud providers, but are vital for an individual organization.

There is a need for new application patterns that enable sound business decisions and
flexibility. Hybrid application patterns that run across clouds, taking advantage of the
characteristics of both private and public cloud when appropriate. By designing
applications to the platform, mobility between clouds becomes a runtime decision
based on business rules. In this way, development work can be done once with the
original design of being able to run in the public or private cloud environments.
Similarly, when applications moves from dev-test to production, the code is the same
(all the way to the declarative approach to the infrastructure) and so functionality is
the same.

The next generation of application modernization is coming from the incredible


innovation that only a hyper-scale cloud provider can develop. Virtualization
alone is certainly not enough. Infrastructure as a Service is only the beginning of
the cloud transformation, not the end. However, a vendor who is only interested
in a public cloud strategy is not meeting the full set of challenges of their
customers as outlined above. By the same token, one-off deployments of highly
customized on-premises solutions lead to fragmented and dissimilar
technologies – preventing the development of the ecosystem as well as
investment in people and processes required to realize the promise of cloud.

We therefore believe that the notion of “cloud-first” needs to be fully enabled across on-premises and public
environments. For organizations looking for the agility and innovation of cloud computing in their datacenter,
Microsoft Azure Stack offers the only hybrid cloud platform that is truly consistent with a leading public cloud. Only
Microsoft can bring proven innovation – including higher level PaaS services – from hyper-scale datacenters to
enterprise-scale environments to flexibly meet customers’ business requirements. Ultimately, we want
organizations to be able to embrace the notion of cloud-first on their terms – every journey to cloud computing is
different and we want to support all of them. In this way, we believe customers can realize the value of the cloud
paradigm across their organization, regardless of location, and thereby refocus their resources on the investments
that competitively differentiate them in the marketplace.
MICROSOFT AZURE STACK VISION
Azure Stack extends the Azure vision by bringing the cloud model of computing to every datacenter. Azure Stack is
a new hybrid cloud platform product that enables organizations to deliver Azure services from their own
datacenter in a way that is consistent with Azure. In this way customers can focus on business innovation rather
than spend time building their own cloud computing platform.

Organizations can create these Azure services from datacenter resources - enabling developers and IT
professionals to quickly provision and scale services using the same self-service experience found in Azure. The
product also allows IT organizations to leverage the same management and automation tools used with Azure to
customize the service delivery experience to the business units they serve.

This all adds up to an environment in which application developers can maximize their productivity using a ‘write
once, deploy to Azure or Azure Stack’ approach, because the Azure APIs are consistent regardless of where the
resources are provisioned - Azure Stack is simply an extension of Azure. Part of the value of this approach is
bringing the large ecosystem of operating systems, frameworks, languages, tools, and applications we are building
in Azure to individual datacenters. Developers can create applications based on a variety of technologies such as
Windows, Linux, .NET, PHP, Ruby or Java that can be deployed and operated the same way on-premises or in
Microsoft Azure datacenters. They are also able to leverage the rich Azure ecosystem of templates, tools, and
applications to jumpstart their Azure Stack development and operational efforts.

REINVENTING DATACENTER AGILITY THROUGH CLOUD SERVICE DELIVERY


In order to understand how Azure Stack works, it’s important to take a step
back and think about the people and processes involved in running and
operating a cloud. The cloud era of computing more closely aligns the
economic principles of supply and demand with IT operations. There
are people, Cloud Consumers, such as developers, IT pros, DevOps,
who need resources to do their job, building and running business
applications, and there are organizations, Cloud Providers, who
provide those resources to these customers. In Azure, Microsoft acts as
the Cloud Provider and anyone who has a subscription to Azure is a
Cloud Consumer. Cloud Providers can be any company, as well.

Cloud Consumers need a variety of resources to accomplish their


business objectives. Those Cloud Resources are provided by Cloud
Services on demand. For example, some Cloud Consumers just want a
virtual machine. This is one kind of resource provided by Azure.
Specifically, it is a form of Infrastructure as a Service (IaaS). On the other
hand, if a Cloud Consumer does not need an entire virtual machine, but instead just a website resource, this
discrete resource can be supplied to them. This would be a type of resource delivered through Platform as a
Service (PaaS) – where the infrastructure is abstracted away and the Cloud Consumer need not worry about it.
Different types of customers use different kinds of resources and tools to accomplish their goals. Developers, IT,
and DevOps users are all welcome customers for Azure. The thing they all have in common is that they need
resources and are looking for someone to supply them.

Within the Cloud Provider, there is are two important roles of enabling Cloud Consumers to request and access
Cloud Resources. The first is the Cloud Service Administrators. Cloud Service Administrators both support existing
services and identify new Cloud Consumer needs that can be fulfilled through the cloud. For example, we have a
group of people in the Azure team responsible for creating new services to meet the needs of Cloud Consumers as
well as developing, delivering and managing the Cloud Service once it is deployed. The second role is the Cloud
Infrastructure Administrator that works to ensure sufficient supply of physical infrastructure to fulfill the request
from the Cloud Consumer that the Service Administrator has facilitated. Service Administrators and Infrastructure
Administrators work together to ensure that there is a very large pool of physical infrastructure so that new Cloud
Services developed by the Service Administrators have a place to run and fulfill customer Cloud Resource requests.
In offering Azure, Microsoft is responsible for both providing Service Administrators and Infrastructure
Administrators.

CLOUD SERVICE DELIVE RY WITH AZURE STACK


Since Azure Stack is fundamentally born from Azure, the design challenge was clear. In Azure, Microsoft operates
datacenters around the world each with tens of thousands of servers and a minimum “scale unit” of twenty racks.
When we talk about ‘hyper-scale’ in Azure, this is what we mean. But designing services to also run effectively in
the enterprise required creating a shared framework that allowed the services to be scaled and operated at
‘enterprise-scale’ – meaning significantly smaller scale point. The graphic below provides a simplified view of the
Azure Stack product architecture.

DEVELOPER AND IT PRO FESSIONAL EXPERIENCES


Developers and IT pros have an experience with Azure Stack that is consistent to that which they experience in
Azure. This is fundamentally made possible because the Azure Stack portal environment is the same code as
Azure. However, the real innovation of Azure Stack is the implementation of a consistent cloud API as Azure, so
there is a consistent developer experience across clouds. Simply connecting to a portal to choose from
preconfigured patterns is not enough; the definition of self-service has evolved to include programmatic access to
the cloud API for the creation, deployment and operations of workloads in a cloud.

A consistent API surface area between Azure and Azure Stack is the path to a set of experiences, tools, application
patterns, automation capabilities, deployment and configuration, and operations that work across clouds.

 Experiences: The first engagement with Azure and Azure Stack usually comes through the portal which
provides a web-accessible conduit into the system. The portal is a graphical expression of the cloud API.
 Tools: Cloud Consumers can use the tools they use in Azure and know they will work in Azure Stack. Cloud
Consumers can focus on solving business problems, rather than constant tooling and deployment
transitions.
 Application Patterns: Programmatic and abundant access to Cloud Services is changing the way that
applications are being designed, developed and operated. You can work with the resources in your
application as a group – mixing resources across IaaS and PaaS services.
 Automation Capabilities: Having a consistent API means that Cloud Consumers can invest in automating
development, deployment and operational activities knowing that they will not have to be rewritten to be
used with a cloud supplier that offers Azure services.
 Deployment and Configuration: Deploy, update or delete all of the resources for your application in a
single, coordinated operation. This can be done from the portal or programmatically through the SDK as
code.
 Operations: Templated deployments work for different environments such as testing, staging and
production. Role based access control, usage and audit capabilities are standardized across all cloud
resources in the deployment. Updates made to application resources can be performed in an incremental
and non-destructive manner.

These are all examples the breadth of impact enabled by this hybrid cloud platform. In each area we believe Azure
customers should be confident that their investments in people, processes and technologies will be fully
transferable between Azure and Azure Stack.

UNIFIED APPLICATION MODEL


As stated earlier, running and operating a cloud is fundamentally about Cloud Consumers requesting resources and
the cloud fulfilling them on-demand. In order to do this, a key component of Azure and, accordingly, Azure Stack is
Azure Resource Manager. Azure Resource Manager plays two important roles that enable Azure to operate at
hyper-scale. The first role is focused on enabling Cloud Consumers to create, organize and control their cloud
resources. The second is that it coordinates the fulfillment of that request in Azure datacenters. Each service that a
Cloud Consumer sees in the portal is one, or a combination of several, Resource Providers (RPs). Azure Compute,
for example, is a combination of what we call the Compute Resource Provider (CRP), Storage Resource Provider
(SRP) and Network Resource Provider (NRP).

Resource Requests: First, Azure Resource Manager is the cloud endpoint to which your Cloud Consumers will
connect. This is true for Azure Stack, just at it is true for Azure. Every installation of Azure Resource Manager is a
cloud endpoint and the Azure APIs are the set of commands that can be implemented against that single system.
Every command initiated by Cloud Consumers can be organized into groups that correspond to the individual
services with which they work. So the service that provides virtual machine resources has commands like “Create
VM” or “Delete VM”. Similarly, the service to create website resources has commands like “Create Website” or
“Scale Website”.

Resource Fulfillment: Second, Azure Resource Manager enables customers to describe their resource
requirements of the available Azure Services (such as VMs for IaaS or Websites for PaaS), and then actively places
them into the backend cloud infrastructure through the Resource Providers. As all cloud application resources are
really just software (e.g. Virtual Machines, NICs, Databases, etc.), Azure Resource Manager breaks the description
apart and manages them individually against the Azure Service to which they correspond, whether the resources
reside in our datacenter or in yours. As each resource type is fundamentally unique (databases are very different
from virtual machines), each individual Azure Resource Provider is responsible for integration with Azure Resource
Manager to achieve the fulfillment of the Cloud Consumer’s request.

EXTENSIBLE SERVICE F RAMEWORK


Microsoft Azure is composed of over fifty services today. As Microsoft continues to innovate and release new
Azure services, we needed a way to deploy and manage their corresponding Resource Providers throughout our
Azure datacenters. A key principle of operating a hyper-scale cloud is constant innovation - new capabilities, new
insights and new customer needs are the norm. Azure is fundamentally designed to enable the release of new
innovation on a regular and on-going cadence. With Azure Stack, the framework we use to install and publish new
capabilities is the same. The only difference is that we’ve tuned the size and automation requirements for
enterprise-scale vs. hyper-scale. This installation of new Resource Providers and Cloud Services happens in a
controlled and predictable manner all without disrupting the underlying infrastructure or the running services and
doesn’t require changes to the Cloud Consumer-facing experience and API.

CORE SERVICES
There are characteristics of Cloud Resources that are common across all types of IaaS and PaaS services and are
essential enterprise features of a hybrid cloud platform. These include business level information, such as a
subscription, which contain identity and quota information. Essentially this establishes financial responsibility for
billing purposes. Another core service is role-based access control, which handles authorization rules describing
what anyone can do with a particular resource. Additionally, all cloud resources need to be “tracked” for how
much they are used and who made changes, so usage and audit are common requirements. Azure Resource
Manager provides this support for all services, helping the Cloud Consumer focus on the resources they need to do
their work, such as virtual machines and databases.

FOUNDATIONAL SERVICES
Foundational Services include both IaaS and PaaS services. These services are important for two reasons. First,
they are available to be part of the Cloud Consumer-facing set of services in a given cloud. Second, they can be the
backbone to higher order PaaS services. The Foundational Services can create Cloud Resources such as VMs and
VM Extensions, VM images, Virtual Networks, Software Load Balancer, Gateway as well as Storage Disks and Blobs.
They are also used by Cloud Providers to establish the foundation of additional services as they are layered into
Azure Stack.

ADDITIONAL SERVICES
When a Cloud Provider chooses to add a service to their cloud, they are changing the API surface area of the cloud,
in effect, growing the capabilities they can offer to their Cloud Consumers. As we move forward, we will grow the
list of additional services that can be installed to Azure Stack. It is important to keep in mind that in Azure, in our
datacenters, services take dependencies on other services – they layer. So installing an additional service might
include also installing the services on which it depends. Each service in Azure is a candidate for being distributed
through Azure Stack and we will listen to customer input and consider technical feasibility in determining the
roadmap.

It is also worth noting that the extensible service framework serves not only to easily enable the onboarding of
incremental Azure services, but it also provides a framework through which 3 rd parties can also create resource
providers to instantiate completely new services. This is a whole new world of extensibility for ISVs, SIs and Service
Providers who want to create new services to offer to their customers.
Below is a table containing the services that we plan to have available when Azure Stack goes to general
availability.

Category Azure Stack Services at GA


Compute Virtual Machines (including extensions & availability sets), Service
Fabric*
Data & Storage Blobs, Tables, Queues*
Networking Virtual Network, Load Balancer, VPN Gateway
Mgmt. & Security Microsoft Azure Portal, Key Vault*
Web & Mobile App Service (Web Apps, Logic Apps*, Mobile Apps*, API Apps*)
Developer Services Azure SDK
* PLANNED TO BE IN PREVIEW AT AZURE STACK GENERAL AVAILABILITY

CLOUD INFRASTRUCTURE
As stated earlier, in Azure the Azure Resource Manager works with each service Resource Provider to fulfill the
customer request onto the back end cloud infrastructure we run in our datacenters. For the purposes of this paper
you can think of the Cloud Infrastructure as the underlying physical capacity of the cloud. Every resource
requested by a Cloud Consumer and fulfilled by Azure Resource Manager and the Resource Providers, ultimately
takes up capacity in the Cloud Infrastructure. As that capacity fills, Microsoft accepts the responsibility for growing
the back end capacity. When we grow the footprint of Azure capacity, we think in terms of scale units. For Azure,
our minimum scale unit is twenty racks of servers. We enjoy many benefits from that operational model, but one
of the most important is the purpose-built nature of the datacenters. They are not general purpose. They serve
only one purpose: to provide the backend capacity for Azure. That is the only way to run a hyper-scale cloud.
Additionally, in order to operate at hyper-scale, a software-defined approach is required across the basic
infrastructure components of Compute, Storage, and Network. A software-defined approach helps free us from
the “physics” of scaling. Each component of the Cloud Infrastructure can be automated, provisioned and managed.
In this way, we can create a standardized abstraction layer that helps us grow our cloud capacity.

A software-defined approach is equally important when thinking about the Cloud Infrastructure for Azure Stack.
For enterprise-scale, planning in terms of twenty rack increments is neither realistic nor desirable. Accordingly,
Azure Stack includes Cloud Infrastructure management technologies that are purpose built to supply Azure Service
capacity and to do it at enterprise-scale. The foundational IaaS RPs of Compute, Storage and Network (CRP, SRP
and NRP) are integrated with their corresponding platform at an infrastructure level: Hyper-V Clusters, Scale out
File Server, and Network Controller. The software installed to the physical systems is part of Azure Stack.

DELIVERING CONTINUOUS INNOVATION


Based on all of this context, it should be readily apparent that true cloud computing is fundamentally different
than a highly virtualized and automated traditional datacenter. With this understanding it is important to know
that in Azure we are continuously innovating and deploying new technologies to our datacenters. The services
listed in this paper are indicative of the beginning of a stream of continuous innovation and not a final list.
Additionally, in order to maintain the consistent API, Azure Stack operators should expect to implement updates
more frequently than with traditional software.
TAKING THE FIRST STEPS WITH AZURE IN YOUR DATACENTER
A key to success with the cloud model are the people and processes around it. Getting ready for Azure Stack may
require change from the way you currently administer applications and systems, so it is recommended you begin
evaluating your existing approach.

Consuming cloud services is something you can do today in Azure. You can use Azure today to learn what you can
do with the services that will be made available with the release of Azure Stack. A great place to start is by learning
how Azure Resource Manager works and trying some of the templates available for Azure Stack in both Azure and
Azure Stack.

You can also begin prioritizing the services you want to offer from your datacenter and what kind of scale and SLA
you will require. Download the single server deployment of Azure Stack and set it up. Once you do, you can
accomplish several things.

 First, work with your developers and IT teams to use the Azure SDK to develop and create automations,
deployments and configurations that work against Azure and Azure Stack.
 Second, work with your datacenter team to learn and understand the Service Administrator concepts in
the single server deployment of Azure Stack (e.g. Plans, Subscriptions, RBAC, etc.).
 Finally, all Service Administrators should begin familiarizing themselves with the Foundational Services in
Azure and Azure Stack that they will use to run the additional services.

CONCLUSION
The cloud computing paradigm is a transformational shift in technology that happens only once in a generation.
The potential for agility and innovation is clear, but the vast majority of organizations still grapple with questions of
how to fully enable cloud computing in their organization … on their terms. When focusing on how to most readily
empower developers and IT Pros to do their best work, working within the global ecosystem of a leading public
cloud provider is critical. This allows limited resources to be focused on innovation and not simply creating and
supporting an isolated private cloud environment. It also allows for a unified, declarative approach to dealing with
applications and the resources that power those applications. Microsoft is the only company taking the
innovations from their hyper-scale cloud environment and enabling enterprises and service providers to fully
participate in that ecosystem with a truly consistent hybrid cloud platform. With Microsoft Azure Stack,
organizations can stop asking how their applications and datacenters can move to the cloud and start thinking
about how they can also bring the cloud to their datacenter.

This document is provided “as-is”. Information and views expressed in this document, including URL and other Internet Web site references, may
change without notice. This document does not provide you with any legal rights to any intellectual property in any Microsoft product. You may
copy and use this document for your internal, reference purposes. Microsoft Azure, Microsoft Azure Stack, Windows Server, and Microsoft
System Center are trademarks of the Microsoft group of companies. All other trademarks are property of their respective owners.
EBOOK
FLASH TO CLOUD: WHY A
DATA FABRIC WILL HELP YOU
TAKE CONTROL OF YOUR DATA

NEXT
Introduction across multiple external clouds. In other words, there’s
a great need for data management designed for
Today’s data management challenges are being hybrid clouds. Hybrid clouds provide a wide range of
shaped by the demands of emerging third platform deployment options and a seemingly limitless pool of
technologies: social, mobile, analytics, and the cloud. compute, network, and storage resources, which is why
These new technologies have made current data most companies are quickly moving into the cloud.
management approaches for enterprise data centers
The Data Fabric enabled by NetApp is designed to
obsolete. At the same time, digital innovators that have
meet the needs of hybrid cloud environments. It
quickly adopted these technologies are exerting intense
enables our customers to respond and innovate more
pressure on established businesses to be more agile and
quickly because data is free to be accessed where it is
innovative while simultaneously reducing costs.
needed most. Companies can realize the full potential
In response to business demands for agility, innovation, of a hybrid cloud model and make the best decisions
and cost savings, IT organizations have turned to two for their business. And they retain total control of their
powerful new data storage options: on-premises all- data no matter where it resides.
flash systems and public cloud services. By 2017, an
With a Data Fabric, your hybrid cloud can operate
estimated 80% of IT organizations will have committed
like a superhighway on which data can move in either
to all-flash configurations for their primary workloads.
direction—from on-premises all-flash systems to the
Roughly the same number expect at least half of their
cloud and back again—with ease. Your data can go
IT workloads to run in the cloud by then.*
where you need it most, on demand, and always
This means new data management approaches are under your control.
needed to bridge on-premises, flash-based applications * “IDC FutureScape: Worldwide Cloud 2016 Predictions,” IDC Web Conference, November 2015;
“Justifying Investment in All-Flash Arrays,” August 2016, IDC #US41646416.
(and data) with third platform applications spread

BACK NEXT

2
The Hybrid Cloud Is migrating to cloud compute services. Every transition
required an entirely new storage ecosystem, driven by
Driving Big Changes the advances and realities of storage intelligence and
its corresponding compute resources.
in Storage
Companies are diverting compute and storage
Data storage systems have entered a new era. In the resources into public clouds to reduce the costs and
1990s, advanced software technologies brought about resources required by on-premises data centers. The
storage virtualization, with storage volumes abstracted favorable economics of the public cloud are driving the
from underlying physical devices. Now cloud services transition toward hybrid cloud services, which helps
have created a new model, one that combines bridge on-premises and cloud-based systems. Once
virtualization with economies of scale, the likes of again, storage architectures are evolving to meet the
which have never been seen. As a result, the cloud is needs of this new computing paradigm.
changing the face of IT.
Moving a virtual machine and its associated
applications between physical servers and cloud-
The Hybrid Cloud Is the New Ecosystem
hosted servers within a hybrid cloud is well understood
Each transition in data storage architecture has been and efficient, but importing and exporting application
driven by a corresponding shift in computer platforms, data are not as seamless. This data must be housed in
from mainframe to client/server, from client/server permanent storage, where it is not easily moved.
to virtualized server, and now virtualized servers are

BACK NEXT

3
Data Fabric Enabled by NetApp The challenges of moving data between data centers
and public cloud providers have been a significant
barrier to hybrid cloud adoption. A recent survey of
CIOs reported that 78% of enterprise IT organizations
view the ability to manage data across multiple
clouds as critical or very important, yet only 29% of
these organizations say their ability to do so is either
excellent or good.*

What’s needed is a way to manage, secure, protect,


share, and move data among different clouds, with
consistent, connected data management capabilities
that form a coherent, integrated, and compatible
system. Essentially, what’s needed is a Data Fabric that
joins on-premises storage systems—especially all flash,
which is often cloud ready—with numerous public
cloud storage systems.
*www.idgenterprise.com/report/idg-enterprise-cloud-computing-study-2014

BACK NEXT

4
A Data Fabric Eliminates easily moved back on premises or to another more
Hybrid Cloud Silos secure cloud environment.
To realize the vision of a Data Fabric, a method is • Better utilization of resources. Mature applications
needed to seamlessly control and manage data often require lots of data center space, power, and IT
between on-premises storage arrays (such as all flash) staff resources. A Data Fabric allows movement of
and the many storage endpoints within the hybrid selected applications to a public cloud infrastructure
cloud. Using a common structure and architecture, while freeing internal IT resources to focus on the
a Data Fabric provides efficient data transport, applications that deserve their attention.
software-defined management, and a consistent data • Cloud-based disaster recovery. A Data Fabric
architecture to allow data to move easily from on- enables economical multisite disaster recovery (DR).
premises systems to public clouds and back, as needed. Storage area network (SAN) replication between
Data Fabric endpoints allows for hot site DR with
A connected Data Fabric that enables data portability, very short recovery times and a cloud-based, cost-
control, and security has several benefits, including: effective DR option.
• Economic and data governance flexibility. All-flash and hybrid cloud models are poised to
If application development projects designed in become the dominant models in enterprise IT. A Data
the cloud are discontinued, a Data Fabric allows the Fabric that enables IT organizations to fully capitalize
server instances and accompanying data to simply on these fast-growing models will also enable a
be deleted. However, if the application development faster, more agile response to dynamic business
is successful and moves into production, it can be requirements.

BACK NEXT

5
Isolated Incompatible A Data Fabric
Connecting Clouds with
Data Silos a Consistent Data Fabric
To deliver the full potential of the cloud, storage
vendors need to provide a way to easily manage and
Public move data between on-premises flash storage and
Public
multiple clouds.
Colocation
DATA Colocation With a unified set of data services spanning multiple
clouds, it’s no longer necessary to keep applications
locked away in siloed clouds. Instead, entire
applications become portable as they (and their
Private Private data) move seamlessly across on-premises and off-
premises clouds. As requirements change, storage
and deployments change, too. At the same time,
companies maintain total control of their data, allowing
improved security, governance oversight, and visibility
into performance and usage. All of these things come
together with a Data Fabric.

BACK NEXT

6
The Three Data Fabric Enablers must be understood and translated where needed.
Only then can data in AWS S3 buckets be quickly
The three basic requirements of a Data Fabric are
copied to on-premises, block-based storage or data
consistent data format, software-defined data
contained in one cloud provider be instantly migrated
management, and fast and efficient data transport.
to another provider.
These characteristics deliver flexibility, mobility, and
control across vastly different cloud resources.
Software-defined data management
Consistent data format A Data Fabric requires a single control plane that
executes and monitors the flow of data throughout
Enterprise data storage is composed of a myriad
the Data Fabric, regardless of endpoint. This
of different interfaces, protocols, and formats. For
management interface must contain policy engines
example, FC, iSCSI, and FCoE are popular block storage
that enforce service-level and governance objectives
protocols, while NFS, pNFS, and CIFS (now known as
for data availability and protection and should alert
SMB) are the dominant file storage protocols. To add
administrators when a policy falls out of limits.
complication, cloud storage providers frequently use
object storage APIs based on the RESTful interface, The management interface should also apply storage
such as Amazon’s Web Services (AWS) S3 API and the efficiency policies such as deduplication, compression,
OpenStack community’s Swift protocol. and cloning where appropriate and provide a single
view of all data for greater visibility into data usage
To create a fluid and efficient Data Fabric, all of these
and performance.
different methods of data storage communication

BACK NEXT

7
Fast and efficient data transport A Data Fabric also enables the fast and efficient
transportation of application data as I/O resources
One barrier to hybrid cloud adoption is the slow
change throughout the application’s lifecycle.
data transfer rates offered by public cloud providers
Applications that begin as a pilot project can reside
(including hosted and hyperscalar services) compared
in a slower cloud-hosted environment and move to a
to those of on-premises storage arrays. According to
faster and more secure private cloud environment once
recent data, the fastest cloud storage providers using
in production. Then they can return to another hosted
SSD devices max out around 30,000 input/output
cloud when ready for sunsetting. In each case, the
operations per second (IOPS). Compare that to on-
application data is seamlessly transported across the
premises all-flash arrays, which can easily sustain
hybrid cloud through Data Fabric endpoints to keep
200,000 IOPS or more.
up with resource demands.
Storage efficiencies can significantly boost data
transport speeds within a Data Fabric. Backup and
replication, deduplication, and data compression based
on NetApp® Snapshot® technology all work to reduce
the actual number of bytes transferred.

BACK NEXT

8
Application Lifecycles
in the Data Fabric
The need to accelerate application development is driving organizations
to adopt agile development models—referred to as DevOps—that are
often implemented on cloud-based services. This makes sense, because
public clouds offer undeniable economic benefits, including:

• Pay-as-you-go pricing
• On-demand availability
• Rapid up or down scalability

However, a cloud-based application can become locked into a cloud


silo, where its data is the prisoner of one cloud provider or another.
To avoid this lock-in, a Data Fabric is useful to support application
lifecycle management across a hybrid cloud ecosystem.

BACK NEXT

9
Definition Application Lifecycle
• Requirements • Prioritization
An application lifecycle includes three core components:
• Validation • Release Plan
• Definition. The lifecycle begins with a definition of the application,
a corresponding business case, and an agreement to create and fund
the application.
• Development. The actual application writing and developing are not
Application a one-time event, but occur several times over the lifecycle as updates
Lifecycle and rewrites take place.
Operations Phases Development • Operations. The work required by IT to support and manage the
• Monitor • Iteration Plan application begins shortly before deployment and continues until the
• Support • Develop application is retired.
• Upgrade • Test All three phases must be well executed for an application to deliver its
full value to an organization. Fortunately, a Data Fabric can assist in all
three phases of the application lifecycle.

BACK NEXT

10
During Application Definition During Application Development
An organization’s best practices and standards usually In a hybrid cloud environment, the ability to tear
define programming governance, but governance down and rebuild a test environment in minutes—and
should extend to applications designed to operate do it over and over again—is an absolute necessity.
in the cloud. This cloud governance model includes
A Data Fabric with simple and secure cloud
guidelines for architecture and deployment, authorized
management helps automate this process so that
providers, security and protection, and cloud
application versions don’t require intervention or
onboarding and offboarding processes.
special approval from corporate IT to implement.
A governance model that includes a unified Data This shortens development cycles and improves
Fabric with seamless data transfer helps curtail the use collaboration among developer and operations teams.
of “shadow clouds” by business units. A Data Fabric
helps demonstrate the value of supported, established,
and secure cloud models.

BACK NEXT

11
During Application Operations in the face of catastrophic failure, thanks to cloud-
hosted backup and cloud-enabled disaster recovery
IT operations must decide whether to host the
(as enabled by the universal framework of the Data
application in the cloud or on the premises. This
Fabric enabled by NetApp).
decision is determined by:
• Data custody and security requirements Problem or Opportunity?
• Desired application performance
Lifecycle planning should begin long before an
• Cost analysis application is handed off to IT operations. The key
What should not happen is for cloud-born applications benefit of the hybrid cloud and a Data Fabric is the
to remain there forever because there is no convenient flexibility that comes from being able to use various
method to move them from the cloud to on-premises Data Fabric endpoints as enterprise applications are
equipment. Even applications built in the cloud should developed, launched, and subsequently retired. Now
be able to migrate to on-premises systems as needed. an IT organization can do so while still maintaining
complete control over data security and visibility. The
A Data Fabric can facilitate the rapid movement of benefits of this model are undeniable, and with proper
data between public cloud providers, as well as from planning it can help prevent application isolation while
public cloud environments to secure, high-performance making sure that organizations are ready for the cloud-
on-premises environments. It also makes sure that based future of data management.
application data remains continuously available, even

BACK NEXT

12
Conclusion Two powerful new data storage options are driving
the future of information technology: all-flash on-
A Data Fabric makes modern IT systems work. It premises systems and public cloud services. The Data
seamlessly connects different data management Fabric enabled by NetApp enables businesses and IT
environments across disparate clouds and on-premises organizations to get more from both infrastructures by
flash storage into a cohesive, integrated whole. leveraging the hybrid cloud to move their data where
they need it most, on demand and under control.
NetApp enables a Data Fabric across hybrid cloud
endpoints and allows an IT organization to embrace
To learn more about how the Data Fabric
the cloud on its own terms. With data storage as
enabled by NetApp, all-flash systems,
flexible and mobile as computing, a business can
and hybrid cloud solutions can help your
accelerate and expand its capabilities while also
company move forward more quickly,
reducing costs and improving data performance.
visit www.netapp.com.

BACK NEXT

13
© 2016 NetApp, Inc. All Rights Reserved. NETAPP, the NETAPP logo, and the marks listed at http://www.netapp.com/TM are trademarks of NetApp, Inc. Other company and product
names may be trademarks of their respective owners. NA-249-0916
BACK
NIMBLE LABS RESEARCH REPORT

Machine Learning Boosts


Application Uptime

Methodology
Nimble Storage analyzed From a range of IT
more than infrastructures across

12,000
instances of app problems
7,500+
customer environments

! using our Predictive Analytics Platform, the results show:

The App-Data Gap is Real


Applications power your
business and depend on
rapid, uninterrupted access
to data. Disrupted data
delivery makes business Business Applications
wait, creating the
“App-Data Gap”. App-Data Gap

Data Storage

“ Storage is normally the first suspect when identifying


the causes of the App-Data Gap, but the facts tell
a different story.

Key Findings

54%
of issues had nothing to do with
46%
of issues were related to
storage, resulting instead from storage, including hardware
config., interoperability and and software issues, and
other problems occasionally performance

Causes of the App-Data Gap

46% 26% 11% 8% 7%


Storage Configuration Interoperability Non-Storage Host,
Related Issues Issues Best Practices Compute,
Impacting VM
Performance

54% Non-Storage Related

Flash Alone Isn’t Enough

Infrastructure complexity makes it


difficult to pinpoint the real
problem, resulting in:
! • Extended downtime and frustration
• Inefficient use of IT resources and budget
• Long hours spent resolving cross-vendor issues

! And upgrading hardware


wouldn't solve 54% of
application issues

It’s Time to Close the App-Data Gap

“ Predictive Analytics based on machine learning boosts


application performance and eliminates downtime by
closing the App-Data Gap.

Predictive Analytics Platform is Key

• Identify issues early


and before users
• Minimize user
disruption
App-Data Gap

Business Applications

• Eliminate repetitive
issues

Data Storage
• Continually improve
performance

7 Capabilities to Look For When


You Evaluate Storage Solutions

Downtime Cross-stack
prediction application of analytics

Downtime automatic Analytics-driven


prevention tech support

Prescriptive Measured availability


resolution metrics

Rapid root-cause
analysis

Learn how to harness the power of


Machine Learning to boost productivity:

Read the Full Report:


“Can Machine Learning Prevent Application Downtime”

Visit Us
!

!
! !

Fkikvcn!Tgoqvg!Oqpkvqtkpi!cpf!Jqy!
kv!Ejcpigu!Fcvc!Egpvgt!Qrgtcvkqpu!
cpf!Ockpvgpcpeg!

Yjkvg!Rcrgt!348!
Tgxkukqp!1!

d{!Xkevqt!Cxgnct!
!!!

Gzgewvkxg!uwooct{!
Vqfc{”u!fcvc!egpvgt!rqygt!cpf!eqqnkpi!!
kphtcuvtwevwtg!jcu!tqwijn{!4!vkogu!oqtg!
fcvc!rqkpvu!0!pqvkhkecvkqpu!vjcp!kv!fkf!21!
{gctu!ciq/!!Vtcfkvkqpcn!fcvc!egpvgt!tgoqvg!
oqpkvqtkpi!ugtxkegu!jcxg!dggp!cxckncdng!
hqt!qxgt!21!{gctu!dwv!ygtg!pqv!fgukipgf!vq!
uwrrqtv!vjku!coqwpv!qh!fcvc!oqpkvqtkpi!cpf!
vjg!cuuqekcvgf!cnctou-!ngv!cnqpg!gzvtcev!
xcnwg!htqo!vjg!fcvc/!!Vjku!rcrgt!gzrnckpu!
jqy!ugxgp!vtgpfu!ctg!fghkpkpi!oqpkvqtkpi!
ugtxkeg!tgswktgogpvu!cpf!jqy!vjku!yknn!ngcf!
vq!kortqxgogpvu!kp!fcvc!egpvgt!qrgtcvkqpu!
cpf!ockpvgpcpeg/!

!
v# (?==?>#9>6A1BCAD3CDA5#3?A5#6?A# M\`c_\P\gl#[lcXeVbaiXeZXW# v# +<?21<#0>9"54#-1>175=5>C%##
1<<#F?A;<?14B#3A51C5B#1>#179<5#1>4# \aYeTfgehVgheX#WX_\iXef#ba#g[X# M\`c_\P\gl#[Tf#\`cebiXW#g[X#
!5G92<5#,/#?A71>9I1C9?>%##<XVThfX# VTcTU\_\g\Xf#bY#T#V_bhW#bcXeTg\aZ# XkcXe\XaVX#Ybe#TW`\a\fgeTgbef#Ul#
T__#jbe^_bTWf#eha#ba#T#Vbaf\fgXag)# `bWX_#j\g[#TWW\g\baT_#UXaX"gf#g[Tg#TeX# VXageT__l#fhccbeg\aZ#i\eghT_\mXW#
fVT_X*bhg#Vb``bW\gl#k53#c_TgYbe`# ha\dhX#gb#\gf#c_TgYbe`7 jbe^_bTWf#TaW#eXfbheVXf#hf\aZ#
j\g[#fbYgjTeX#gb#W\i\WX#hc#g[X# YT`\_\Te#`TaTZX`Xag#gbb_f#_\^X#
v# )1C1#56"395>3H#F9C8?DC#@5A6?A=1>35#
eXfbheVXf)#\aYeTfgehVgheX#VTa#XTf\_l# PGjTeX#i=XagXe+#<l#TUfgeTVg\aZ#
@5>1<C95B%#Ia#TiXeTZX)#M\`c_\P\gl#
UX#eXchecbfXW#Ybe#bg[Xe#jbe^_bTWf# cb_\V\Xf#Yeb`#g[X#haWXe_l\aZ#
Vhfgb`Xef#eXT_\mX#1-7.#WTgT#
gb#TVVb``bWTgX#V[TaZ\aZ#Uhf\aXff# [TeWjTeX)#M\`c_\P\gl#WX_\iXef#
XY"V\XaVl)#j[\_X#HLBOD@PHLF#
eXdh\eX`Xagf+##;f#Uhf\aXffXf#_bb^# f\`c_\"XW#bcXeTg\baf#j\g[#cb_\Vl*
Tcc_\VTg\ba#cXeYbe`TaVX+##N[\f#
gb#_XTea#g[ebhZ[#ge\T_*TaW*Xeebe# UTfXW#PG*VXage\V#`TaTZX`Xag#TaW#
ha\dhX#Vb`U\aTg\ba#\f#`TWX#
XkcXe\`XagTg\ba)#XaTU_\aZ#hfXef#gb# XY"V\Xag#PG#`bU\_\gl+
cbff\U_X#Ul#M\`c_\P\glsf#\aabiTg\iX#
Uhl#_Xff)#hfX#`beX)#TaW#eXfcbaW#
Z_bUT_#\a_\aX#WXWhc_\VTg\ba)# N[X#fgTgXW#UXaX"gf#bY#
`beX#dh\V^_l#gb#g[X#Uhf\aXff#\f#
Vb`ceXff\ba#TaW#bcg\`\mTg\ba)# [lcXeVbaiXeZXaVX)#Vb`U\aXW#j\g[#
XfcXV\T__l#iT_hTU_X+
j[\V[#WeT`Tg\VT__l#eXWhVX# M\`c_\P\glsf#ha\dhX#TccebTV[)#[b_W#
v# /85#129<9CH#C?#BC1AC#B=1<<#1>4# eXdh\eX`Xagf#Ybe#CIJM)#fgbeTZX# g[X#ceb`\fX#bY#WX_\iXe\aZ#Ta#TZ\_X#
B31<5#?DC#?>#45=1>4#9>CA?4D35B# VTcTV\gl)#TaW#aXgjbe^#UTaWj\Wg[+## \aYeTfgehVgheX#g[Tg#cebi\WXf#g[X#fT`X#
179<9CH%##NeTW\g\baT_#CN#eX_\Xf# YTibeTU_X#XVbab`\Vf#TaW#TZ\_\gl#bY#
v# 'D9<C$9>#41C1#@A?C53C9?>%##<l#gT^\aZ#
[XTi\_l#ba#YbeXVTfg\aZ#YhgheX# chU_\V#V_bhW#cebi\WXef)#j\g[bhg#g[X#
TWiTagTZX#bY#M\`c_\P\glsf#[\Z[_l#
WX`TaW#TVebff#`h_g\c_X#lXTef)# geTWXbYYf#bYgXa#TffbV\TgXW#j\g[#`bi\aZ#
XY"V\Xag#Uh\_g*\a#WTgT#cebgXVg\ba#
j[\V[#bYgXag\`Xf#\fasg#YXTf\U_X+## WTgT#bYY#f\gX+##
M\`c_\P\gl#Vhfgb`Xef#TeX#TU_X#gb#
BlcXeVbaiXeZXaVX#X_\`\aTgXf#
eXWhVX#UTV^hc#g\`Xf#f\Za\"VTag_lx <MROBD-"<HKNJHSHQV
g[X#aXXW#gb#ZhXff#Tcc_\VTg\ba#
bYgXa#Yeb`#[bhef#gb#fXVbaWfxTaW#
eXdh\eX`Xagf#\a#g[X#YhgheX+##M\`c_l#
WX_\iXe#LJIf#TaW#LNIf#g[Tg#jXeX#
fgTeg#j\g[#g[X#eXfbheVXf#aXXWXW#
ceXi\bhf_l#haTV[\XiTU_X+#N[\f#_XiX_#
abj#TaW#XkcTaW#gb#`Tff\iX#
bY#UTV^hc#TaW#eXVbiXel#cXeYbe`TaVX#
fVT_X#Tf#jbe^_bTWf#W\VgTgXxT__#
\f#g[X#eXfh_g#bY#\aV_hW\aZ#WTgT#
j\g[#`\a\`T_#XYYbeg#TaW#Vbfg)#
cebgXVg\ba#Tf#Ta#\agXZeT_#Vb`cbaXag#
TaW#ab#Tcc_\VTg\ba#cXeYbe`TaVX#
bY#g[X#TeV[\gXVgheX#Yeb`#g[X#fgTeg)#
WXZeTWTg\ba+#Ig[Xej\fX)#eXchecbfX#
eTg[Xe#g[Ta#UX\aZ#Ub_gXW#ba#TYgXe#g[X#
[lcXeVbaiXeZXW#abWXf#\Y#
YTVg+#
eXdh\eX`Xagf#jTaX+

#" (H=8JAE?"8E"*>>=;JAL="-OG=H;FEL=H?=E;="4JH8J=?O"MAJ@"4ADGCA6AJO""""".IIK="$
3=I=8H;@"+HFD",8HJE=H

+AL="/=OI"JF"(H=8JAE?"8E"*>>=;JAL="-OG=H;FEL=H?=E;="4JH8J=?O
N[X#gehX#iT_hX#cebcbf\g\ba#\a# v# JTefX#iXaWbefs#V_T\`f#fceXTW#TVebff# V[b\VXf#bY#fbYgjTeX#TaW#[TeWjTeX#
[lcXeVbaiXeZXW#flfgX`f#\f#bYgXa# g[X#"iX#WXV\f\ba#WXgXe`\aTagf# cTegaXef8#XVbab`\VT_8#TaW#Tf#[Ti\aZ#
`\ffXW#\a#XiT_hTg\baf#UXVThfX#bY# Ul#iT_\WTg\aZ#g[X\e#Tcc_\VTg\ba)# ceXfVe\cg\iX#dhT_\g\Xf#TaW#fcXV\"VTg\baf#
XkVXff\iX#[lcX#T`baZ#iXaWbef+#BXeXsf# hfX#TaW#UXaX"gf#gb#lbhe#fgeTgXZ\V# g[Tg#T_jTlf#`Tk\`\mXW#cXeYbe`TaVX)#
T#YeT`Xjbe^#g[Tg#C&I#_XTWXef#VTa#hfX# \aYeTfgehVgheX#bU]XVg\iXf+ hg\_\mTg\ba)#TiT\_TU\_\gl#TaW#bg[Xe#
g[Tg#Vhgf#g[ebhZ[#g[X#[lcX+ UXaX"gf+#QX#T_fb#bUfXeiXW#g[Tg#`Tal#
v# >X"aX)#jX\Z[#TaW#eTa^#XTV[#bY#
bY#bhe#V_\Xagf#jXeX#VbaYhfXW#TUbhg#
g[XfX#YTVgbef#TVVbeW\aZ#gb#lbhe#
/=O"+AE<AE?I [bj#UXfg#gb#`T^X#T#WXV\f\ba#g[Tg#Vbh_W#
aXXWf)#TaW#g[X\e#\`cbegTaVX#gb#
[TiX#_Tfg\aZ#cbf\g\iX#'be#aXZTg\iX(#
v# CN#_XTWXef#TeX#\ahaWTgXW#Ul#j[Tg# ceb]XVgf)#hfX#VTfXf)#\a*[bhfX#
XYYXVgf#\a#g[X\e#WTgT#VXagXef)#WXcXaW\aZ#
[lcXeVbaiXeZXaVX#`XTaf#TaW#g[X# gXV[a\VT_#XkcXeg\fX)#UhWZXg#TaW#
ba#g[X#j\fWb`#bY#g[X\e#fX_XVg\ba+#Mb#
cbgXag\T_#UXaX"gf)#bYgXa#Ul#[lcX# bU]XVg\iXf+
jX#WXiX_bcXW#T#YeT`Xjbe^#bY#"iX#
Yeb`#g[X#iXaWbef+
v# =b`U\aX#g[XfX#"iX#^Xl#Tgge\UhgXf# ^Xl#WXV\f\ba#WXgXe`\aTagf#g[Tg#VTa#UX#
v# N[X#\agXZeTgXW#flfgX`#x#VbageTel# j\g[#bg[Xe#gXV[a\VT_#XiT_hTg\ba# hfXW#\a#[lcXeVbaiXeZXaVX#\agXZeTgXW#
gb#\gf#\WXag\"VTg\ba#j\g[#f\`c_\V\gl# Ve\gXe\T#ba#cXeYbe`TaVX)#fVT_\aZ)# flfgX`#'B=CM(#L@Jf#gb#Tee\iX#Tg#g[X#
x#TVghT__l#chgf#T#[XTil#UheWXa#bY# eXf\_\XaVX)#fXVhe\gl#TaW#TiT\_TU\_\gl+ `bfg#Tccebce\TgX#fX_XVg\ba7#f\`c_\V\gl)#
Vb`c_Xk\gl#ba#CN#_XTWXef#j[b#`T^X# !Xk\U\_\gl)#fX_XVg\i\gl)#ceXfVe\cg\iX#TaW#
fgeTgXZ\V#\aYeTfgehVgheX#WXV\f\baf#Ybe# v# MX_XVg#g[X#UXfg#fhcc_\Xe#"aT_\fgf#Ul# XVbab`\V#'fXX#@\ZheX#.(+
CN#VbecbeTgX#fXei\VXf)#bcXeTg\baf# cebbYf#bY#VbaVXcg#g[Tg#WX_\iXe#ba#
Ubg[#lbhe#cXeYbe`TaVX#bU]XVg\iXf# BXeX#TeX#XkT`c_Xf#bY#g[X#VTWXaVX#
TaW#WXiX_bc`Xag+
TaW#g[X#"iX#WXgXe`\aTagf#`bfg# \a#g[X#_\gXeTgheX#jX#YbhaW#ba#gjb#
v# BlcXeVbaiXeZXaVX#XkcTaWf#g[X# \`cbegTag#gb#lbhe#CN#TaW#Uhf\aXff# bY#g[X#`bfg*bYg#V\gXW#B=CM#UXaX"gf7#
iTe\Xgl#bY#V[b\VXf#TiT\_TU_X#gb#CN# aXXWf+ f\`c_\V\gl#TaW#!Xk\U\_\gl+#.9=@<939CH&#
_XTWXef)#Uhg#`Tl#TWW#Vb`c_Xk\gl# '.(#Vb`U\aXf#T__#g[X#\aYeTfgehVgheX#
TaW#VbaYhf\ba#TUbhg#j[Tg#V_T\`f# UX_bj#g[X#[lcXei\fbe)#X_\`\aTg\aZ#
&E8COIAI
bY#f\`c_\V\gl#TaW#!Xk\U\_\gl#`XTa#gb# g[X#aXXW#Ybe#TUbhg#T#WbmXa#W\fVeXgX#
lbh#fcXV\"VT__l+ 2HAFHAJAP="8E<")=!E="+AL="/=O" \aYeTfgehVgheX#TaW#fbYgjTeX#cebWhVgf8#
-OG=H;FEL=H?=E;=")=J=HDAE8EJI"9O".5# '/(#f\`c_\"Xf#TaW#fgeXT`_\aXf#Vb``ba#
v# @\iX#^Xl#Tgge\UhgXf#VTa#XaTU_X# 'KIAE=II"19B=;JAL=I jbe^!bjf)#X_\`\aTg\aZ#g[X#aXXW#Ybe#
c_TaaXef#gb#Vhg#g[ebhZ[#g[X# W\fcTeTgX#`TaTZX`Xag#fb_hg\baf8#
QX#f\YgXW#g[ebhZ[#[haWeXWf#bY#QXU#
[lcX#TaW#`T^X#`beX*XYYXVg\iX# TaW#'0(#cbb_f#TaW#T__bVTgXf#fbYgjTeX*
cTZXf)#ceXfXagTg\baf)#Ue\X"aZf)#abgXf#
[lcXeVbaiXeZXaVX#WXV\f\baf+ WX"aXW#TaW#c[lf\VT_#eXfbheVXf#g[ebhZ[#
TaW#bg[Xe#`TgXe\T_f#Yeb`#iXaWbef)#
Vbafh_gTagf#TaW#V_\Xagf#biXe#g[X#cTfg# T#f\aZ_X)#hfXe*Ye\XaW_l#\agXeYTVX+#
3=;FDD=E<8JAFEI *<5G929<9CH&#'.(#cebi\WXf#g[X#!Xk\U\_\gl#
gjb#gb#g[eXX#lXTef+#IaX#eXTfba#Ybe#g[\f#
v# =eXTgX#T#Vb`cX__\aZ#fgeTgXZ\V# XYYbeg#jTf#g[Tg)#\aVeXTf\aZ_l#\a#g[\f# gb#cbb_#Vb``bW\gl#_bVT_#[TeW*W\f^#
[lcXeVbaiXeZXaVX#XiT_hTg\ba# VebjWXW#`Te^Xg#gXX`\aZ#j\g[#aXj# We\iX#'B>>(#fgbeTZX#j\g[#L;G#TaW,
Vb`cbfXW#bY#g[X#Yb__bj\aZ#"iX# XageTagf#TaW#V_T\`f#bY#fhcXe\be\gl)# be#!Tf[#TVebff#`h_g\c_X#fXeiXe#YTe`f8#
^Xl#WXV\f\ba#Tgge\UhgXf7#f\`c_\V\gl)# fXiXeT_#Tgge\UhgXf#eXcXTgXW_l#^Xcg# '/(#YXTgheXf#cTl*Tf*lbh*Zebj#ce\V\aZ#
!Xk\U\_\gl)#fX_XVg\i\gl)#ceXfVe\cg\iX# TccXTe\aZ+#MlfgX`f#jXeX#T_`bfg# g[Tg#bYYXef#`beX#!Xk\U\_\gl#gb#fVT_X#
TaW#XVbab`\V+ T_jTlf#WXV_TeXW#Tf#f\`c_X#TaW#XTfl#gb# g[X#Xai\eba`Xag#Tf#aXXWf#Zebj8#TaW#
hfX#TaW#WXc_bl8#[\Z[_l#!Xk\U_X#Ybe#T# '0(#XaTU_Xf#g[X#fT`X#flfgX`f#gb#TVg#Tf#
j\WX#eTaZX#bY#gTf^f8#bYYXe\aZ#`h_g\c_X# UTV^hc,W\fTfgXe#eXVbiXel#gTeZXgf#TaW#
eXfgbeX#jbe^_bTWf#j[Xa#aXXWXW+

.IIK="$""""(H=8JAE?"8E"*>>=;JAL="-OG=H;FEL=H?=E;="4JH8J=?O"MAJ@"4ADGCA6AJO"""""""""$
\agXebcXeTU\_\gl)#bY!bTW\aZ#TaW#g\Xe\aZ+# _XiX_f)#Vb`c_X`XagXW#Ul#e\V[)# Uhg)#\a#`bfg#VTfXf)#\f#iXel#W\Y"Vh_g#
N[Xl#`Tl#T_fb#Tffh`X#V[T`X_Xba# YhaVg\baT_#fbYgjTeX#g[Tg#TUfgeTVgf# gb#XkXVhgX+#Gbfg#beZTa\mTg\baf#_TV^#
cebcXeg\Xf#Tf#Tcc_\VTg\baf#V[TaZX+# TaW#`TaTZXf#Vb`cbaXagf#gb#ZXaXeTgX# T#fgTU_X#UTfX#bY#Vb`cTe\fba)#YTVgbe#
N[Xl#`Tl#UX#TU_X#gb#_\aXTe_l#fVT_X# `Tk\`h`#flfgX`#hg\_\mTg\ba+#N[X#^Xl# \a#g[X#`\ZeTg\ba#be#`bWXea\mTg\ba#
\aWXcXaWXag_l#Ul#Tff\ZaXW#eb_Xf#Ul# \f#TV[\Xi\aZ#ceXW\VgTU_X#cXeYbe`TaVX# Vbfgf)#TaW#`Tl#T_fb#V_T\`#g[Tg#g[X\e#
abWXf#Ybe#Vb`chgX)#fgbeTZX)#fXVhe\gl# TaW#TiT\_TU\_\gl)#j\g[#Ta#TU\_\gl#gb# XaZ\aXXe\aZ#cebjXff#T_eXTWl#Xk\fgf#gb#
TaW#aXgjbe^\aZ+ [TaW_X#T_`bfg#Talg[\aZ#lbh#VTa#g[ebj# VeXTgX#aXTe_l#g[X#fT`X#Xdh\iT_XaVX#Tf#
Tg#\g)#Tf#T#eXfh_g#bY#VTeXYh__l#XaZ\aXXeXW# g[X#cTV^TZXW#flfgX`f+#QX#[TiX#[XTeW#
4=C=;JALAJO WXf\Za+#N[X#iXaWbe#j\__#UXg#\gf#Uhf\aXff# g[X#_TggXe#TeZh`Xag#bYgXa#XabhZ[#gb#
`bWX_sf#fhVVXff#ba#cXeYbe`TaVX#Tf# Xfg\`TgX#g[Tg#iXaWbef#bY#T__#glcXf#bY#
Nb#UX#fX_XVg\iX#XkgXaWf#UXlbaW#UX\aZ#
\gf#W\fg\aZh\f[\aZ#geTWX`Te^+#N[X#CN# \agXZeTgXW#flfgX`f#`Tl)#\a#eXT_\gl)#ba_l#
!Xk\U_X)#j\g[#cebWhVg)#`bWh_X)#eTV^#
beZTa\mTg\ba)#\a#ghea)#j\__#TVVXcg#g[X# [TiX#Ta#TWWeXffTU_X#`Te^Xg#bY#2-%#
be#abWX#V[b\VXf8#fbYgjTeX#Thgb`Tg\ba#
ceXfVe\UXW#Vba"ZheTg\ba#Tf#_baZ#Tf# bY#g[X#gbgT_#flfgX`#`Te^Xg#g[ebhZ[#
TaW#`TaTZX`Xag8#[lcXei\fbe#fX_XVg\ba8#
\g#ehaf#\gf#Tcc_\VTg\baf#Tg#ceXW\VgTU_X# g[X#aXkg#g[eXX#lXTef+#;_gXeaTg\iX_l)#
VXageT_\mXW#TaW#W\fge\UhgXW#CN#fXei\VXf#
fXei\VX#_XiX_f#j\g[#[\Z[#VTcTV\gl# g[bfX#c_Taa\aZ#tZeXXa"X_Wfu#TaW#aXj#
`TaTZX`Xag)#XgV+#IY#[\Z[#\`cbegTaVX#
TaW#hg\_\mTg\ba#Ybe#CN#TaW#Uhf\aXff# WTgT#VXagXe#_bVTg\baf#TeX#`bg\iTgXW#
\f#j[Xg[Xe#T#flfgX`#fhcc_\Xe#ceXfXagf#
hfXe#aXXWf+#N[XfX#flfgX`f#`Tl# Ul#]h`c*fgTeg\aZ#TZ\_X#TaW#f\`c_Xe#
T#_bV^XW*Wbja#Tcc_\TaVX#j\g[#T#
fb`Xj[Tg#Vb`ceb`\fX#!Xk\U\_\gl#\a# \aYeTfgehVgheX#gb#`TaTZX#TaW#`T\agT\a#
"kXW#`Xah#bY#bcg\baf)#be#XaTU_Xf#
beWXe#gb#WX_\iXe#g[X#[\Z[Xe#ce\be\gl#bY# Tg#_bjXe#Vbfgf+
^Xl#cTegaXef[\cf#j\g[#\aabiTg\iX#
cXeYbe`TaVX#TaW#ceXW\VgTU_X#UX[Ti\be+#
[TeWjTeX#TaW#fbYgjTeX#iXaWbef#j[b# Cgsf#\`cbegTag#gb#abgX#g[Tg#lbh#aXXW#
;_gXeaTg\iX_l)#g[Xl#`Tl#cb\ag#gbjTeW#
TZeXX#gb#\agXZeTgX)#gXfg#TaW#iT_\WTgX# abg#eXfge\Vg#lbhefX_Y#gb#g[XfX#"iX#
T#aba*B=CM#\agXZeTgXW#TaW#VbaiXeZXW#
g[X\e#fb_hg\baf#ba#g[X#`T\a#fhcc_\Xesf# VTgXZbe\Xf#XkV_hf\iX_l+#Rbh#`Tl#"aW#
fb_hg\ba+
c_TgYbe`+#MX_XVg\i\gl#Tf#T#V[TeTVgXe\fg\V# T#fhUfXg#Tf#fTg\fYl\aZ#lbhe#XiT_hTg\ba#
`Tl#XiXa#Vba!\Vg#j\g[#f\`c_\V\gl)# aXXWf)#be#lbh#`Tl#j\f[#gb#TWW#gb#g[X`#
eXdh\e\aZ#g[X#CN*Uhf\aXff#c_Taa\aZ#TaW# *;FEFDA;
'Ybe#XkT`c_X)#Ta#TeZh`Xag#`\Z[g#UX#
eXi\Xj#Vb``\ggXX#gb#`T^X#geTWX*bYYf# t?Vbab`\Vu#\f#T#gXe`#g[Tg#f[bh_W#UX# `TWX#gb#\aV_hWX#TZ\_\gl(+#QX#ceXYXe#
Tf#cTeg#bY#f[beg*gXe`#TaW#_baZXe* WX"aXW#Ul#c_TaaXef#Tf#jX__#Tf#"aTaV\T_# \g#UX#fhUfh`XW#haWXe#f\`c_\V\gl#be#
gXe`#ZbT_f+#CY#CN#[TeWjTeX#f^\__f# TaW#cebVheX`Xag#`TaTZXef+#Mb`X# !Xk\U\_\gl)#be#Ubg[+#M\`c_\V\gl)#Ybe#
Xk\fg#Uhg#beV[XfgeTg\ba#`TaTZX`Xag# CN#beZTa\mTg\baf#be#cebVheX`Xag# XkT`c_X)#`Tl#WX_\iXe#YXTgheXf#g[Tg#
[Tf#UXXa#jXT^)#g[X#U\Tf#Vbh_W#UX# WXcTeg`Xagf#fXX^#gb#bcg\`\mX#VTc\gT_# Vbage\UhgX#gb#g[X#\aVeXTfXW#TZ\_\gl#bY#
f[\YgXW#gbjTeW#T#fgebaZ#[TeWjTeX, XkcXaW\gheX)#j[\_X#bg[Xef#fXX^#gb#f[\Yg# g[X#flfgX`+#Q[Tg#TeX#g[bfX#YXTgheXf9#
fbYgjTeX#cTegaXef[\c)#j[XeX#g[XfX#gjb# Vbfg#UheWXaf#gb#bcXeTg\aZ#XkcXaW\gheX# ;f#T#fXcTeTgX#VTgXZbel)#lbh#`Tl#jTag#
W\fV\c_\aXf#TeX#WX_\iXeXW#geTafcTeXag_l8# 'X+Z+)#V_bhW(+#;a#B=CM#WXV\f\ba#VTa# g[X#fhcc_\Xe#gb#Teg\Vh_TgX#g[X#ceXV\fX#
fTiX#jTfgXYh_#[bhef#bY#WXiX_bc`Xag)# YbVhf#ba#g[X#cbgXag\T_#gbgT_#Vbfg# YXTgheXf#g[Tg#WX_\iXe#\aVeXTfXW#_XiX_f#bY#
gXfg)#eha#TaW#eXi\fX#g\`X8#TaW#\aVeXTfX# bY#bjaXef[\c#TaW#bcXeTg\baT_#Vbfg# TZ\_\gl#Ybe#GbWX#/#bcXeTg\baf+#BTi\aZ#
hfXYh_#_\YX+ fTi\aZf#bY#Tcc_\TaVXf)#j\g[#eX_Tg\iX_l# T#fXcTeTgX#VTgXZbel#VTa#`T^X#g[X#
_\`\gXW#fVT_TU\_\gl+#Oag\_#g[XfX#flfgX`f# WX_\iXeTU_Xf#`beX#Vb`cX__\aZ#TaW#V_XTe+
2H=I;HAGJAL= `TgheX)#g[Xl#`Tl#abg#TccebTV[#abe#
X`h_TgX#g[X#fVT_TU\_\gl#TaW#`\ff\ba* .CSHBD-"2LPROD"QG@Q"SDLCMOP"DUNJ@HL"HL"
N[X#ceXfVe\cg\iX#TccebTV[#_XTaf#
Ve\g\VT_#Tgge\UhgXf#bY#VbaiXeZXW# CDNQG"QGDHO"@NNJHB@QHML"ME"QGD"!SD"IDV"
[XTi\_l#ba#`Xg\Vh_bhf#Vb`cbaXag#
\aYeTfgehVgheX#flfgX`f+#;#Vbfg#TaT_lf\f# CDQDOKHL@LQP"HL"QGDHO"PMJRQHMLP"QM"@CC"
fX_XVg\ba)#\agXZeTg\ba#TaW#gha\aZ)#Tg#
TaW#Vb`cTe\fba#j\g[#Xk\fg\aZ# NODBHPHML"@LC"CDNQG"QM"VMRO"CDBHPHML"
Ubg[#[TeWjTeX#TaW#flfgX`#fbYgjTeX#
\aYeTfgehVgheX#\f#T_jTlf#eXVb``XaWXW# ML"GMT"TDJJ"QGDV"K@QBG"VMRO"LDDCP$

.IIK="$""""(H=8JAE?"8E"*>>=;JAL="-OG=H;FEL=H?=E;="4JH8J=?O"MAJ@"4ADGCA6AJO"""""""""&
(FE;CKIAFEI%"3=MHAJ="J@="08HH8JAL= N[X#"iX#^Xl#WXgXe`\aTagf#TeX# N[X#"iX#WXgXe`\aTagf#jXeX#WXiX_bcXW#
CN#TaW#Uhf\aXff#_XTWXef#j[b#j\__#UX# WXf\ZaXW#gb#UeXT^#g[X#fcX__#bY#YTfgXe)# \a#eXfXTeV[#VbaWhVgXW#biXe#T#g[eXX*
eXfcbaf\U_X#Ybe#\`cbegTag#fXei\VX# _Xff#XkcXaf\iX#eXYeXf[#VlV_Xf#biXe#TaW# `bag[#cXe\bW#\agb#i\eghT__l#T__#
WX_\iXel#`hfg#gXT`#gbZXg[Xe#TaW# biXe#TZT\a#Ul#eXje\g\aZ#g[X#aTeeTg\iX+# iXaWbefs#Vb``ha\VTg\baf)#QXU*UTfXW#
Teg\Vh_TgX#g[X\e#\aW\i\WhT_#cXefcXVg\iXf)# Q[Xa#\g#Vb`Xf#g\`X#Ybe#T#eXYeXf[)# cebWhVg#WXfVe\cg\baf)#iXaWbe#Ue\X"aZ#
WXe\iXW#Yeb`#g[X#\`cbegTag#XiT_hTg\ba# CN#c_TaaXef)#XaZ\aXXef)#TeV[\gXVgf# WbVh`Xagf#TaW#ceXfXagTg\baf)#TaW#
Tgge\UhgXf#bY#g[X#"iX#^Xl#WXgXe`\aTagf+# TaW#Uhf\aXff#_XTWXef#f[bh_W#WXf\Za# \a*cXefba#W\fVhff\baf+#Ca#TWW\g\ba)#
<l#_Tl\aZ#ba#g[X#gTU_X#\ffhXf#fhV[#Tf7 T#aXj#aTeeTg\iX+#N[X#aXj#TVgbef# ah`Xebhf#V_\Xag#\agXeTVg\baf#eXiXT_XW#
\a#g[\f#aTeeTg\iX#j\__#Wba#W\YYXeXag# \agXeXfg#YTVgbef#TaW#`bg\iTg\baf#\a#
v# Q[Tg#TeX#g[X#cT\a#cb\agf#g[Tg#f_bj# V_bg[\aZ#Yeb`#g[X#fgTaWTeW#tWbhU_X# \aiXfg\ZTg\aZ#\agXZeTgXW#flfgX`f+
bhe#eXfcbafX#g\`Xf#Wbja9 g[X#cXeYbe`TaVX#Tg#0-%#_bjXe#Vbfg+u#
<MROBD-"3@OQLDO";DPD@OBG"8MQD"3%%','*+(#""
Hbj)#g[X#fXTeV[#f[bh_W#haVbiXe#eXT_#
v# Q[l#TeX#jX#YT\_\aZ#fb#bYgXa#TaW# 3DMOFD"6$"?DHPP#"',"9BQMADO"'%&)
aXXW*We\iXa#iT_hX#j[XeX#g[X#WXi\_#j\__#
gT^\aZ#fb#_baZ#gb#eXVbiXe9
UX#\a#g[X#WXgT\_f+
v# Q[l#WbXf#\g#gT^X#fb#`hV[#g\`X#gb#
fXg#hc#Vba"ZheTg\baf#XTV[#g\`X#T# *LA<=E;=
aXj#Tcc_\VTg\ba#\f#ceXfXagXW#gb#hf9 Mb`X#bY#g[X#ce\aV\c_Xf#\a#g[\f#eXfXTeV[#
v# Q[l#Wb#jX#[TiX#gb#UX#tc_h`UXefu# jXeX#gXfgXW#\a#T#ATegaXe#LXfXTeV[#
TaW#Uheebj#\agb#g[X#ahgf#TaW#Ub_gf# =\eV_X#_\iX#V[Tg#Ybeh`+#N[X#ATegaXe#
bY#g[X#flfgX`#gb#"aW#j[l#TaW#j[XeX# LXfXTeV[#=\eV_X#\f#T#`TaTZXW#cTaX_#
cXeYbe`TaVX#f_bjXW9 bY#CN#TaW#Uhf\aXff#_XTWXef+#;#fVeXXaXe#
dhXfg\baaT\eX#gb#XkT`\aX#VheeXag#
v# Q[l#Wb#bhe#L@Jf#YT\_#gb#WX_\iXe# cbf\g\baf#ba#[lcXeVbaiXeZXaVX#jTf#
j[Tg#jX#Tag\V\cTgXW9 fXag#gb#`X`UXef#\a#Hbeg[#;`Xe\VT#
ba#/#MXcgX`UXe#/-.2+#H\aXgl*baX#
v# Q[l#TeX#jX#XaZTZXW#fb#bYgXa#j\g[#
`X`UXef#eXfcbaWXW)#TaW#./#`X`UXef#
iXaWbef#WXal\aZ#g[X\e#eXfcbaf\U\_\gl#
jXag#ba#gb#cTeg\V\cTgX#\a#T#`bWXeTgXW#
Ybe#bhgTZXf#be#WXZeTWXW#bcXeTg\baf9
_\iX#V[Tg#ba#.4#MXcgX`UXe+#;__#_\iX#V[Tg#
cTeg\V\cTagf#jXeX#YT`\_\Te#j\g[#g[X#
gXe`#\a#g[X#XTe_l#fgTZXf#bY#W\fVhff\ba#
be#XiT_hTg\ba+#LXfXTeV[#jTf#WXiX_bcXW#
WXfVe\U\aZ#g[X#eXfh_gf#\a#WXgT\_+

'" (H=8JAE?"8E"*>>=;JAL="-OG=H;FEL=H?=E;="4JH8J=?O"MAJ@"4ADGCA6AJO""""".IIK="$
[lcXeVbaiXeZXW#\aYeTfgehVgheX#abWXf# Xai\eba`Xag+##M\`c_\P\gl#jbe^f#j\g[# Rbh#VTa#_XTea#`beX#TUbhg#[bj#
gb#TVVb``bWTgX#jbe^_bTWf#eXdh\e\aZ# Xk\fg\aZ#aTg\iX#PG#`TaTZX`Xag#gbb_f# g[X#jbe_Wsf#_XTW\aZ#Vb`cTa\Xf#eX_l#
TWW\g\baT_#Vb`chgX#VTcTV\gl+ fhV[#Tf#PGjTeX#i=XagXe8#beV[XfgeTg\ba# ba#M\`c_\P\gl#[lcXeVbaiXeZXW#
TaW#Thgb`Tg\ba#fbYgjTeX#_\^X#PGjTeX# \aYeTfgehVgheX#gb#geTafYbe`#g[X\e#WTgT#
;f#WX"aXW#Ul#ATegaXe)#Tg#g[X#VbeX#
iLXT_\mX#;hgb`Tg\ba)#=\fVb#O=M#>\eXVgbe)# VXagXef#Ul#i\f\g\aZ#M\`c_\P\gl+Vb`+#
bY#fX_XVg\i\gl#\f#g[X#VbaVXcg#bY#
TaW#IcXaMgTV^8#TaW#L?MN#;JCf+##FTfg_l)#
V[b\VX+##M\`c_\P\gl#XaTU_Xf#V[b\VX# <MROBD-"<HKNJH>HQV
M\`c_\P\gl#bYYXef#V_bhW#V[b\VX#Ybe#
ba#`h_g\c_X#W\`Xaf\baf#Yeb`#g[X#
beZTa\mTg\baf#\agXeXfgXW#\a#TeV[\gXVg\aZ#T#
Vhfgb`Xesf#V[b\VX#bY#k53#fXeiXe#
[lUe\W#V_bhW#fb_hg\ba+
c_TgYbe`#TaW#IM#i\eghT_\mTg\ba#
gXV[ab_bZl+##GTaTZX`Xag#V[b\VX# M\`c_\P\gl#f\Za\"VTag_l#\`cebiXf#
\f#_XYg#gb#g[X#TW`\a\fgeTgbe#f\aVX# XY"V\XaVl#TaW#TZ\_\gl)#TaW#`\g\ZTgXf#e\f^x
M\`c_\P\gl#WbXf#abg#\agebWhVX#Tal# T__#j[\_X#WX_\iXe\aZ#T#g[eXX*Yb_W#eXWhVg\ba#
aXj#`TaTZX`Xag#gbb_f#\agb#g[X# \a#gbgT_#Vbfg#bY#bjaXef[\c#'N=I(+#

=eXTg\aZ#Ta#?YYXVg\iX#BlcXeVbaiXeZXaVX#MgeTgXZl#j\g[#M\`c_\P\gl#\f#chU_\f[XW#Ul#M\`c_\P\gl+#?W\gbe\T_#VbagXag#fhcc_\XW#Ul#M\`c_\P\gl#\f#\aWXcXaWXag#bY#ATegaXe#TaT_lf\f+#;__#ATegaXe#eXfXTeV[#\f#hfXW#j\g[#
ATegaXesf#cXe`\ff\ba)#TaW#jTf#be\Z\aT__l#chU_\f[XW#Tf#cTeg#bY#ATegaXesf#flaW\VTgXW#eXfXTeV[#fXei\VX#TiT\_TU_X#gb#T__#Xag\g_XW#ATegaXe#V_\Xagf+#{#/-.2#ATegaXe)#CaV+#TaW,be#\gf#TY"_\TgXf+#;__#e\Z[gf#eXfXeiXW+##N[X#hfX#
bY#ATegaXe#eXfXTeV[#\a#g[\f#chU_\VTg\ba#WbXf#abg#\aW\VTgX#ATegaXesf#XaWbefX`Xag#bY#M\`c_\P\glsf#cebWhVgf#TaW,be#fgeTgXZ\Xf+#LXcebWhVg\ba#be#W\fge\Uhg\ba#bY#g[\f#chU_\VTg\ba#\a#Tal#Ybe`#j\g[bhg#ATegaXesf#ce\be#
je\ggXa#cXe`\ff\ba#\f#YbeU\WWXa+#N[X#\aYbe`Tg\ba#VbagT\aXW#[XeX\a#[Tf#UXXa#bUgT\aXW#Yeb`#fbheVXf#UX_\XiXW#gb#UX#eX_\TU_X+#ATegaXe#W\fV_T\`f#T__#jTeeTag\Xf#Tf#gb#g[X#TVVheTVl)#Vb`c_XgXaXff#be#TWXdhTVl#bY#fhV[#
\aYbe`Tg\ba+#N[X#bc\a\baf#XkceXffXW#[XeX\a#TeX#fhU]XVg#gb#V[TaZX#j\g[bhg#abg\VX+#;_g[bhZ[#ATegaXe#eXfXTeV[#`Tl#\aV_hWX#T#W\fVhff\ba#bY#eX_TgXW#_XZT_#\ffhXf)#ATegaXe#WbXf#abg#cebi\WX#_XZT_#TWi\VX#be#fXei\VXf#TaW#
\gf#eXfXTeV[#f[bh_W#abg#UX#VbafgehXW#be#hfXW#Tf#fhV[+#ATegaXe#\f#T#chU_\V#Vb`cTal)#TaW#\gf#f[TeX[b_WXef#`Tl#\aV_hWX#"e`f#TaW#YhaWf#g[Tg#[TiX#"aTaV\T_#\agXeXfgf#\a#Xag\g\Xf#VbiXeXW#\a#ATegaXe#eXfXTeV[+#ATegaXesf#
<bTeW#bY#>\eXVgbef#`Tl#\aV_hWX#fXa\be#`TaTZXef#bY#g[XfX#"e`f#be#YhaWf+#ATegaXe#eXfXTeV[#\f#cebWhVXW#\aWXcXaWXag_l#Ul#\gf#eXfXTeV[#beZTa\mTg\ba#j\g[bhg#\achg#be#\a!hXaVX#Yeb`#g[XfX#"e`f)#YhaWf#be#g[X\e#
`TaTZXef+#@be#Yheg[Xe#\aYbe`Tg\ba#ba#g[X#\aWXcXaWXaVX#TaW#\agXZe\gl#bY#ATegaXe#eXfXTeV[)#fXX#tAh\W\aZ#Je\aV\c_Xf#ba#CaWXcXaWXaVX#TaW#IU]XVg\i\glu#ba#\gf#jXUf\gX)##[ggc7,,jjj+ZTegaXe+Vb`,gXV[ab_bZl,TUbhg,
b`UhWf`Ta,b`USZh\WX/+]fc+

"!" (H=8JAE?"8E"*>>=;JAL="-OG=H;FEL=H?=E;="4JH8J=?O"MAJ@"4ADGCA6AJO""""".IIK="$
#
#
#
#

XEOMRK#ZMVXYEPM^EXMSR#JYVXLIV#
Jsv#}ievw/#gsqtermiw#lezi#wiir#xli#firijmxw#sj#gsrwspmhexmsr#xlvsykl#
zmvxyepm~exmsr1#[lmpi#zmvxyep#qeglmriw#lezi#qmxmkexih#mwwyiw#wygl#ew#tl}wmgep#wivziv#
wtve{p/#ts{iv#yweki/#erh#viwsyvgi#yrhivyxmpm~exmsr/#sxliv#xiglrspskmiw#mr#xli#
mrjvewxvygxyvi/#wygl#ew#wlevih#wxsveki#erh#rix{svomrk/#lezi#viqemrih#tl}wmgep#mr#
vikevhw#xs#eppsgexmsr#erh#qerekiqirx1#Xvehmxmsrepp}/#pevki#erh#i|tirwmzi#xvehmxmsrep#
wxsveki#evve}w#erh#tvstvmixev}#w{mxglmrk#erh#vsyxmrk#xiglrspskmiw#viuymvi#ehhmxmsrep#
wtegi#erh#wtigmep#qerekiqirx1#F}#ywmrk#e#ZQ{evi#wsjx{evi0hijmrih#hexe#girxiv/#}sy#
lezi#xli#sttsvxyrmx}#xs#xeoi#}syv#wxsveki#erh#rix{svomrk#xs#xli#ri|x#pizip#xlvsykl#
wxsveki#erh#rix{svo#zmvxyepm~exmsr1##
[mxl#ZQ{evi#zWtlivi#9/#ZQ{evi#fymphw#sr#mxw#jpekwlmt#zmvxyepm~exmsr#tpexjsvq#
jsv#gsqtyxi#f}#ehhmrk#jiexyviw#erh#zmvxyepm~exmsr#xiglrspskmiw#mr#xli#eview#sj#wxsveki#
erh#rix{svomrk1#[mxl#ZQ{evi#Zmvxyep#WER/#}sy#ger#gviexi#zmvxyep#wxsveki#tsspw#egvsww#
}syv#wivziv#levh{evi/#viqszmrk#xli#rigiwwmx}#jsv#xvehmxmsrep/#hihmgexih#wxsveki#evve}w/#
erh#epps{mrk#jsv#wmqtpiv#wxsveki#tvszmwmsrmrk1#Ehhmxmsrepp}/#{lir#vyrrmrk#sr#wyttsvxih#
w{mxgliw/#ZQ{evi#RW\#irefpiw#jpi|mfpi#zmvxyep#rix{svomrk#wivzmgiw#+i1k1/#jmvi{epp,#erh#
gsrrigxmzmx}#+i1k1/#wterrih#Pe}iv#5#rix{svow,#mqtpiqirxexmsrw#erh#glerkiw#{mxlsyx#
viuymvmrk#tl}wmgep#w{mxgl#vigsrjmkyvexmsr#sv#vi0gefpmrk1#RW\#pixw#}sy#vipsgexi#ZQw#egvsww#
Pe}iv#6#rix{svow#{mxlsyx#xli#riih#jsv#xvehmxmsrep#kyiwx#SW#vi0ehhviwwmrk#erh#ZQ#
hs{rxmqi1#
Ywmrk#ZQ{evi#xiglrspskmiw#egvsww#gsqtyxi/#wxsveki/#erh#rix{svomrk#xs#vyr#
}syv#wsjx{evi0hijmrih#hexegirxiv#lew#xli#tsxirxmep#xs#wtiih#yt#tvszmwmsrmrk#xmqi#erh#
iewi#xli#qerekiqirx#fyvhir#{lmpi#epws#vihygmrk#xli#gswxw#ewwsgmexih#{mxl#
yrhivyxmpm~ih#levh{evi1##
Mr#ehhmxmsr#xs#xliwi#firijmxw/#mx#mw#teveqsyrx#jsv#fywmriww0gvmxmgep#ettpmgexmsrw#xs#
vyr#yrmrxivvytxih1#Tsspmrk#sj#zmvxyep#viwsyvgiw#{mxl#zWtlivi#ex#xli#gsqtyxi/#wxsveki/#
erh#rix{svomrk#pe}ivw#epps{w#}sy#xs#qmkvexi#egxmzi#gvmxmgep#ettpmgexmsrw#jvsq#sri#wix#sj#
levh{evi#xs#ersxliv1#Er#ythexih#zmvxyepm~exmsr#wxvexik}#ger#qier#piww#hs{rxmqi#jsv#
}syv#gvmxmgep#ettpmgexmsrw/#erh#lipt#}sy#ezsmh#xli#ewwsgmexih#tsxirxmep#gexewxvstlmg#gswxw#
xs#}syv#fywmriww1##

E#XEPI#SJ#X[S#XIWXW#
Wmrkpi0wmxi#tivjsvqergi#gsrjmkyvexmsr#
Xli#UGX#wivziv#vego#
Xs#wls{#xlex#ZQ{evi#zWtlivi#9#mw#e#jypp}#getefpi#tpexjsvq#xlex#ger#i|gip#ex#
vyrrmrk#fywmriww0gvmxmgep#ettpmgexmsrw/#{i#jsgywih#sr#e#wmrkpi#wmqypexih#wmxi#jsv#syv#
tlewi#4#xiwxmrk1#Xlmw#wmrkpi#wmxi/#wmqypexmrk#e#tvmqev}#wmxi/#gsrwmwxih#sj#45#UGX#
UyerxeKvmh#H84F04Y#wivzivw/#ywih#xs#gviexi#xlvii#tshw#+gpywxivw,#ew#syxpmrih#mr#xli#

Fywmriww0gvmxmgep#ettpmgexmsrw#sr#ZQ{evi#zWtlivi#9/#ZQ{evi# E#Tvmrgmtpih#Xiglrspskmiw#vitsvx##5#
# Zmvxyep#WER/#erh#ZQ{evi#RW\#+Efvmhkih,#
#
#
#
#
#

ZQ{evi#Zepmhexih#Hiwmkr#+ZZH,#vijivirgi#evglmxigxyvi15#Rsx#epp#sj#xli#levh{evi#
viwsyvgiw#sj#xlmw#mrjvewxvygxyvi#{ivi#gsqqmxxih#hmvigxp}#xs#xli#xiwxw/#ew#wsqi#viwsyvgiw#
{ivi#tevxmepp}#eppsgexih#xs#qerekiqirx#erh#mrjvewxvygxyvi#xs#qship#xli#ZZH#hiwmkr#ew#
{ipp#ew#lmklpmklx#getefmpmxmiw#erh#jyxyvi#sttsvxyrmxmiw1#Xli#Ihki#tsh#gsrxemrih#ZQ{evi#
RW\#gsqtsrirxw#erh#ver#sr#xlvii#tl}wmgep#wivzivw1#Xli#Qerekiqirx#tsh#gsrxemrih#
zGirxiv#Wivziv#erh#sxliv#wyttsvxmrk#mrjvewxvygxyvi#gsqtsrirxw/#erh#ver#sr#xlvii#
tl}wmgep#wivzivw1#Xli#Gsqtyxi#tsh#gsrxemrmrk#syv#lswx#l}tivzmwsv#rshiw#erh#tvshygxmsr#
ZQ{evi#Zmvxyep#WER#ver#sr#xli#viqemrmrk#wm|#tl}wmgep#wivzivw1#Xli#Svegpi#45&#Hexefewi#
zmvxyep#qeglmriw#ver#sr#xli#Gsqtyxi#tsh1#Jsv#e#zmwyep#vitviwirxexmsr#sj#xli#tvmqev}#
wmxi/#wii#Jmkyvi#41#
Xli#Mrxip#WWHw#
[i#gsrjmkyvih#er#epp0jpewl#ZQ{evi#Zmvxyep#WER#sr#iegl#tsh1#Iegl#wivziv#
gsrxemrih#jmzi#Mrxip#WWHw>#sri#Mrxip#WWH#HG#W6;43#wivmiw#WWH#jsv#{vmxi#gegli#erh#jsyv#
Mrxip#WWH#HG#W6843#wivmiw#WWHw#jsv#xli#getegmx}#xmiv/#gsrjmkyvih#ew#e#wmrkpi#hmwo#kvsyt1#
Iegl#wivziv#mr#xli#Gsqtyxi#tsh#ver#x{s#ettpmgexmsr#ZQw#+sri#gsrjmkyvih#{mxl#97KF#sj#
VEQ?#xli#sxliv#{mxl#=9KF#sj#VEQ,#erh#iegl#ZQ#gsrxemrih#er#Svegpi#45&#Hexefewi#
mrwxeppexmsr#{mxl#e#533KF#hexefewi1##

X{s0wmxi#Pmzi#Qmkvexmsr#tivjsvqergi#gsrjmkyvexmsr#
[i#xlir#mhirxmgepp}#?KJBECQNA@ = OA?KJ@=NU OEPAX I=P?DEJC PDA LNEI=NU OEPA\O
gsrjmkyvexmsr/#{mxl#45#wivzivw#fvsoir#mrxs#xlvii#tshw/#epws#vyrrmrk#fywmriww0gvmxmgep#
Svegpi#Hexefewi#sr#45#ZQw1#Jsv#tlewi#5/#{i#izegyexih#45#ZQw#jvsq#xli#tvmqev}#wmxi#xs#
xli#wigsrhev}#wmxi#jsv#e#xsxep#sj#57#ZQw/#ywmrk#xli#ZQ{evi#Zmvxyep#WER#wxvixglih#gpywxiv#
jiexyvi#xs#wxvixgl#syv#wxsveki#viwsyvgiw#egvsww#fsxl#wmxiw1#Jsv#wtigmjmgw#sr#ZQ#wm~mrk#erh#
Svegpi#Hexefewi#gsrjmkyvexmsr/#wii#Ettirhm|#G#mr#xli#jypp#zivwmsr#sj#xlmw#vitsvx61#Jmkyvi#4#
mppywxvexiw#xli#wmqypexih#tvmqev}#erh#wigsrhev}#wmxiw1#

###########################################################
5
#lxxt>22{{{1zq{evi1gsq2wsjx{evi0hijmrih0hexegirxiv2zepmhexih0hiwmkrw1lxqp##
3 Xlmw#mw#er#efvmhkih#zivwmsr#sj#e#Tvmrgmtpih#Xiglrspskmiw#vitsvx1#Jsv#xli#jypp#zivwmsr/#zmwmx#
lxxt>22{{{1tvmrgmtpihxiglrspskmiw1gsq2zq{evi2ZQ{evicZepmhexihcHiwmkrc36491thj#

Fywmriww0gvmxmgep#ettpmgexmsrw#sr#ZQ{evi#zWtlivi#9/#ZQ{evi# E#Tvmrgmtpih#Xiglrspskmiw#vitsvx##6#
# Zmvxyep#WER/#erh#ZQ{evi#RW\#+Efvmhkih,#
#
#
#
#
#

#
Jmkyvi#4>#Xli#wmqypexih#tvmqev}#erh#wigsrhev}#wmxiw#{i#wmqypexih#jsv#xiwxmrk1#
#

Ls{#hmh#xli#wmrkpi0wmxi#ZQ{evi#Zepmhexih#Hiwmkrw#WHHG#tivjsvqC#
Fywmriww0gvmxmgep#ettpmgexmsrw#riih#wxvsrk/#wxieh}#tivjsvqergi#xs#oiit#fywmriww#
qszmrk1#Mr#syv#xiwxw/#{i#wix#syx#xs#wii#ls{#{ipp#xli#ZQ{evi#Zepmhexih#Hiwmkrw#WHHG#
wspyxmsr#tivjsvqih#yrhiv#xli#qswx#i|xviqi#hexefewi#pseh1#Xs#eggsqtpmwl#xlmw/#{i#
iqtps}ih#xli#Wmpp}#Pmxxpi#Svegpi#Hexefewi#Firglqevo#+WPSF,#xs#kirivexi#er#i|xviqip}#
liez}#{svopseh#sr#epp#x{ipzi#Svegpi#ZQw#mr#syv#wmqypexih#tvmqev}#wmxi1#[lmpi#ywivw#ger#
gsrjmkyvi#WPSF#xs#wmqypexi#e#qsvi#viepmwxmg#hexefewi#pseh/#{i#wix#mx#xs#xli#qe|mqyq#
ryqfiv#sj#ywivw#+45<,#{mxl#~ivs#xlmro#xmqi#xs#lmx#iegl#hexefewi#{mxl#qe|mqyq#
viuyiwxw/#jypp}#wxviwwmrk#xli#zmvxyep#viwsyvgiw#sj#iegl#ZQ1#[i#epws#wix#xli#{svopseh#xs#e#
qm|#sj#;8(#viehw#erh#58(#{vmxiw#xs#qmqmg#e#xverwegxmsrep#hexefewi#{svopseh1#Ew#Jmkyvi#
5#wls{w/#xli#ZQ{evi#Zepmhexih#Hiwmkr#tvmqev}#wmxi#eglmizih#e#wxvieq#sj#lmkl#MSTW/#
ettvseglmrk#533/333#hyvmrk#xli#wxieh}#wxexi#tsvxmsr#sj#xli#xiwx1#Xli#jmvwx#53#qmryxiw#
{ivi#e#{evq0yt#tivmsh1#Ew#wxviww#sr#xli#gegli#xmiv#mrgviewih#{mxl#{vmxi#egxmzmx}/#xli#
MSTW#tivjsvqergi#vieglih#e#wxieh}#wxexi#fix{iir#483/3330533/333#MSTW/#
vitviwirxexmzi#sj#xli#hexefewi#tivjsvqergi#e#fywmriww#gsyph#i|tigx#jsv#e#wmqmpev#
{svopseh#vyrrmrk#gsrxmrysywp}#ex#tieo#yxmpm~exmsr1#Xlmw#qierw#xlex#syv#wsjx{evi0hijmrih#

Fywmriww0gvmxmgep#ettpmgexmsrw#sr#ZQ{evi#zWtlivi#9/#ZQ{evi# E#Tvmrgmtpih#Xiglrspskmiw#vitsvx##7#
# Zmvxyep#WER/#erh#ZQ{evi#RW\#+Efvmhkih,#
#
#
#
#
#

hexegirxiv#wspyxmsr#tvszmhih#vipmefpi#tivjsvqergi#jsv#e#fywmriww0gvmxmgep#ettpmgexmsr#pmoi#
Svegpi#Hexefewi/#hiwtmxi#xli#lmkl#mrxirwmx}#sj#xli#{svopseh1##

#
Jmkyvi#5>#MSTW#xli#wsjx{evi0hijmrih#hexegirxiv#wspyxmsr#eglmizih#sziv#xli#wxieh}0wxexi#tivmsh#sj#syv#wmrkpi#wmxi#
tivjsvqergi#xiwx1#
#
Pexirg}#mw#e#gvmxmgep#qiewyvi#sj#ls{#{ipp#}syv#{svopseh#mw#vyrrmrk>#Xli#psrkiv#mx#
xeoiw#xli#wivziv#xs#viwtsrh/#xli#psrkiv#ettpmgexmsrw#erh#xlimv#viwtigxmzi#ywivw#qywx#
{emx1#Ew#Jmkyvi#6#wls{w/#mr#xli#wxieh}#wxexi#tivmsh#ejxiv#e#530qmryxi#{evqyt#tivmsh/#
syv#ZQ{evi#zWtlivi#Zepmhexih#Hiwmkr#WHHG#wspyxmsr#ver#{mxl#vipexmzip}#ps{#pexirgmiw#
gsrwmhivmrk#mx#{ew#lerhpmrk#wywxemrih#mrxirwmzi#{svopsehw1#Xlmw#wls{w#xlex#xlmw#ZQ{evi#
Zepmhexih#Hiwmkr#WHHG#wspyxmsr#ger#tvszmhi#uymgo#viwtsrwi#xmqiw#jsv#fywmriww0gvmxmgep#
ettpmgexmsrw#xs#lipt#oiit#gywxsqivw#wexmwjmih#izir#mr#er#i|xviqip}#liez}#M2S#wgirevms1#
Syv#tyvtswi#{ew#xs#hiqsrwxvexi#qe|mqyq#tivjsvqergi#hyvmrk#e#tieo#yxmpm~exmsr#
wgirevms?#x}tmgep#viep0{svph#irzmvsrqirxw#wlsyph#lezi#ps{iv#M2S#pexirgmiw1#

Fywmriww0gvmxmgep#ettpmgexmsrw#sr#ZQ{evi#zWtlivi#9/#ZQ{evi# E#Tvmrgmtpih#Xiglrspskmiw#vitsvx##8#
# Zmvxyep#WER/#erh#ZQ{evi#RW\#+Efvmhkih,#
#
#
#
#
#

#
Jmkyvi#6>#Pexirg}#jsv#xli#wsjx{evi0hijmrih#hexegirxiv#wspyxmsr#sziv#xli#wxieh}#wxexi#tivmsh#sj#e#wmrkpi0wmxi#tivjsvqergi#
xiwx1#
#
Jmkyvi#7#wls{w#xli#eziveki#lswx#GTY#yxmpm~exmsr#jsv#syv#wmrkpi0wmxi#tivjsvqergi#
xiwx1#Xlmw#ps{#eziveki#GTY#yxmpm~exmsr#wls{w#xlex#xli#GTY#{ew#rsx#xli#fsxxpirigo#erh#
mrhmgexiw#ehhmxmsrep#tvsgiwwmrk#viwsyvgiw#{syph#fi#ezempefpi#jsv#jyxyvi#kvs{xl1#

#
Jmkyvi#7>#GTY#yxmpm~exmsr#jsv#xli#wsjx{evi0hijmrih#hexegirxiv#wspyxmsr#sziv#xli#wxieh}#wxexi#tivmsh#sj#wmrkpi0wmxi#
tivjsvqergi#xiwx1#
# #

Fywmriww0gvmxmgep#ettpmgexmsrw#sr#ZQ{evi#zWtlivi#9/#ZQ{evi# E#Tvmrgmtpih#Xiglrspskmiw#vitsvx##9#
# Zmvxyep#WER/#erh#ZQ{evi#RW\#+Efvmhkih,#
#
#
#
#
#

Ls{#hmh#xli#x{s0wmxi#ZQ{evi#Zepmhexih#Hiwmkrw#WHHG#tivjsvq#hyvmrk#e#wmxi#izegyexmsrC#
Hiwtmxi#fiwx#tperrmrk/#tvsfpiqw#sggyv#sv#qemrxirergi#mw#viuymvih1#Sri#sj#xli#
qensv#firijmxw#sj#e#ZQ{evi#wsjx{evi0hijmrih#hexegirxiv#wspyxmsr#mw#xlex#}syv#
zmvxyepm~ih#wivzivw/#wxsveki/#erh#rix{svomrk#evi#iew}#xs#qszi#xs#sxliv#levh{evi#sv#wmxiw#
zme#e#wmrkpi#gsrwspi#xs#oiit#}syv#ettpmgexmsrw#yt1#Mr#xlmw#tlewi#sj#xli#wxyh}/#{i#ver#xli#
weqi#liez}#{svopseh#mr#x{s#tpegiw#_#sr#fsxl#xli#tvmqev}#erh#wigsrhev}#wmxi1#Xlir/#{i#
i|igyxih#e#tvmqev}#wmxi#izegyexmsr#{lmpi#gsrxmrysywp}#vyrrmrk#xli#liez}#hexefewi#
{svopsehw#sr#fsxl#wmxiw1#
Ew#Jmkyvi#8#wls{w/#{i#{ivi#efpi#xs#qmkvexi#erh#gsrwspmhexi#syv#gvmxmgep#
{svopsehw#ex#tieo#yxmpm~exmsr#jvsq#xli#tvmqev}#xs#xli#wigsrhev}#wmxi#wivzivw#mr#nywx#<#
qmryxiw#47#wigsrhw#{mxl#rs#ettpmgexmsr#hs{rxmqi#sv#vipsgexmrk#sj#hexe1#Jsv#fevi#qixep#
mrjvewxvygxyviw#sv#xlswi#{mxl#fewmg#zmvxyepm~exmsr#xiglrspskmiw/#mx#gsyph#xeoi#he}w#xs#
qmkvexi#epp#{svopsehw#xs#ersxliv#psgexmsr#sv#xs#wxevx#{svopsehw#ekemr#jvsq#fegoytw1#Xlmw#
{syph#viwypx#mr#gsrwmhivefpi#hs{rxmqi/#wsqixlmrk#xlex#xli#ZQ{evi#Zepmhexih#Hiwmkrw#
WHHG#ger#lipt#tvizirx1#[i#jsyrh#xlex#xli#ekmpmx}#sj#xli#ZQ{evi#Zepmhexih#Hiwmkrw#
WHHG#oitx#iziv}#{svopseh#vyrrmrk#{mxl#rs#hs{rxmqi/#{lmgl#qierw#xlex#}syv#fywmriww#
@KAOJ\P OQBBAN SDAJ EP\O PEIA PK IKRA SKNGHK=@O[ #
ZWER#tivjsvqergi#hyvmrk#xli#qmkvexmsr#{mrhs{#{ew#75;/546#MSTW/#gsqfmrmrk#
xli#eziveki#MSTW#sj#iegl#wmxi/#{mxl#eziveki#{vmxi#pexirgmiw#sj#491<5qw#erh#vieh#
pexirgmiw#sj#8139qw#egvsww#fsxl#wmxiw1#[lir#xli#{svopsehw#{ivi#epp#gsrwspmhexih#xs#e#
wmrkpi#wmxi/#tivjsvqergi#{ew#ex#e#wxieh}#eziveki#sj#5;8/65=#xsxep#MSTW1#Xli#xsxep#
SKNGHK=@\O Kyxtyx#higviewih#figeywi#mx#pswx#lepj#xli#ryqfiv#sj#tl}wmgep#viwsyvgiw#
ezempefpi#xs#mx1#Qswx#mqtsvxerxp}/#xli#wigsrhev}#wmxi/#rs{#lswxmrk#epp#57#liez}#
{svopsehw/#i|tivmirgih#~ivs#hs{rxmqi/#hmh#rsx#viuymvi#e#tl}wmgep#hexe#qmkvexmsr/#erh#
gvswwih#Pe}iv#6#rix{svow#{mxlsyx#xli#riih#jsv#kyiwx#SW#vi0ehhviwwmrk1#
#

Fywmriww0gvmxmgep#ettpmgexmsrw#sr#ZQ{evi#zWtlivi#9/#ZQ{evi# E#Tvmrgmtpih#Xiglrspskmiw#vitsvx##;#
# Zmvxyep#WER/#erh#ZQ{evi#RW\#+Efvmhkih,#
#
#
#
#
#

#
Jmkyvi#9>#Eziveki#GTY#yxmpm~exmsr#ex#fsxl#xli#tvmqev}#erh#wigsrhev}#wmxiw1#
#

GSRGPYWMSR#
Qszmrk#xs#xli#zmvxyepm~ih/#wsjx{evi0hijmrih#hexegirxiv#ger#sjjiv#viep#firijmxw#xs#
PK@=U\O svkerm~exmsrw1#Ew#syv#xiwxmrk#wls{ih/#zmvxyepm~mrk#fywmriww0gvmxmgep#ettpmgexmsrw#
{mxl#ZQ{evi#zWtlivi/#ZQ{evi#Zmvxyep#WER/#erh#ZQ{evi#RW\#rsx#srp}#hipmzivih#
vipmefpi#tivjsvqergi#mr#e#tieo#yxmpm~exmsr#wgirevms/#fyx#epws#hipmzivih#fywmriww#
gsrxmrymx}#hyvmrk#erh#ejxiv#e#wmqypexih#wmxi#izegyexmsr1##
Ywmrk#xlmw#ZQ{evi#Zepmhexih#Hiwmkr#{mxl#UGX#levh{evi#erh#Mrxip#WWHw/#{i#
hiqsrwxvexih#e#zmvxyepm~ih#gvmxmgep#Svegpi#Hexefewi#ettpmgexmsr#irzmvsrqirx#hipmzivmrk#
wxvsrk#tivjsvqergi/#izir#{lir#yrhiv#i|xviqi#hyviww1##
Vigskrm~mrk#xlex#qer}#svkerm~exmsrw#lezi#qypxmtpi#wmxiw/#{i#epws#tvszih#xlex#
syv#irzmvsrqirx#tivjsvqih#vipmefp}#yrhiv#e#wmxi#izegyexmsr#wgirevms/#qmkvexmrk#xli#
tvmqev}#wmxi#ZQw#xs#xli#wigsrhev}#mr#nywx#sziv#imklx#qmryxiw#{mxl#rs#hs{rxmqi1#
[mxl#xliwi#jiexyviw#erh#wxvirkxlw/#xli#ZQ{evi#Zepmhexih#Hiwmkr#WHHG#mw#e#
tvszir#wspyxmsr#xlex#epps{w#jsv#ijjmgmirx#hitps}qirx#sj#gsqtsrirxw#erh#ger#lipt#
mqtvszi#xli#vipmefmpmx}/#jpi|mfmpmx}/#erh#qsfmpmx}#sj#}syv#qypxm0wmxi#irzmvsrqirx1#
#
#
Xlmw#mw#er#efvmhkih#zivwmsr#sj#e#Tvmrgmtpih#Xiglrspskmiw#vitsvx1#Jsv#xli#jypp#zivwmsr/#zmwmx#
lxxt>22{{{1tvmrgmtpihxiglrspskmiw1gsq2zq{evi2ZQ{evicZepmhexihcHiwmkrc36491thj#

Fywmriww0gvmxmgep#ettpmgexmsrw#sr#ZQ{evi#zWtlivi#9/#ZQ{evi# E#Tvmrgmtpih#Xiglrspskmiw#vitsvx##=#
# Zmvxyep#WER/#erh#ZQ{evi#RW\#+Efvmhkih,#
#
#
#
#
#

EFSYX#TVMRGMTPIH#XIGLRSPSKMIW#
[i#tvszmhi#mrhywxv}0piehmrk#xiglrspsk}#ewwiwwqirx#erh#jegx0fewih#
qevoixmrk#wivzmgiw1#[i#fvmrk#xs#iziv}#ewwmkrqirx#i|xirwmzi#i|tivmirgi#
{mxl#erh#i|tivxmwi#mr#epp#ewtigxw#sj#xiglrspsk}#xiwxmrk#erh#erep}wmw/#jvsq#
# viwievglmrk#ri{#xiglrspskmiw/#xs#hizipstmrk#ri{#qixlshspskmiw/#xs#
#
xiwxmrk#{mxl#i|mwxmrk#erh#ri{#xsspw1##
Tvmrgmtpih#Xiglrspskmiw/#Mrg1##
433;#Wpexiv#Vseh/#Wymxi#633# #
Hyvleq/#RG/#5;;36# [lir#xli#ewwiwwqirx#mw#gsqtpixi/#{i#ors{#ls{#xs#tviwirx#xli#viwypxw#xs#
{{{1tvmrgmtpihxiglrspskmiw1gsq#
e#fvseh#verki#sj#xevkix#eyhmirgiw1#[i#tvszmhi#syv#gpmirxw#{mxl#xli#
qexivmepw#xli}#riih/#jvsq#qevoix0jsgywih#hexe#xs#ywi#mr#xlimv#s{r#
gsppexivep#xs#gywxsq#wepiw#emhw/#wygl#ew#xiwx#vitsvxw/#tivjsvqergi#
ewwiwwqirxw/#erh#{lmxi#tetivw1#Iziv}#hsgyqirx#vijpigxw#xli#viwypxw#sj#
syv#xvywxih#mrhitirhirx#erep}wmw1##
#
8A LNKRE@A ?QOPKIEVA@ OANRE?AO PD=P BK?QO KJ KQN ?HEAJPO\ EJ@ERE@Q=H
viuymviqirxw1#[lixliv#xli#xiglrspsk}#mrzspziw#levh{evi/#wsjx{evi/#[if#
wmxiw/#sv#wivzmgiw/#{i#sjjiv#xli#i|tivmirgi/#i|tivxmwi/#erh#xsspw#xs#lipt#syv#
gpmirxw#ewwiww#ls{#mx#{mpp#jevi#ekemrwx#mxw#gsqtixmxmsr/#mxw#tivjsvqergi/#mxw#
qevoix#viehmriww/#erh#mxw#uyepmx}#erh#vipmefmpmx}1#
#
Syv#jsyrhivw/#Qevo#P1#Zer#Reqi#erh#Fmpp#Gexglmrkw/#lezi#{svoih#
xskixliv#mr#xiglrspsk}#ewwiwwqirx#jsv#sziv#53#}ievw1#Ew#nsyvrepmwxw/#xli}#
tyfpmwlih#sziv#e#xlsywerh#evxmgpiw#sr#e#{mhi#evve}#sj#xiglrspsk}#wyfnigxw1#
Xli}#gviexih#erh#pih#xli#^mjj0Hezmw#Firglqevo#Stivexmsr/#{lmgl#
hizipstih#wygl#mrhywxv}0wxerheN@ >AJ?DI=NGO =O <EBB %=REO .A@E=\O
[mrwxsri#erh#[ifFirgl1#Xli}#jsyrhih#erh#pih#iXiwxmrk#Pefw/#erh#ejxiv#
xli#eguymwmxmsr#sj#xlex#gsqter}#f}#Pmsrfvmhki#Xiglrspskmiw#{ivi#xli#
#
lieh#erh#GXS#sj#ZivmXiwx1##

Tvmrgmtpih#Xiglrspskmiw#mw#e#vikmwxivih#xvehiqevo#sj#Tvmrgmtpih#Xiglrspskmiw/#Mrg1#
Epp#sxliv#tvshygx#reqiw#evi#xli#xvehiqevow#sj#xlimv#viwtigxmzi#s{rivw1#

Hmwgpemqiv#sj#[evverxmiw?#Pmqmxexmsr#sj#Pmefmpmx}>#
TVMRGMTPIH#XIGLRSPSKMIW/#MRG1#LEW#QEHI#VIEWSREFPI#IJJSVXW#XS#IRWYVI#XLI#EGGYVEG]#ERH#ZEPMHMX]#SJ#MXW#XIWXMRK/#LS[IZIV/#
TVMRGMTPIH#XIGLRSPSKMIW/#MRG1#WTIGMJMGEPP]#HMWGPEMQW#ER]#[EVVERX]/#I\TVIWWIH#SV#MQTPMIH/#VIPEXMRK#XS#XLI#XIWX#VIWYPXW#ERH#
EREP]WMW/#XLIMV#EGGYVEG]/#GSQTPIXIRIWW#SV#UYEPMX]/#MRGPYHMRK#ER]#MQTPMIH#[EVVERX]#SJ#JMXRIWW#JSV#ER]#TEVXMGYPEV#TYVTSWI1#
EPP#TIVWSRW#SV#IRXMXMIW#VIP]MRK#SR#XLI#VIWYPXW#SJ#ER]#XIWXMRK#HS#WS#EX#XLIMV#S[R#VMWO/#ERH#EKVII#XLEX#TVMRGMTPIH#
XIGLRSPSKMIW/#MRG1/#MXW#IQTPS]IIW#ERH#MXW#WYFGSRXVEGXSVW#WLEPP#LEZI#RS#PMEFMPMX]#[LEXWSIZIV#JVSQ#ER]#GPEMQ#SJ#PSWW#SV#
HEQEKI#SR#EGGSYRX#SJ#ER]#EPPIKIH#IVVSV#SV#HIJIGX#MR#ER]#XIWXMRK#TVSGIHYVI#SV#VIWYPX1##
#
MR#RS#IZIRX#WLEPP#TVMRGMTPIH#XIGLRSPSKMIW/#MRG1#FI#PMEFPI#JSV#MRHMVIGX/#WTIGMEP/#MRGMHIRXEP/#SV#GSRWIUYIRXMEP#HEQEKIW#MR#
GSRRIGXMSR#[MXL#MXW#XIWXMRK/#IZIR#MJ#EHZMWIH#SJ#XLI#TSWWMFMPMX]#SJ#WYGL#HEQEKIW1#MR#RS#IZIRX#WLEPP#TVMRGMTPIH#XIGLRSPSKMIW/#
*/$[\4 -*"#*-*5;X */$-6%*/( '03 %*3&$5 %"."(&4X &9$&&% 5)& ".06/54 1"*% */ $0//&$5*0/ 8*5) 13*/$*1-&% 5&$)/0-0(*&4X */$[\4
5&45*/([ $6450.&3\4 40-& "/% &9$-64*7& 3&.&%*&4 "3& "4 4&5 '035) )&3&*/[#

Fywmriww0gvmxmgep#ettpmgexmsrw#sr#ZQ{evi#zWtlivi#9/#ZQ{evi# E#Tvmrgmtpih#Xiglrspskmiw#vitsvx##43#
# Zmvxyep#WER/#erh#ZQ{evi#RW\#+Efvmhkih,#
#
WHITE PAPER SEPTEMBER 2016

Taming the
Long Tail
in Hadoop:
Storage Tiering
for Big Data
This white paper explores how tiered storage can
help organizations store and analyze more data to
glean deeper insights—at a realistic total cost of
ownership (TCO).
WHITE PAPER TAMING THE LONG TAIL IN HADOOP: STORAGE TIERING FOR BIG DATA

Contents Introduction
Introduction ……………………………………… 1 More data analyzed over longer time horizons can lead to
breakthrough insights. However, keeping everything on traditional
Hadoop Then and Now …………………… 1 Hadoop clusters leads to massive server sprawl and high operating
expenses (OpEx). For this reason, older data sets are often
Keeping and Analyzing More discarded or sometimes archived to tape, then reloaded later
with Tiered Storage …………………………… 1 for analytics. But in this scenario, reloading all archived data for
processing is an impossible task if the Hadoop cluster has not
Hadoop Workflow:
grown with the size of the archive. So what ends up happening is
Load, Analytics and Store ……………… 2
most historical analytics tasks just take a few data sets from a few
Load Data …………………………………………… 2 tapes, hoping that the analytics will be “good enough.”
Analytics to Get Insights …………………… 3 This white paper explores how tiered storage can help organizations
Store in Active Archive……………………… 3 store and analyze more data in order to glean deeper insights—at a
realistic total cost of ownership (TCO).
Total Cost of Ownership …………………… 4
Sprawling DataNodes, CapEx Hadoop Then and Now
and OpEx Implications ……………………… 4
Hadoop was developed over 10 years ago, when performance
Seven Year View of was limited to small capacity HDDs that rarely exceeded 2TB.
Data Retention……………………………………… 4 Furthermore, every DataNode had to match its counterpart in
number and type of HDD. Recent, game-changing innovations like
High Fidelity Big Data with
HelioSeal® technology HDDs and enterprise-grade SSDs enable
a Superior TCO ……………………………… 5 management of the three major stages of Hadoop workloads (Load,
Analytics and Store) using different types of drives, replication
factors and data management practices.
With the traditional symmetrical Hadoop design, server sprawl is
nearly instantaneous. To make matters worse, OpEx to power all of
the DataNodes grows in direct proportion to the daily load.

By leveraging Hadoop’s new storage policies, DataNodes can


be optimized with SSD and HDD tiers for different workload
requirements like Throughput and IOPS, while increasing the HDD
sizes to store more data for longer periods of time.

By adding tiering, organizations can now


process more data and keep it longer,
at a lower cost than ever before.

Keeping and Analyzing More


with Tiered Storage
Comparing a traditional symmetrical design to a tiered storage
architecture shows just how far the Hadoop community has come
in 10 years. By adding tiering, organizations can now process more
data and keep it longer, at a lower cost than ever before. Consider
the following architectures in Figure 1.

1
WHITE PAPER TAMING THE LONG TAIL IN HADOOP: STORAGE TIERING FOR BIG DATA

Figure 2: Comparison of complex analytics function

cost. The SSD tier accelerates Load and Analytics jobs.


Processing data faster means users can increase the
size and frequency of workloads as well. Figure 2 above
provides an example of how SSDs in a Hybrid DataNode
can help process workloads much faster than the
traditional design.

Hadoop Workflow:
Load, Analytics and Store
Given the rapid innovation in storage technologies,
the three major stages of Hadoop workloads
(Load, Analytics and Store) can be managed using
different types of drives, replication factors and data
management practices.
Figure 1: Traditional design vs. tiered storage architecture
Load Data
By striping across many DataNodes, Hadoop can
The “After” architecture uses NVMe SSDs on
achieve massively parallel loading. Lots of striped HDDs
NameNodes to accelerate all command and control
deliver excellent throughput, and adding DataNodes
functions.
scales out in a near-linear fashion. But an all-HDD
Hadoop now defines an SSD tier for DataNodes, where DataNode is impacted by mechanical seek time which
SSDs can be allocated for Load and Analytics functions can increase a large data set’s load time as much as
using a replication factor of 1. Depending on the 40% based on the number of drives, type of drive and
workload, different mixes of SSDs and HDDs can be block size of the chunks to be loaded.
selected to optimize performance, capacity and cost.
While SSDs are known to shine in I/O centric
There are commands to move data from DataNodes to applications, they also deliver for sequential write/read
Archive. This means that the number of DataNodes can workloads. A single NVM Express™ (NVMe) compliant
be sized to match processing requirements, not storage SSD can deliver 1,600MB/s of sequential write
requirements. throughput, or the equivalent 12 HDDs. For reads, it
takes 21 HDDs to match the 3GB/s performance of an
With the Archive function, a 3-way stripe of the data
NVMe SSD.
is no longer needed. Erasure Coding and HDD-based
Object Storage, like HGST Active Archive have a data
survivability rate of 15 9’s so the replication factor can Now consider rapid ingest of daily content that can be
be set to 1. processed, then moved to HDD tiers, freeing up space
to do other processing against a variety of batch loads
An SSD/HDD Hybrid DataNode could do more work
on that same SSD tier simultaneously. Suddenly your
while using less space, power and compute resources.
infrastructure becomes much more efficient to load
With high capacity HDDs, the tiering approach allows
and process without compromise for data volume,
for large data sets to be kept online longer, for a lower
variety or velocity.

2
WHITE PAPER TAMING THE LONG TAIL IN HADOOP: STORAGE TIERING FOR BIG DATA

Figure 3: Number of drives needed to load 250TB in Figure 4: 300TB complex sort time comparison
24 hours or less

With a batch-based load with Hadoop 2.7 and later, Figure 4 illustrates how much time it would take to run
users can now load to an SSD tier with Replication a complex Analytics job of 300TB with a traditional
Factor_1, run processing and then de-stage to HDDs or design versus tiered storage.
Active Archive at a later time.
Deeper Insights with Active Archives
Figure 3 above shows an example of how efficient an
Organizations like Yahoo!, eBay, Facebook and Google
SSD is for loading data.
collectively have Exabytes of data in Hadoop clusters.
Analytics to Get Insights But what about the rest of the world that wish to
manage Big Data in Hadoop but on a fixed budget?
Harnessing the parallel nature of multiple DataNodes to
process large amounts of data, analytics like Spark or By leveraging storage policies, I/O can be managed on
MapReduce are used to find the needle in the haystack. SSDs and HDD sizes can be increased to keep more
These tools generate two kinds of I/O patterns—large data for longer periods of time.
sequential reads and writes as well as small random The Long Tail4 is a theory on how large amounts of
reads and writes. data, kept for long periods of time, analyzed with
The small block size and random nature of shuffle is an Bayesian classification can lead to disruptive marketing
ideal use-case for SSDs. According to Cloudera1, SSDs with lower competition, higher prices/margins with
can accelerate Analytics workloads by as much as 70%. lower marketing costs. This is a perfect problem for the
Another study by Kang, Koo, et. al.2 shows SSDs deliver Hadoop framework as users must iterate against a large
5-6x faster Sort and Join functions. population to find the specific insight.

If one had an unlimited budget, hundreds of machines If the project is budget constrained and all of the
could be used to process 300TB in under an hour. In available data cannot be stored, the accuracy of Long
fact, Google has proven that 1PB can be sorted in six Tail analytics becomes questionable.
hours using 4,000 computers equipped with 48,000 The larger the sample size (p) of a given population (n),
HDDs3. the better the odds of accurate results. Margin of Error
However, this is not a reality for most. With many describes the odds.
different input sources and data types all needing to At 1TB/day over 7 years and with a budget of $250K/yr,
be compared, SSDs can have a tremendous impact 2TB DataNodes will only be able to store 30% of the
for the majority. A single NVMe SSD can perform statistical sample. 4TB DataNodes will store 50% of the
740,000 random 4K reads, or ten times the IOPS that sample, compared to 100% with Active Archive and the
can be achieved from 60 fully-loaded 2TB HDD-based storage tiering approach. At 100% of sample and
DataNodes.

1 http://blog.cloudera.com/blog/2014/03/the-truth-about-mapreduce-performance-on-ssds/
2 http://www.sersc.org/journals/IJCA/vol6_no1/17.pdf
3 https://googleblog.blogspot.com/2008/11/sorting-1pb-with-mapreduce.html
4 http://www.longtail.com/about.html

3
WHITE PAPER TAMING THE LONG TAIL IN HADOOP: STORAGE TIERING FOR BIG DATA

(perhaps six months of data). This would cap the total


DataNodes (and associated OpEx) moving the long-
term storage cost to a more appropriate platform.

Figure 6: Code snippet to request an HDFS directory be


placed in archive storage.

What is An Archive Tier?


An archive tier is simply an administrator specified
HDFS directory as shown in Figure 6. Batch or
Figure 5: Statistical accuracy (margin of error) based on volume interactive jobs move their completed results to the
of data stored directory and the HDFS mover process will
transparently migrate data from SSD or HDD to the
archive.
100% of population, there would theoretically be a zero
margin of error, yielding the deepest and most
accurate insights (see Figure 5). Seven Year View of Data Retention
Data retention for seven years matches up with many
Total Cost of Ownership regulatory requirements. Table 1 below outlines how
much data is accumulated each year, with 1TB of
Sprawling DataNodes, CapEx daily Load and where the replication factor to Archive
and OpEx Implications = 1 unless otherwise specified.
A small daily Load of 1TB yields 1.2PB a year with Year Data Kept on Moved to
Hadoop striping and space allocation. Without proper Growth (TB) DataNodes (TB) Archive (TB)
planning, DataNodes quickly sprawl into an operational
1 1,095 1,095 --
nightmare.
2 2,190 2,190 --
Assuming a $164/device list price for 2TB HDDs5,
storage spend is $98,400 in Year 1. Adding an average 3 3,285 3,285 --
“commodity” server for every 10 HDDs at $5,925/unit6 4 4,380 3,285 1,095
yields another $355,500 for capital expenses (CapEx)
5 5,475 3,285 2,190
for a total spend of $453,900 in Year 1.
6 6,570 3,285 3,285
Alongside CapEx, there is also OpEx to consider, which
includes maintenance, admin costs, power and cooling. 7 7,665 3,285 4,380
Bottom line, the combined cost of CapEx and OpEx Table 1: 1TB of Daily Load Accumulated Over Seven Years
means really big data problems.
“Store” is the most expensive component of the
Hadoop framework. Adding more DataNodes as the
volume of data increases may not be the best solution
for budget-conscious Hadoop customers. But with
SSDs that can deliver very high I/O and multi-core
servers that can handle CPU problems, a symmetrical
architecture may not be necessary.

Hadoop’s Archive Tier has a set of commands to move


data from DataNodes to the Archive. This means that
we could potentially limit the number of DataNodes to
the size of the maximum expected Analytics function

5 Source = New Egg HGST 7K4000 2TB as of 2/6/2016


6 Source = Dell.com (R730 w/ 12 HDD Slots, Dual Xeon 2.6GHz with 18GB RAM) Figure 7: TCO analysis comparison
as of 2/6/2016
4
WHITE PAPER TAMING THE LONG TAIL IN HADOOP: STORAGE TIERING FOR BIG DATA

Depending on the Hadoop system architecture that is used, the TCO to store and use all of this data varies
significantly, as shown in Figure 7.

Across the board for both CapEx and OpEx, a Hybrid tiered storage system with SSDs and HDDs minimizes
out-of-pocket costs, delivering to users a smarter storage solution for Big Data.

High Fidelity Big Data with


a Superior TCO
While the concept of storage tiering is not new, it brings new life to struggling Big Data deployments and is the
blueprint for future implementations. This is especially true for organizations with fixed budgets. As illustrated in
the TCO analysis in Figure 7, the ability for a Hybrid DataNode to Load and run Analytics faster with fewer
nodes, and the addition of an Active Archive for online storage is far superior.

The numbers are magnified by the following factors:

1. Fewer DataNodes in the traditional symmetrical approach yields less throughput and IOPS
2. SSDs in the Hybrid speed up certain tasks and complete many others faster than multiple HDDs
3. A significant portion of the traditional approach is dedicated to capacity storage, leaving less room for
throughput and IOPS

Load, Analytics and Store deliver amazing results when properly designed. By applying storage tiering, users
can optimize ingest from many sources, stage it for MapReduce or Spark, move it to Active Archive for rapid
access and keep more data than ever before, enabling faster, more accurate analytics and greater business
value.

Learn more about smarter storage solutions from HGST at www.hgst.com

© 2016 HGST, Inc. 3403 Yerba Buena Road, San Jose, CA 95135 USA. Produced in the United States 4/16. All rights reserved.

References in this publication to HGST’s products, programs or services do not imply that HGST intends to make these available in all countries in which it operates. Information is true as
of the date of publication and is subject to change and does not constitute a warranty. Individual performance may vary. Users are responsible for evaluating their own requirements.

WPXX-Hadoop-Storage-Tiering-US-EN-01

Das könnte Ihnen auch gefallen