Sie sind auf Seite 1von 56

Reference Architecture

HPE Reference Architecture for


VMware Cloud Foundation on HPE
Synergy
Reference Architecture

Contents
Executive summary................................................................................................................................................................................................................................................................................................................................3
Solution overview .....................................................................................................................................................................................................................................................................................................................................4
Solution components............................................................................................................................................................................................................................................................................................................................6
Hardware...................................................................................................................................................................................................................................................................................................................................................6
Software .................................................................................................................................................................................................................................................................................................................................................12
Design and configuration guidance .................................................................................................................................................................................................................................................................................... 16
HPE Synergy solution design and configuration ............................................................................................................................................................................................................................................. 16
Demonstration of HPE storage solutions for VMware Cloud Foundation workload domain ......................................................................................................................................... 23
Demonstrate VMware Enterprise PKS as a cloud-native platform on HPE Synergy ............................................................................................................................................................ 34
Demonstrate integrated composability with VCF and HPE Synergy using HPE OneView connector for VCF ............................................................................................ 38
Demonstrate patching and upgrading in VMware Cloud Foundation ................................................................................................................................................................................................ 39
Demonstrate stretching a vSAN cluster of a VMware Cloud Foundation management domain ............................................................................................................................... 40
Demonstrate ease of monitoring and reporting of VCF infrastructure using HPE OneView for vRealize Operations........................................................................... 41
Demonstrate HPE Synergy firmware upgrade using HPE OneView for VMware vRealize Orchestrator........................................................................................................... 46
Consolidated architecture deployment of VMware Cloud Foundation .............................................................................................................................................................................................. 52
Summary ......................................................................................................................................................................................................................................................................................................................................................53
Appendix A: Bill of materials ...................................................................................................................................................................................................................................................................................................... 54
Resources and additional links ................................................................................................................................................................................................................................................................................................ 56
Reference Architecture Page 3

Executive summary
Businesses face challenges to turn ideas into services faster, respond quicker to new customer demands, and innovate better by building new
services with technology to stay competitive. To meet these business demands, IT is increasingly adopting new cloud technologies, to replace
expensive hardware with a software-defined model. Enterprises need an ideal Hybrid IT model that supports both traditional and cloud-native
applications. Therefore, businesses are moving to digital transformation and software-defined data center (SDDC) solutions to support this shift.
Hewlett Packard Enterprise and VMware® collaborate to help customers accelerate the journey to hybrid cloud and bring the promise of the
software-defined data center to life. The combination of HPE Synergy Composable Infrastructure and VMware SDDC solution dramatically
improves business outcomes as well as overall value for our customers. HPE Synergy combined with VMware Cloud Foundation™ (VCF) delivers
a simplified and more secure private cloud - that is flexible, easy to deploy, seamless to manage, and simple to operate. For enterprise customers
looking to accelerate their journey to hybrid cloud, HPE Synergy with VMware Cloud Foundation is the right solution to support and run all your
enterprise apps—both traditional and containerized—in cloud environments.
This Reference Architecture provides guidance for deploying and managing VMware Cloud Foundation on HPE Synergy for virtual infrastructure
(VI) in a single VCF workload domain. It demonstrates the integrated composability with VCF and HPE Synergy using HPE OneView connector
for VCF. Also, the lifecycle management for HPE Synergy firmware using HPE OneView for VMware vRealize® Orchestrator™ based firmware
update workflow is discussed in further sections.
This Reference Architecture also covers using the HPE Primera, HPE 3PAR StoreServ and HPE Nimble as a principal storage as well as an
ancillary storage for VMware Cloud Foundation workload domain. The following are the highlighted benefits of this solution:
• The value of combining HPE Synergy with VCF, from a deployment and lifecycle perspective, in a cost-effective and simplified management
for faster time to value
• An easy-to-operate VI traditional data center in a VCF workload domain
• Agility to expand and contract physical and virtual infrastructure, on-demand, to quickly meet changing business requirements with HPE
OneView connector - a unique Integration between VMware SDDC Manager and HPE Synergy Composer that provides a single console to
dynamically compose software-defined and physical infrastructure
• An efficient environment for monitoring VCF using HPE OneView for VMware vRealize® Operations™ and custom dashboard
• An easy and automated way to upgrade HPE Synergy firmware using VMware vRealize Orchestrator workflow
• Guidance to attach and configure HPE Primera, HPE 3PAR StoreServ, and HPE Nimble Storage to be used as a principal storage and ancillary
storage for VCF VI workload domain
• Consolidated architecture deployment of VMware Cloud Foundation

Target audience: This document is intended for IT decision makers as well as architects and implementation personnel who want to understand
enterprise ready private cloud solutions using the HPE Synergy Composable Infrastructure capabilities offered by the HPE Synergy platform and
VMware Cloud Foundation. The reader should have a solid understanding and familiarity with VMware Cloud Foundation, VMware vRealize
Orchestrator and HPE Synergy.
Document purpose: The purpose of this document is to demonstrate enterprise-ready private cloud solutions by combining the value of VCF on
HPE Synergy Composable Infrastructure that is flexible and, easy to deploy.
This Reference Architecture describes solution testing performed in March 2020.
Reference Architecture Page 4

Solution overview
This Reference Architecture demonstrates best practices for customers building a cloud solution in an enterprise data center and deploying
business critical applications in an automated manner. The solution design is based on VMware Cloud Foundation on HPE Synergy. VMware
Cloud Foundation provides a unified software-defined data center (SDDC) platform comprising of VMware vSphere® Hypervisor, VMware Virtual
SAN™ Storage (vSAN) and VMware NSX® networking.
HPE Synergy is a single infrastructure of pools of compute, storage, and fabric resources, along with a single management interface, HPE
OneView, that allows IT to rapidly assemble, disassemble and re-assemble resources in any configuration. HPE Synergy architecture eliminates
hardware and operational complexity, so that IT can deliver applications faster and with greater precision and flexibility. HPE OneView is your
infrastructure automation engine built with software intelligence. It streamlines provisioning and lifecycle management across compute, storage,
and fabric resources in the HPE Synergy system.
This Reference Architecture demonstrates the following solutions for VMware Cloud Foundation on HPE Synergy:
• HPE Storage for VMware Cloud Foundation – HPE Primera, HPE 3PAR StoreServ, and HPE Nimble Storage as a principal storage and also
as an ancillary storage to a virtual infrastructure (VI) workload domain on HPE Synergy
• VMware Enterprise PKS - VMware Enterprise PKS as a cloud-native platform on a VMware Cloud Foundation NSX-T workload domain on
HPE Synergy
• HPE OneView connector for VCF - Integrated composability with VCF and HPE Synergy using HPE OneView connector for VCF
• HPE OneView for vRealize Orchestrator – HPE OneView for vRealize Orchestrator workflow to update the HPE Synergy compute firmware
• VMware Cloud Foundation lifecycle management - Offline VMware Cloud Foundation update for HPE Synergy
• High Availability of VMware Cloud Foundation management domain using VMware vSAN Stretched cluster

This solution demonstrates the concept using two (2) HPE Synergy 12000 Frames each equipped with eight (8) HPE Synergy 480 Gen10
servers, and an HPE Synergy D3940 Storage Module. Each HPE Synergy 12000 Frame uses HPE Virtual Connect SE 100Gb F8 Module to
provide uplink to the data center network. The sixteen (16) HPE Synergy 480 Gen10 servers are used to configure one VCF management
domain and three (3) VCF workload domains. The HPE Synergy Storage Module D3940 hosts software-defined VMware vSAN storage. HPE
Primera, HPE 3PAR StoreServ or HPE Nimble Storage provides the ancillary storage for the VI workload domains.
HPE Primera, HPE 3PAR StoreServ or HPE Nimble Storage in FC mode is also used to provide principal storage for the VI workload domains,
when VMware vSAN is not used as a principal storage.
Reference Architecture Page 5

Figure 1 shows the physical rack layout showcasing the solution components involved.

Figure 1. Physical rack layout showcasing the solution components involved


Reference Architecture Page 6

Solution components
Hardware
HPE Synergy Composable Infrastructure
HPE Synergy is a composable infrastructure platform that empowers IT to create and deploy resources instantly and continuously, gain control
of IT resources efficiently, and simplifies IT operation using a single software-defined infrastructure for physical, virtual, and containerized
workload. Developers and independent software vendor (ISVs) can programmatically control an HPE Synergy Composable Infrastructure
through a single, open API that is native in HPE Synergy powered by HPE OneView. This Reference Architecture is built upon the following
composability concepts and capabilities of the HPE Synergy platform.
Fluid resource pools
HPE Synergy allows the transformation of traditionally rigid physical systems into flexible virtual resource pools. HPE Synergy creates resource
pools of “stateless” compute, storage, and fabric capacity that can be configured almost instantly to rapidly provision infrastructure for a broad
range of applications.
Software-defined intelligence
The software-defined intelligence in HPE Synergy reduces operational complexity and enables IT organizations to make needed programmatic
changes quickly and confidently, with minimal human intervention. HPE Synergy abstracts operational details and replaces them with high-level,
automated operations. HPE Synergy uses templates to automatically implement change operations such as updating firmware, adding additional
storage to a service, or modifying a network.
Unified API
HPE Synergy delivers automation through a unified API that provides a single interface to discover, inventory, configure, provision, update, and
diagnose the composable infrastructure in a heterogeneous environment. This fully programmable interface integrates with dozens of popular
management tools such as Microsoft® System Centre, VMware vCenter® and open source automation and DevOps tools such as Chef, Docker,
and OpenStack.
Figure 2 describes the three architectural principles of HPE Synergy Composable Infrastructure.

Figure 2: Three architectural principles of HPE Synergy Composable Infrastructure

HPE Synergy Composer 2


HPE Synergy Composer 2 provides enterprise-level management to compose and deploy system resources, for all your application needs. This
management appliance uses software-defined intelligence with embedded HPE OneView to aggregate compute, storage and fabric resources in
a manner that scales to your application needs, instead of being restricted to the fixed ratios of traditional resource offerings. HPE OneView
Server Profiles and profile templates capture the entire server configuration in one place, enabling administrators to replicate new Server Profiles
and to modify them as needed to reflect changes in the data center. With HPE OneView REST API and automation tools, the entire process of
Reference Architecture Page 7

server personality definition and configuration can be automated. For this Reference Architecture the HPE OneView REST API and PowerShell
library were used to automate the Server Profile application to “stateless” servers.
HPE Synergy 12000 Frames
The HPE Synergy 12000 Frame is a base infrastructure of compute, storage, network fabric, and power into a scalable solution that addresses
and scales with various customer workloads and infrastructures. The HPE Synergy 12000 Frames reduces complexity in the IT infrastructure by
unifying all these resources into a common bus, and with the myriad of available network and storage interconnects which allows the frame to
interoperate with any other IT environment. At a high level, the HPE Synergy Frame supports the following:
• Twelve half-height or six full-height compute modules. The HPE Synergy design additionally allows for the inclusion of double-wide modules
as well, such as the HPE Synergy D3940 Storage Module.
• Ten fans and a single frame link module for in-band and out-of-band management.
• Up to six 2650 Watt power supplies.
• Up to six interconnect modules for full redundancy of three fabrics.

The HPE Synergy 12000 Frame features a fully automated and managed composer module using HPE OneView, contained within the HPE
Synergy Composer module. HPE OneView handles all the setup, provisioning, and management both at the physical and logical level.
Figure 3 shows front and rear view of HPE Synergy 12000 Frame.

Synergy UID
12000
HPE Synergy
Frame UID UID UID UID
12 Port 12G SAS Switch

Bay Bay UID Q1 1 2 3 4 5 6 7 8


Q2

1 6 HPE VC SE
1 2 1 2

32Gb FC Module 3 4 3 4
Reset

Appliance Synergy
EM
Bay 1 Composer2
UID

M GM T
Active
UID
Power
1 1 1
1

LINK
2 2 2
2

PID L/A UID

HPE VC SE 100Gb
F32 Module
Mode
Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8
X1 X2
L1 L2 L3 L4

iLO iLO iLO


iLO

UID
2650W 200-240VAC 2650W 200-240VAC 2650W 200-240VAC
96% 96% 96% 96% 96% 96%
Synergy D3940
Storage Module

Synergy Synergy Synergy


480 480 Synergy 480
Gen10 Gen10 480 Gen10

FC Module
Gen10
UID

HPE Synergy
UID UID UID UID 12 Port 12G SAS Switch
UID UID

UID 1 2 3 4 5 6 7 8
Q1 Q2
1 2 1 2
HPE VC SE
32Gb FC Module 3 4 3 4
Reset

Appliance EM
Bay 2 UID

Synergy
Composer2
M GM T
UID

UID

Active

Power 1
1
1
1 1 1 Front
Panel

LINK

2 2 2 2
2 2

UID

HPE Synergy 50Gb


Interconnect Link Module
Reset
L1 L2

iLO iLO iLO iLO iLO iLO

2650W 200-240VAC 2650W 200-240VAC 2650W 200-240VAC


96% 96% 96% 96% 96% 96%

Bay Bay
Synergy Synergy Synergy Synergy Synergy Synergy
7 480 480 480 480 480 480 12
Gen10 Gen10 Gen10 Gen10 Gen10 Gen10

HPE Synergy 50Gb


InterConnect Link
Module

Figure 3. HPE Synergy Frame showcasing SY480 servers, D3940 storage module, SAS interconnects, VC SE 100Gb F32 and VC SE 32Gb FC modules.

HPE Synergy 480 Gen 10 Compute Module


The HPE Synergy 480 Gen10 Compute Module delivers superior capacity, efficiency, and flexibility in a two-socket, half-height, single-wide form
factor to support demanding workloads. Powered by the latest 2nd Generation Intel® Xeon® Scalable Processors and featuring support for up to
4.5TB of HPE DDR4 SmartMemory, flexible storage controller options, three I/O connectors, and designed to create a pool of flexible compute
capacity within a composable infrastructure, the HPE Synergy 480 Gen10 Compute Module is an ideal platform for general-purpose enterprise
workload performance now and in the future.
The solution as presented in this Reference Architecture contains two HPE Synergy 12000 Frames. Each HPE Synergy 12000 Frames in a rack
consist of four (4) HPE Synergy 480 Gen10 Servers.
Reference Architecture Page 8

Figure 4 shows the HPE Synergy 480 Gen10 Compute Module.

Figure 4. HPE Synergy 480 Gen 10 Compute Module

HPE Synergy D3940 Storage Module


The HPE Synergy D3940 Storage Module is a direct attached storage module with 40 Small Form Factor (SFF) drive bays designed for use in
HPE Synergy 12000 Frames. Through the HPE Synergy 12Gb SAS Connection module, it provides composable storage for up to 10 compute
modules in a single frame. HPE Synergy Storage is optimized to use as either a direct attached storage array or as software-defined storage. HPE
Synergy D3940 Storage Modules support a family of 12G SAS or 6G SATA HDD and SSD Smart Drives.
Figure 5 shows the HPE Synergy D3940 Synergy Module.

Figure 5. HPE Synergy Storage Module

HPE Virtual Connect SE 100Gb F32 Module for Synergy


The HPE Virtual Connect SE 100Gb F32 Module, master module based on synergy composable fabric, is designed for composable infrastructure.
The disaggregated, rack-scale design uses a master/satellite architecture to consolidate data center network connections, reduce hardware, and
scale network bandwidth across multiple HPE Synergy 12000 Frames. The master module contains intelligent networking capabilities that
extends connectivity to satellite frames through Interconnect Link Modules. This decreases top of rack switch needs and substantially reduces
costs. The components reduction simplifies fabric management at scale while consuming fewer ports at the data center aggregation layer.
The HPE Virtual Connect SE 100Gb F32 Module for Synergy eliminates network sprawl at the edge with one device that converges traffic inside
the HPE Synergy 12000 Frames and directly connects to external LANs.
Reference Architecture Page 9

HPE Synergy 50Gb Interconnect Link Module


The HPE Synergy 50Gb Interconnect Link Module (satellite module) is designed for composable infrastructure. Based on a disaggregated, rack-
scale design, it uses a master/satellite architecture to consolidate data center network connections, reduce hardware, and scale network
bandwidth across multiple HPE Synergy 12000 Frames.
HPE Virtual Connect SE 40Gb F8 Module
The HPE Virtual Connect SE 40Gb F8 Module is designed for composable infrastructure. Its disaggregated, rack-scale design uses a
master/satellite architecture to consolidate data center network connections, reduce hardware, and scales network bandwidth across multiple
HPE Synergy Frames. The master module contains intelligent networking capabilities that extend connectivity to satellite frames through
Interconnect Link Modules. This eliminates top of rack switch need and substantially reduces cost. The reduction in components also simplifies
fabric management at scale while consuming fewer ports at the data center aggregation layer.
HPE Synergy 20Gb Interconnect Link Module
The HPE Synergy 20Gb Interconnect Link Module (satellite module) is designed for composable infrastructure. Based on a disaggregated, rack-
scale design, it uses a master/satellite architecture to consolidate data center network connections, reduce hardware and scale network
bandwidth across multiple HPE Synergy 12000 Frames.
HPE Virtual Connect SE 16Gb FC Module
The HPE Virtual Connect SE 16Gb FC Module for HPE Synergy is a Storage Area Network (SAN) interconnect with a wire-once change-ready
technology. The Fibre Channel (FC) module in conjunction with Synergy composer powered by HPE OneView allows workloads to be moved or
managed without modifying the network. HPE OneView provides an intuitive management console for seamless connectivity between virtualized
Synergy Compute Modules 16Gb FC SAN fabrics.
HPE Virtual Connect SE 32Gb FC Module
The HPE Virtual Connect SE 32Gb FC Module for HPE Synergy is a Storage Area Network (SAN) interconnect with a wire-once change-ready
technology. The Fibre Channel (FC) module in conjunction with HPE Synergy Composer powered by HPE OneView allows workloads to be
moved or managed without modifying the network. HPE OneView provides an intuitive management console for seamless connectivity between
virtualized Synergy Compute Modules and 32Gb FC SAN fabrics.
HPE Synergy 3830C 16Gb Fibre Channel Host Bus Adapter
HPE Synergy 3830C 16Gb Fibre Channel Host Bus Adapter connects Synergy compute resource pools to SANs over 16Gb native Fibre Channel
(FC) fabrics. The Synergy 3830C supports advanced virtualization, security, dual-port isolation, dynamic power management, and low CPU
utilization features.
HPE Synergy 5830C 32Gb Fibre Channel Host Bus Adapter
The HPE Synergy 5830C 32 GB FC Host Bus Adapter (HBA) provides low latency and flexible connectivity to HPE Synergy Virtual Connect FC
modules. The HPE Synergy 5830C 32 GB FC HBA helps minimize CPU utilization for storage I/O and is ideal for environments with greater
virtual machine density and bandwidth requirements.
HPE Synergy 6820C 25/50Gb Converged Network Adapter
The HPE Synergy 6820C 25/50Gb Converged Network Adapter is a key element in synergy composable fabric connecting pools of compute
resources to networks with reliable, high-performing converged Ethernet connectivity up to 50Gb. The HPE Synergy 6820C 25/50Gb Ethernet
Adapter provides more than two times the bandwidth compared to 10/20Gb adapters. Therefore, it simplifies hardware implementation by
reducing the number of adapters, associated switches and compute modules required to achieve higher bandwidth. This Synergy 6820C adapter
converges Ethernet and FCoE onto a single connection, simplifying hardware and reducing costs.
HPE Synergy 3820C 10/20Gb Converged Network Adapter
The HPE Synergy 3820C 10/20Gb Converged Network Adapter is a key element in synergy composable fabric connecting pools of compute
resources to networks with reliable, high-performing converged 10 or 20 Gb Ethernet connectivity. With Flex-20 technology, the Synergy 3820C
converges Ethernet and FCoE onto a single connection simplifying hardware and reducing costs. Each adapter replaces a 10Gb NIC and 8Gb
Fibre Channel HBA simplifying I/O hardware by 50% and reducing costs up to 60%. The Synergy 3820C is an ideal choice for any virtualized or
converged data center.
Reference Architecture Page 10

HPE FlexFabric 5940 Switch


The HPE FlexFabric 5940 Switch Series is a family of high performance and low-latency 10GbE, 40GbE top-of-rack (ToR) data center switches.
This switch series includes 100G uplink technology and a 2-slot and 4-slot modular form factor providing ultimate flexibility for ever-changing
data center requirements.
Figure 6 shows the front view of the HPE FlexFabric 5940 switch.

Figure 6. HPE FlexFabric 5940 JH396A Switch

HPE StoreFabric SN6600B Fibre Channel Switch


The HPE SN66x0B Fibre Channel Switch meets the demands of hyper-scale virtualization, larger cloud infrastructures, and growing flash-based
storage environments by delivering market-leading 32Gb Fibre Channel technology and capabilities. It provides a high-density building block for
increased scalability, designed to support growth, demanding workloads, and data center consolidation in small to large-scale enterprise
infrastructures. It is built for maximum flexibility, scalability, and ease of use. Organizations can scale from 24 to 128 ports, all in an efficient 1U or
2U package. It also provides a simplified deployment process and a point-and-click user interface, making it both powerful and easy to use. With
the SN66x0B Switch, organizations gain the best of both worlds: high-performance access to industry-leading storage technology and "pay-as-
you-grow" scalability to support an evolving storage environment. Figure 7 shows HPE StoreFabric SN6600B FC switch.

Figure 7. HPE StoreFabric SN6600B Fibre Channel Switch

If using a fabric attached Fibre Channel topology, a minimum of two (2) SAN switches are required. If more than two switches are needed in the
solution, switches of the same vendor/type must be added in pairs, so there is always an even number of switches. It is important to ensure that
you select the correct model switch for your future growth plans. Select a bigger switch with only the ports requires license to start, if you plan on
expanding your HPE Synergy solution in the future.
HPE 3PAR StoreServ
Hewlett Packard Enterprise leveraged HPE 3PAR StoreServ to meet the architectural guidelines of an HPE Synergy 480 solution to eliminate
any single point of failure (hardware or software) in the system. To mitigate single points of failure at the hardware layer, the system is designed
with redundant components, including redundant power domains. At a minimum, there are two controller nodes and two copies of the HPE
3PAR Operating System, even in the smallest system configuration.

To further the resiliency of HPE 3PAR StoreServ, nodes are powered by two (1+1) redundant power supplies and backed up by a string of two
batteries. Each battery has sufficient capacity to power the controller nodes and have enough space to save all necessary data in memory into
the local physical drive. Although many architectures use “cache batteries,” they are not suitable for extended downtimes that are usually
associated with natural disasters and unforeseen catastrophes.
To further mitigate downtime due to component failure, HPE 3PAR Persistent Cache is a resiliency feature built into the HPE 3PAR Operating
System that enables graceful handling of an unplanned controller failure or planned maintenance of a controller node. This feature eliminates the
substantial performance penalties that are associated with traditional modular arrays and the cache “write-through” mode they enter under
Reference Architecture Page 11

certain conditions. For the HPE 3PAR StoreServ 8440 4-node array, any node that loses its adjacent node can dynamically form a mirrored
cache relationship with another storage controller node. Figure 8 shows the HPE 3PAR 8440 4-Node Storage.

Figure 8. HPE 3PAR 8440 4-Node Storage

HPE Nimble Storage HF20 Adaptive Flash Array


The HPE Nimble Storage HF20 Adaptive Flash Array is designed for both principal and secondary flash workloads. It is a hybrid flash array for
mixed, principal workloads, where cost-efficient flash performance is important. It is a secondary flash array for backup and DR while allowing you
to put your backup data to work. Figure 9 shows the HPE Nimble Storage.

Figure 9. HPE Nimble Storage

HPE Primera 600 Storage


HPE Primera storage system features a high-speed, full-mesh passive interconnect that joins multiple controller nodes (the high-performance
data movement engines of the HPE Primera architecture) to form an all-active cluster. This low-latency interconnect allows for tight coordination
among the controller nodes and a simplified software model.
HPE Primera storage redefines mission-critical storage for tier-0 applications. Designed for NVMe and Storage Class Memory, HPE Primera
delivers remarkable simplicity, app-aware resiliency for mission-critical workloads, and intelligent storage that anticipates and prevents issues
across the infrastructure stack.
HPE Primera delivers on the promise of intelligent storage, advanced data services, and simplicity for your mission-critical applications. With a
services-centric OS that sets up in minutes, HPE Primera also upgrades seamlessly to minimize risk and be transparent to applications. Figure 10
shows the HPE Primera Storage.

Figure 10. HPE Primera A670 Storage


Reference Architecture Page 12

Software
The table below lists the software components used in this Reference Architecture. Further below mentions the layers of the full solution stack.
Table 1 defines the HPE Synergy 480 Gen10 software components and versions used in this Reference Architecture.
Table 1. HPE software and firmware components used in this Reference Architecture

Component Version

HPE Synergy Composer 5.00.02

HPE Synergy Composer 2 5.00.02

Custom Synergy SPP 2019.12.20200326


(HPE_Synergy_Custom_SPP_2019.12.20200326_Z7550-96866.iso)

HPE Nimble OS 5.0.8

HPE 3PAR OS 3.3.1 MU2

HPE 3PAR SSMC 3.6.1

HPE Primera OS 4.0.3

B-series SAN Fabric OS (FOS) 8.2.1c

VMware Cloud Foundation 3.9.1

Table 2 defines the VMware Cloud Foundation software components.


Table 2. VMware software and firmware components used in this Reference Architecture

Component Version

Cloud Builder VM 2.2.1.0

SDDC Manager 3.9.1

VMware vCenter Server® 6.7 (Appliance) 6.7 Update 3d(Build 15679281)

VMware vSphere 6.7 ESXi 6.7 U3 (Build: 15160138)

HPE OEM Customized vSphere Distribution 6.7 VMware-ESXi-6.7.0-Update3-14320388-HPE-Gen9plus-670.U3.10.4.5.19-Aug2019.iso


(Build: 14320388)
ESXi670-201912001.zip
VMware vSphere 6.7 patch bundle
(Build: 15160138)

VMware vSAN™ 6.7 Update 3b

VMware NSX®-V Data Center for vSphere 6.4.6

VMware NSX-T Data Center 2.5

Pivotal Container Service 1.5

VMware vRealize® Suite Lifecycle Manager 2.1 Patch 2

VMware vRealize® Log insight™ 4.8

vRealize Log Insight Content Pack for NSX for vSphere 3.9

vRealize Log Insight Content Pack for Linux 2.0.1

vRealize Log Insight Content Pack for vRealize Automation 7.5+ 1.0

vRealize Log Insight Content Pack for vRealize Orchestrator 7.0.1+ 2.1

vRealize Log insight Content Pack for NSX-T 3.8.2

vRealize Operations Manager™ 7.5


Reference Architecture Page 13

Component Version

vRealize Automation™ 7.6

VMware Horizon™ 7 7.10.0

VMware Cloud Foundation


VMware Cloud Foundation is the industry’s most advanced enterprise-ready hybrid cloud platform providing a complete set of software-defined
services for compute, storage, networking, security and cloud management to run enterprise apps whether it is traditional or containerized. VCF
drastically simplifies data center operations by deploying a standardized and validated architecture with built-in lifecycle automation of the cloud
stack. It orchestrates, provisions, and deploys a software-defined data center (SDDC) platform by integrating VMware vSphere, vSAN, and NSX
into a full stack HCI solution that delivers enterprise-ready cloud infrastructure.
Figure 11 shows VMware Cloud Foundation components.

Figure 11. VMware Cloud Foundation

VMware Cloud Foundation components


The core components for VMware Cloud Foundation are explained below.
Cloud Foundation Builder VM
The Cloud Foundation Builder VM is a one-time use VM which deploys and configures the management domain and transfers inventory and
control to SDDC Manager. During the deployment process, the Cloud Foundation Builder VM validates network information provided in the
deployment parameter spreadsheet such as DNS, network (VLANS, IPs, MTUs), and credentials. After the management domain is up and the
SDDC Manager is running, the Cloud Foundation Builder VM must be powered off and archived. Table 3 shows Cloud Foundation Builder VM
resource requirements.
Table 3. Cloud Foundation Builder VM resource requirement
Components Requirements

CPU 4 vCPUs

Memory 4GB

Storage 350GB
Reference Architecture Page 14

SDDC Manager
SDDC Manager manages the bring-up of the Cloud Foundation system, creates and manages workload domains, and performs lifecycle
management to ensure the software components remain up to date. SDDC Manager also monitors the logical and physical resources of VCF. It
allows data center administrators to configure the additional hosts and racks into a logical pool of resources and thus multiple racks can be
managed as a single VCF System. SDDC Manager controls these processes by using workflows. Each workflow comprises of a series of tasks,
which are executed by SDDC Manager. There are two VMs installed by VCF 3.0 for SDDC Manager, each VM is performing its own functions.
VMware vCenter Server
VMware vCenter Server provides management of a VMware virtualized environment with one or more ESXi hosts. SDDC Manager deploys one
VMware vCenter Server per workload domain. By default, all vCenter Servers are configured in enhanced linked mode.
VMware Platform Services Controller
During bring-up, SDDC Manager deploys two platform services controllers in the management domain. These instantiate a Single Sign On
domain. All vCenter Servers (management domain and compute workload domains) are registered with the SSO domain and configured in
enhanced link mode.
VMware vSphere (ESXi)
ESXi is a type 1 hypervisor used to implement virtualization on bare metal systems. ESXi provides compute virtualization within the software-
defined data center and it is foundational building block for implementing a private cloud.
VMware vSAN
VMware vSAN aggregates local or direct-attached data storage devices to create a single storage pool shared across all hosts in the vSAN
cluster. vSAN eliminates the need for external shared storage, simplifies storage configuration and virtual machine provisioning.
VMware NSX data center for vSphere
NSX is the network virtualization platform for the SDDC, delivering the operational model of a virtual machine for entire networks. With NSX
network functions including switching, routing, and firewall are embedded in the hypervisor and distributed across the environment.
VMware NSX for vSphere is a virtualized networking component in the software-defined data center (SDDC) architecture, which
programmatically creates, snapshots, deletes, and restores software-based virtual networks. With network virtualization, the functional equivalent
of a network hypervisor, NSX reproduces the complete set of Layer 2 to Layer 7 networking services (e.g., switching, routing, firewall, and load
balancing) in software. It allows these services to be programmatically assembled in any arbitrary combination to produce unique, isolated virtual
networks in a matter of seconds. NSX also provides a platform for various security services both network and endpoint based. NSX provides
various built-in services, including L2-L4 firewall and activity monitoring. Additionally, security vendors can leverage its guest introspection and
network introspection frameworks to deliver service chained next-generation firewall, IDS/IPS, agentless AV, file integrity monitoring, and
vulnerability management capabilities.
VMware NSX-T
VMware NSX-T is designed to address application frameworks and architectures that have heterogeneous endpoints and technology stacks. In
addition to vSphere, these environments may include other hypervisors, containers, bare metal, and public clouds. NSX-T allows IT and
development teams to choose the technologies best suited for their applications. NSX-T is also designed for management, operations, and
consumption by development organizations in addition use by IT.
VMware vRealize Log insight
vRealize Log Insight delivers heterogeneous and highly scalable log management with intuitive, actionable dashboards, sophisticated analytics
and broad third-party extensibility, providing deep operational visibility and faster troubleshooting.
VMware Cloud Foundation also has the following optional components for which separate licenses are needed:
• VMware vRealize Operations Manager: vRealize Operations Manager delivers intelligent operations management with application-to-
storage visibility across physical, virtual, and cloud infrastructures. Using policy-based automation, operation teams automate key processes
and improve IT efficiency. This is an optional component.
• VMware vRealize Automation: vRealize Automation is a cloud automation tool that accelerates the delivery of IT services through
automation and pre-defined policies, providing high-level of agility and flexibility for developers, while enabling IT teams to maintain
frictionless governance and control. This is an optional component.
Reference Architecture Page 15

• vRealize Suite Orchestrator: vRealize Orchestrator is a development- and process-automation platform that provides an extensive library of
workflows and a workflow engine. It simplifies the automation of complex IT tasks.

HPE OneView for VMware vRealize Operations


HPE OneView for VMware vRealize Operations provides integrated and highly automated performance, capacity, configuration compliance, and
cost management tools to the vRealize Operations custom GUI. The plugin seamlessly integrates the manageability features of HPE Synergy
with VMware analytics engine that analyzes what is normal and then applies that baseline to a dynamic server environment.
When the HPE OneView for VMware vRealize Operations is installed, the custom HPE OneView dashboards are added to the vRealize Operation
custom GUI. The HPE OneView dashboards allow you to monitor resources in a vRealize environment. The attributes that can be monitored
include, resource health, power, temperature (server and enclosure), and system alerts. The analytics engine allows for proactive monitoring of
the HPE OneView resource environment and indicates the state of the resources. If a problem occurs, an alert is triggered and displayed. The
analytics engine also provides proactive prediction, which can determine the point in the future when a resource will reach a predefined critical
level.
HPE OneView for VMware vRealize Orchestrator
HPE OneView for VMware vRealize Orchestrator helps customers automate complex IT tasks in an extensible and repeatable manner. It provides
a predefined collection of HPE tasks and workflows that can be used in vRealize Orchestrator (VRO) with easy-to-use, drag and drop access to
automation of HPE OneView managed hardware deployment, firmware updates, and other life-cycle tasks. HPE OneView for VMware vRealize
Orchestrator allows the advanced management features of HPE OneView to be incorporated into larger IT workflows. HPE OneView workflows
and actions can also be integrated in VMware vRealize Automation via vRealize Orchestrator.
HPE OneView connector for VCF
HPE OneView connector for VCF provides a REST service to answer DMTF Redfish® compliant requests by querying HPE OneView. HPE
OneView is a fresh approach to converged infrastructure management, inspired by the way you expect to work, with a single integrated view of
your IT infrastructure.
DMTF Redfish is an open industry standard specification and schema that specifies a RESTful interface and utilizes JSON and OData to help
customers integrate solutions within their existing tool chains.
Reference Architecture Page 16

Design and configuration guidance


HPE Synergy solution design and configuration
The solution involves two (2) HPE Synergy 12000 frames each equipped with eight (8) HPE Synergy 480 Gen10 servers, and an HPE Synergy
D3940 Storage Module. Each HPE Synergy D3940 storage module consists of 40 drive enclosures comprising a mix of 1.92TB SATA SSD drives
for capacity tier and 800GB SAS SSD for cache tier, as per VMware vSAN requirement. The HPE Synergy 12000 Frames have a redundant pair
of HPE Synergy 12Gb SAS Connection Modules, to provide powerful and redundant connectivity to the HPE Synergy D3940 Storage Module,
and a redundant pair of HPE Synergy Virtual Connect SE 40Gb F8 Modules, for high-speed uplink connectivity to multiple networks.
Figure 12 shows the network layout of this Reference Architecture.

IRF link
HPE 5940 FlexFabric A HPE 5940 FlexFabric B
1 2 3 4 5 6 QSFP+ Green=40Gbps,Yellow=10Gbps 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 1 2 3 4 5 6 QSFP+ Green=40Gbps,Yellow=10Gbps 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32

HPE FlexFabric HPE FlexFabric


5940 Series 5940 Series
Switch Switch
JH396A JH396A

SYS SYS

80G LAG-A 80G LAG-B

ICM 3 ICM 6
UID UID

HPE VC SE 40Gb PID HPE VC SE 40Gb PID


F8 Module L/A F8 Module L/A

L1 L2 Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 L3 L4 Reset L1 L2 Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 L3 L4 Reset
Eth / X-Link Eth / X-Link

ICM 6 ICM 3
UID UID

HPE Synergy 20Gb HPE Synergy 20Gb


Interconnect Link Module Interconnect Link Module
L1 L2 L1 L2

P1 P2 P1 P2

HPE 3820c CNA (FlexNIC's) HPE 3820c CNA (FlexNIC's)


Synergy

Synergy
Gen10

Gen10
iLO

iLO
480

480
UID

UID

Synergy Synergy
12000 12000
Frame UID UID UID UID Frame UID UID UID UID
900 GB
900 GB

900 GB

900 GB

900 GB

900 GB

900 GB

900 GB
SAS
10K
SAS

SAS

SAS

SAS

SAS

SAS

SAS
10K

10K

10K

10K

10K

10K

10K

Bay Bay Bay Bay


1 6 1 6

Appliance SynergyComposer
Appliance SynergyComposer
Bay 1 UID Bay 1 UID

Active Active

Power Power
1
1 1 1 1 1 1 1

2
2 2 2 2 2 2 2
900 GB
SAS
10K
900 GB

900 GB

900 GB

900 GB

900 GB

900 GB

900 GB
SAS

SAS

SAS

SAS

SAS

SAS

SAS
10K

10K

10K

10K

10K

10K

10K

iLO
iLO iLO iLO iLO iLO iLO iLO

UID UID

Synergy
Synergy 480 Synergy Synergy Synergy Synergy Synergy Synergy
480 Gen10 480 480 Synergy D3940 480 480 480 480 Synergy D3940
Storage Module Storage Module
Gen10 Gen10 Gen10 Gen10 Gen10 Gen10 Gen10

UID
UID UID UID UID UID UID UID
900 GB
SAS
10K
900 GB

900 GB

900 GB

900 GB

900 GB

900 GB

900 GB
SAS

SAS

SAS

SAS

SAS

SAS

SAS
10K

10K

10K

10K

10K

10K

10K

Appliance Appliance
Bay 2 UID
Bay 2 UID

1 1 1 1 Front 1 1 1
1 Front
Panel Panel

2
2 2 2 2 2 2 2
900 GB
SAS
900 GB

900 GB

900 GB

900 GB

900 GB

900 GB

900 GB

10K
SAS

SAS

SAS

SAS

SAS

SAS

SAS
10K

10K

10K

10K

10K

10K

10K

iLO iLO iLO iLO iLO iLO iLO iLO

Bay Bay Bay Bay


Synergy Synergy Synergy Synergy Synergy Synergy Synergy Synergy
7 480 480 480 480 12 7 480 480 480 480 12
Gen10 Gen10 Gen10 Gen10 Gen10 Gen10 Gen10 Gen10

HPE Synergy Frame 1 HPE Synergy Frame 2

Figure 12. Network layout of this Reference Architecture


Reference Architecture Page 17

Table 4 defines the hardware configuration used in this Reference Architecture.


Table 4. Hardware configuration

Hardware Details

Number of racks 1

HPE Synergy 12000 Frames in rack 2

HPE Synergy 480 Gen 10 Compute Module per Synergy 12000 Frame 8

HPE Synergy D3940 Storage Module per HPE Synergy 12000 Frame 1

HPE Virtual Connect SE 40Gb F8 Module for Synergy per Synergy 12000 Frame 1

HPE Synergy 20GB Interconnect Link Module per Synergy 12000 Frame 1

HPE Virtual Connect SE 100Gb F32 Module for Synergy per Synergy 12000 Frame 1

HPE Synergy 50GB Interconnect Link Module per Synergy 12000 Frame 1

HPE Virtual Connect SE 16Gb FC Modules per Synergy 12000 Frame 2

HPE Virtual Connect SE 32Gb FC Modules per Synergy 12000 Frame 2

HPE Synergy Composer Module per Synergy 12000 Frames 1

HPE Synergy 3830C 16Gb Fibre Channel Host Bus Adapter 1

HPE Synergy 5830C 32Gb Fibre Channel Host Bus Adapter 1

HPE Synergy 6820C 25/50Gb Converged Network Adapter 1

Note
Since HPE Nimble Storage supports bandwidth only up to 16Gb, HPE Nimble Storage configurations in this solution are tested with HPE VC SE
16Gb FC Module and HPE VC SE 40Gb F8 Module/HPE 20Gb Interconnect Link Module.

Table 5 defines the HPE Synergy 480 Gen10 hardware components used in this Reference Architecture.
Table 5. HPE Synergy 480 Gen10 hardware components (quantities are per node)

Hardware Quantity Description

CPU 2 Intel(R) Xeon(R) Gold 6254 CPU (3.1 GHz / 18-core)

24 HPE 16GB Single Rank x4 DDR4-2933 CAS-21-21-21 Registered Smart Memory Kit
Memory
Total 384GB memory on each node

FC HBA 1 HPE Synergy 5830C 32Gb Fibre Channel Host Bus Adapter

25/50Gb CNA 1 HPE Synergy 6820C 25/50Gb Converged Network Adapter

Table 6 defines the ancillary storage hardware configuration used in this Reference Architecture.
Table 6. HPE Synergy 480 Gen10 hardware components (quantities are per node)

Hardware Details

Number of racks 1

HPE Nimble Storage HF20 Adaptive Dual Controller Array 1

HPE Primera 600 Storage 1

HPE 3PAR StoreServ 8440 Storage 1


Reference Architecture Page 18

The VMware Cloud Foundation infrastructure needs different external services for initial deployment and deployment of other optional
components like vRealize Operations or vRealize Automation. Those services as Active Directory, Dynamic Host Configuration Protocol (DHCP),
Domain Name Service (DNS) and Network Time Protocol (NTP) are part of customer’s data center environment.
The Cloud Foundation Builder virtual machine was installed on one of the ESXi host that will be the part of the HPE Synergy VCF environment. It
is configured to have network connectivity to the management network of all ESXi hosts to be added to the VMware Cloud Foundation solution
as well network connectivity to Virtual Machines providing external services as Active Directory, Dynamic Host Configuration Protocol (DHCP),
Domain Name Service (DNS), and Network Time Protocol (NTP).
HPE FlexFabric 5940 configuration for VMware Cloud Foundation
HPE FlexFabric 5940 Switches should be connected and configured for IRF. All the VLANs required for VCF deployment should be created on
the top-of-rack (ToR) switches or allowed from customer network data center uplinks. The Ethernet downlink ports on the HPE FlexFabric 5940
should be trunk enabled allowing all VLANs required for the VMware Cloud Foundation stack. Create a LAG by selecting single port from each
switch and ensure the LAG connects to the HPE Synergy Virtual Connect ports as shown in the Figure 12.
VMware introduced ‘Application Virtual Networks’ in VCF 3.9.1 where BGP configuration becomes necessary to deploy VCF 3.9.1 via Cloud
Builder VM. For VCF 3.9.1, Figure 13 and 14 highlights the fields in VCF deployment parameter sheet. This is required for setting up network
and routing configuration required for BGP on HPE FlexFabric 5940 switches.

Figure 13. BGP VLANs shown in VCF deployment parameter sheet


Reference Architecture Page 19

As per the above VCF 3.9.1 parameter sheet, two additional VLANs needs to be configured as ‘Uplinks’ for VMware NSX in VCF Management
Domain. These uplinks will represent the NSX Edge VM uplinks for North-South communication.

Figure 14. Application Virtual Network details shown in VCF deployment parameter sheet

Before deploying the Cloud Builder VM for VCF 3.9.1 deployment, ensure to configure the HPE 5940 FlexFabric switches with BGP as shown in
the following command list.
Configure VLANs required for VMware Cloud Foundation deployment with the following commands.

[snet01]display current-configuration
#
vlan 1
#
vlan 1611 to 1618
#
vlan 2711
description ESGUplink-1
#
vlan 2712
description ESGUplink-2
#
irf-port 1/1
port group interface FortyGigE1/0/1
port group interface FortyGigE1/0/2
#
irf-port 2/2
port group interface FortyGigE2/0/1
port group interface FortyGigE2/0/2
#
stp global enable
#
Reference Architecture Page 20

interface NULL0
#
interface LoopBack0
ip address 1.1.1.1 255.255.255.255
#
interface Vlan-interface1
#
interface Vlan-interface1611
description Mgmt-MGMT
mtu 9000
ip address 172.16.11.1 255.255.255.0
#
interface Vlan-interface1612
description vMotion-MGMT
mtu 9000
ip address 172.16.12.1 255.255.255.0
#
interface Vlan-interface1613
description vSAN-MGMT
mtu 9000
ip address 172.16.13.1 255.255.255.0
#
interface Vlan-interface1614
description VXLAN-MGMT
mtu 9000
ip address 172.16.14.1 255.255.255.0
#
interface Vlan-interface1615
description Mgmt-WLD
ip address 172.16.15.1 255.255.255.0
#
interface Vlan-interface1616
description vMotion-WLD
ip address 172.16.16.1 255.255.255.0
#
interface Vlan-interface1617
description vSAN-WLD
ip address 172.16.17.1 255.255.255.0
#
interface Vlan-interface1618
description VXLAN-WLD
ip address 172.16.18.1 255.255.255.0
#
interface Vlan-interface2711
mtu 9000
ip address 172.27.11.1 255.255.255.0
#
interface Vlan-interface2712
mtu 9000
ip address 172.27.12.1 255.255.255.0
#

For application virtual networks in VCF 3.9.1, you need to configure BGP on the HPE FlexFabric 5940 switches. In the VCF parameter sheet,
the BGP Autonomous System ID is mentioned as 65001. The universal DLR deployed as part of NSX deployment in the VCF management
domain has the BGP Autonomous System ID as 65004. Hence both the BGP Autonomous Systems need to establish peering for North-South
communication.
Reference Architecture Page 21

Configure the NSX Edge uplinks to BGP peer with the UDLR ESG Autonomous System ID 65004 and enable the RegionA01 and xRegion01
VXLAN logical switch networks in the BGP configuration on HPE 5940 switches as shown in the following BGP configuration.
#
bgp 65001
router-id 1.1.1.1
peer 172.27.11.12 as-number 65004
peer 172.27.11.12 connect-interface Vlan-interface2711
peer 172.27.11.12 password simple <Password>
peer 172.27.11.13 as-number 65004
peer 172.27.11.13 connect-interface Vlan-interface2711
peer 172.27.11.13 password simple <Password>
peer 172.27.12.12 as-number 65004
peer 172.27.12.12 connect-interface Vlan-interface2712
peer 172.27.12.12 password simple <Password>
peer 172.27.12.13 as-number 65004
peer 172.27.12.13 connect-interface Vlan-interface2712
peer 172.27.12.13 password simple <Password>
#
address-family ipv4 unicast
network 172.16.11.0 255.255.255.0
network 172.27.11.0 255.255.255.0
network 172.27.12.0 255.255.255.0
network 192.168.20.0 255.255.255.0
network 192.168.21.0 255.255.255.0
network 192.168.41.0 255.255.255.0
peer 172.27.11.12 enable
peer 172.27.11.13 enable
peer 172.27.12.12 enable
peer 172.27.12.13 enable
#

HPE OneView configuration through HPE Synergy Composer


The following steps summarize the HPE Synergy configuration.
Create HPE OneView networks with respect to the VLANs created on the ToR switches as shown in the Table 7.
Table 7. HPE OneView Networks

HPE OneView Networks VLAN IDs

VCF Management Domain Management VLAN 1611

VCF Management Domain vMotion VLAN 1612

VCF Management Domain vSAN VLAN 1613

VCF Management Domain VTEP (VXLAN) 1614

VCF Workload Domain Management VLAN 1615

VCF Workload Domain vMotion VLAN 1616

VCF Workload Domain vSAN VLAN 1617

VCF Workload Domain VTEP (VXLAN) 1618


Reference Architecture Page 22

Create HPE Oneview network sets to be used in the respective HPE Oneview Server Profile template of VMware Cloud Foundation workload
domain nodes, as shown in Table 8 and Table 9.
Table 8. HPE OneView VCF Management Network Sets

VCF Management Domain Network Set A VLAN VCF Management Domain Network Set B VLAN

Management VLAN 1611 Management VLAN 1611

vMotion VLAN 1612 vMotion VLAN 1612

vSAN VLAN 1613 vSAN VLAN 1613

VTEP (VXLAN) 1614 VTEP (VXLAN) 1614

Table 9. HPE OneView VCF Workload Network Sets

VCF Workload Domain Network Set A VLAN VCF Workload Domain Network Set B VLAN

Management VLAN 1615 Management VLAN 1615

vMotion VLAN 1616 vMotion VLAN 1616

vSAN VLAN 1617 vSAN VLAN 1617

VTEP (VXLAN) 1618 VTEP (VXLAN) 1618

Create one Logical Interconnect Group for each physical type of interconnect used in the solution and assign the proper uplink sets as shown
Table 10.
Table 10. HPE OneView Logical Interconnect Group

Logical Interconnect Group A Values Logical Interconnect Group B Values

Name of Logical Interconnect Group LIG-A Name of Logical Interconnect Group LIG-B

Add HPE OneView Networks 1611-1618 Add HPE OneView Networks 1611-1618

Add Uplink Ports from Virtual Connect ICM3 Add Uplink Ports from Virtual Connect ICM6

Note
The sample VLAN IDs mentioned above has been used in the solution. However, the VLAN IDs need to be replaced as per the customer’s
requirements.

Create an Enclosure Group.


Create a Logical Enclosure and apply the appropriate firmware baseline.
Create the HPE OneView Server Profile templates, selecting the appropriate hardware type and the Enclosure Group created in the previous
steps.
Create a Server Profile template for the VMware Cloud Foundation management domain and workload domain with the following settings, as
listed in Table 11 and Table 12.
Table 11. Sample Values for HPE OneView Server Profile for VCF management domain

HPE OneView Server Profile template for VCF Management Domain Node Sample Values

Server Profile Template Name VCF Management Domain Template

Firmware

Connection 1 VCF Management Domain Network Set A

Connection 2 VCF Management Domain Network Set B


Reference Architecture Page 23

HPE OneView Server Profile template for VCF Management Domain Node Sample Values

Local Storage (2 SSDs) via Integrated Storage Controller RAID-1 for VMware ESXi 6.7 U3

HPE D3940 (SSDs/HDD) via SAS Mezz 1 Storage Controller Cache_Tier (SSD), Capacity Tier (SSD/HDD)
*Configure capacity and cache Tiers in Logical JBODs in HPE OneView in the Storage tab

Table 12. Sample Values for HPE OneView Server Profile for VCF workload domain

HPE OneView Server Profile template for NSX-T Workload Domain Node Sample Values

Server Profile Template Name VCF Workload Domain Template

Firmware

Connection 1 VCF Workload Domain Network Set A

Connection 2 VCF Workload Domain Network Set B

Local Storage (2 SSDs) via Integrated Storage Controller RAID-1 for VMware ESXi 6.7 U3

HPE D3940 (SSDs/HDDs) via SAS Mezz 1 Storage Controller Cache_Tier (SSD), Capacity Tier (SSD/HDD)
*Configure Capacity and Cache Tiers in Logical JBODs in HPE OneView in the Storage tab

VMware Cloud Foundation installation on HPE Synergy


Initiate VMware Cloud Foundation deployment across the four management nodes via VMware Cloud Builder VM and the VCF deployment
parameter sheet. For more details, refer to the VMware Cloud Foundation deployment guide at https://docs.vmware.com/en/VMware-Cloud-
Foundation/3.9/com.vmware.vcf.ovdeploy.doc_39/GUID-F2DCF1B2-4EF6-444E-80BA-8F529A6D0725.html.
After the VMware Cloud Foundation is deployed successfully, log in to the SDDC Manager via browser https://<SDDC_Manager_IP>.

Demonstration of HPE storage solutions for VMware Cloud Foundation workload domain
VMware Cloud Foundation supports block storage though IP based (iSCSI) external storage or FC based external storage enabling addition of
principal and ancillary storage options for virtual infrastructure (VI) workload domain. The datastores provided through external storage system
is connected either through iSCSI or FC protocol and managed independently through the workload domain vCenter instance.
This Reference Architecture describes the deployment of HPE Primera, HPE 3PAR StoreServ, and HPE Nimble Storage as a principal storage
and as an ancillary storage to a VMware Cloud Foundation v3.9.1 VI workload domain on the HPE Synergy hardware platform.
HPE storage choices for virtual infrastructure (VI) workload domain without vSAN
Starting with VCF 3.9.x, workload domain can deployed on block storages. Workload domain deployed on block storage (VMFS on FC) is termed
as principal storage. The term ancillary storage refers to the next storage tier that is managed out of band either through storage specific GUI or
vCenter plugin. Table 13 represents the choice of options tested as part of this Reference Architecture.

Note
If HPE D9340 (vSAN) storage module is not part of the VCF workload domain deployment, HPE OneView connector cannot be used to compose
servers for workload domain. Instead use HPE OneView to compose the servers with the desired HPE storage.
Reference Architecture Page 24

Table 13. List of HPE storages validated for VCF workload domain without vSAN

HPE Storages for VCF Workload Domain Principal Storage Ancillary Storage

3PAR FC  

3PAR Direct Attach  

Nimble FC  

Nimble Direct Attach  

Nimble iSCSI NA 

Primera FC  

HPE storage choices for virtual infrastructure (VI) workload domain with vSAN (HPE D3940)
If you choose to deploy workload domain on vSAN, following options are available as ancillary storages to the workload domain and are managed
out of band either through storage specific GUI or vCenter plugin.
Table 14. List of HPE storages validated for VCF workload domain with vSAN
HPE Storages for VCF Workload Domain with vSAN
(HPE D3940)

3PAR FC 

3PAR Direct Attach 

Nimble FC 

Nimble Direct Attach 

Nimble iSCSI 

Primera FC 

Note
For management domains, vSAN is the only storage option that may be assigned.

HPE Primera storage for virtual infrastructure (VI) workload domain


HPE Primera is an intelligent storage technology in the form of all-flash or flash / spinning media converged architecture. It provides principal
storage as well ancillary storage options for the virtual infrastructure (VI) workload domains. The volumes created from HPE Primera storage,
connected to an HPE Synergy chassis through fibre channel (FC) connections, can be used to create ‘VMFS on FC’ datastores for virtual
infrastructure (VI) workload domain installations. Other volumes created from HPE Primera storage can be presented as ancillary storage options
for the workload domain.
Reference Architecture Page 25

Figure 13 shows the block diagrams for HPE Primera storage options for VI workload domains.

vCenter NSX vCenter vCenter NSX NSX


server/ Controll server/ server/ Control VM VM VM Control VM VM VM
PSC ers PSC PSC lers lers

Management domain Workload domain 1 Workload domain 2

vSAN storage HPE Primera Storage Array


vSAN storage
HPE Primera Storage Array HPE Primera Storage Array

HPE Synergy Frame 1 HPE Synergy Frame 2

HPE Primera 4 Node Storage

Figure 13. HPE Primera Storage as principal storage for VMware Cloud Foundation v3.9.1 workload domains

Note
When HPE Primera is the only choice of storage for workload domain, compose servers using HPE OneView and not via HPE OneView connector
for VCF. However, if you choose to set up your workload domain with D3940 (vSAN) along with Primera, compose/decompose of servers could
be performed using HPE OneView connector for VCF via SDDC Manager.
Reference Architecture Page 26

HPE Primera Fabric-Attach Fibre Channel (FC) topology


Figure 14 shows a fabric-attached SAN networking using SN6600B 32Gb fibre channel switches, HPE Synergy with 32Gb uplinks, and an HPE
Primera A630. For downlink connectivity, HPE Synergy 5830C 32 Gb fibre channel host bus adapters are used. Hewlett Packard Enterprise
recommends cabling odd numbered Primera ports to one switch and even numbered ports to the partner switch. At the minimum, two ports per
component must be used to enable continuous server operation during the failure of a switch, fibre cable, or server fibre port and to enable
continuous operation during a firmware updates to SAN switches.

Figure 14. SAN network topology with HPE Primera storage example

For more information on the deployment of HPE Primera as a principal storage to VMware Cloud Foundation workload domain, see
http://h20195.www2.hpe.com/V2/GetDocument.aspx?docname=a50001409enw.
HPE 3PAR StoreServ for virtual infrastructure (VI) workload domain
VMware Cloud Foundation VI workload domains can be integrated with HPE 3PAR StoreServ for various workloads and applications. VMware
Cloud Foundation workload domain on HPE Synergy can have HPE 3PAR StoreServ integrated as a principal storage and as an ancillary storage
through Fibre Channel (FC) protocol in either of the following ways:
• HPE 3PAR Fabric-Attach Fibre Channel (FC) topology
• HPE 3PAR Direct-Attached topology

In this solution, HPE 3PAR 8450 4-Node external storage was configured to provide FC datastores as principal storage to VMware Cloud
Foundation VI workload domains through Fabric-Attach Fibre Channel topology and Direct-Attach topology. HPE Virtual Connect SE 32Gb FC
Reference Architecture Page 27

Module for HPE Synergy was mounted on the Interconnect Module Bay Set 2 (Interconnect Bay 2 and Interconnect Bay 5). HPE Virtual Connect
SE 32Gb FC Module was connected to two HPE StoreFabric SN6600B Fibre Channel Switch as Storage Area Network (SAN) Switch. The same
configuration can be used to provide ancillary storage to VCF workload domain servers.
Figure 15 shows HPE 3PAR StoreServ as both principal and as an ancillary storage for virtual infrastructure (VI) workload domain.

vCenter NSX vCenter vCenter NSX NSX Principal Storage


server/ Controll server/ server/ Control VM VM VM Control VM VM VM
PSC ers PSC PSC lers lers Ancillary Storage

Management domain Workload domain 1 Workload domain 2

vSAN storage HPE 3PAR Storage Array


vSAN storage
HPE 3PAR Storage Array HPE 3PAR Storage Array

0 11 23
3PAR
StoreServ
12 8450

0 11 23
3PAR
StoreServ
12 8450

Figure 15. HPE 3PAR StoreServ as both principal and as an ancillary storage for VMware Cloud Foundation v3.9.1 workload domain

Note
When HPE 3PAR is the only choice of storage for workload domain, compose servers using HPE OneView and not via HPE OneView connector
for VCF. However, if you choose to set up your workload domain with D3940 (vSAN) along with HPE 3PAR, compose/decompose of servers
could be performed using HPE OneView connector for VCF via SDDC Manager.
Reference Architecture Page 28

HPE 3PAR Fabric-Attach Fibre Channel (FC) topology


HPE 3PAR storage with a dedicated 32Gb Fibre Channel topology provides better storage and networking performance for HPE Synergy
compute modules. HPE OneView is used to provision, manage, and administer the HPE 3PAR StoreServ thus providing a single unified
management pane for storage operations for the virtual infrastructure (VI) workload domain.
Figure 16 shows the cabling of HPE 3PAR StoreServ to SAN fabrics. It is recommended to connect all the odd numbered ports in the storage to
one SAN fabric and the even numbered ports to another. At the minimum, you must have two ports per component to enable continued server
operation during the failure of a switch, fibre cable, or server fibre port and to enable continued operation during a firmware update to the SAN
fabrics in the solution.

Figure 16. SAN network topology with HPE 3PAR storage example
Reference Architecture Page 29

HPE 3PAR Direct-Attach topology


In the HPE 3PAR Direct-Attach configuration, the storage for VMware Cloud Foundation workload domain is provided by direct cabling of HPE
Virtual Connect 40Gb SE for HPE Synergy module to HPE 3PAR. Each HPE Virtual Connect for HPE Synergy module must be connected to at
least two separate controllers in the same array. A minimum of two ports are required per component to allow the continued server operation
during controller failure and during firmware updates.
Figure 17 describes HPE 3PAR StoreServ cabling in the Direct-Attach topology.

Figure 17. HPE 3PAR StoreServ as direct-attach topology example

For more information on the deployment of HPE 3PAR StoreServ as a principal and as an ancillary storage to VMware Cloud Foundation
workload domain, refer to http://h20195.www2.hpe.com/V2/GetDocument.aspx?docname=a50001408enw.
Reference Architecture Page 30

HPE Nimble Storage for VMware Cloud Foundation v3.9.1 workload domain
HPE Nimble Storage is an intelligent, self-managing storage technology in form of all flash and hybrid flash. It provides principal storage as well
ancillary storage options for the VMware Cloud Foundation (VCF) 3.9.1 workload domain. The volumes created from HPE Nimble Storage,
connected to HPE Synergy through Fabric-Attached Fibre Channel (FC) connection or Direct-Attached Fibre Channel (FC) connection, can be
used for VMFS on FC datastore to install VCF workload domain. Volumes created from HPE Nimble Storage can be presented as principal or
ancillary storage options for VCF workload load domain.
Figure 18 shows the block diagram for HPE Nimble Storage options for VMware Cloud Foundation.
• HPE Nimble Fabric-Attach Fibre Channel (FC) topology
• HPE Nimble Direct-Attached topology
• HPE Nimble Storage iSCSI topology

vCenter NSX vCenter vCenter


server/ Controll server/ server/
NSX NSX Principal Storage
Control VM VM VM Control VM VM VM
PSC ers PSC PSC lers lers
Ancillary Storage

Management domain Workload domain 1 Workload domain 2

vSAN storage HPE Nimble Storage Array


vSAN storage
HPE Nimble Storage Array HPE Nimble Storage Array

HPE Synergy Frame 1 HPE Synergy Frame 2

Nimble Storage

HPE Nimble Storage

Figure 18. HPE Nimble Storage HF20 as ancillary storage for VMware Cloud Foundation v3.8.1 workload domain

Note
When HPE Nimble is the only choice of storage for workload domain, compose the servers using HPE OneView and not via HPE OneView
connector for VCF. However, if you choose to set up your workload domain with D3940 (vSAN) along with HPE Nimble, compose/decompose of
servers could be performed using HPE OneView connector for VCF via SDDC Manager.
Reference Architecture Page 31

HPE Nimble Storage Fabric-Attach Fibre Channel (FC) topology


The HPE Nimble Storage 2x 16Gb Fibre Channel 2-port adapter kit was added to the HPE Nimble Storage HF20 Base Array for FC connectivity.
The HPE Virtual Connect SE 16Gb FC module for Synergy was mounted on the Interconnect Module Bay Set 2 (Interconnect Bay 2 and
Interconnect Bay 5) of the HPE Synergy 12000 Frame. The HPE Virtual Connect SE 16Gb FC module was connected to two HPE StoreFabric
SN6600B Fibre Channel Switches as a Storage Area Network (SAN) Switch.
Figure 19 shows the cabling of HPE Nimble Storage to SAN fabrics. All the odd numbered Fibre Channel ports in 2x16Gb Fibre Channel 2-port
adapter of the HPE Nimble Storage is connected to one SAN fabric and the even numbered ports to second SAN fabric. This configuration
ensures continued server operation during the failure of a switch, fibre cable, or server fibre port and to enable continued operation during a
firmware update to the SAN fabrics in the solution.

UID UID

iLO iLO

Synergy Synergy
480 480
Gen10 Gen10

HPE Synergy 480 Gen10 Compute


Modules

UID Q1 Q2 1 2 3 4 5 6 7 8 Q3 Q4 UID Q1 Q2 1 2 3 4 5 6 7 8 Q3 Q4
1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2
3 4 3 4 3 4 3 4 3 4 3 4 3 4 3 4
HPE VC SE 16Gb Reset HPE VC SE 16Gb Reset
FC Module FC Module

HPE Synergy Virtual Connect SE HPE Synergy Virtual Connect SE


16GB FCModule InterConnect 16GB FCModule InterConnect
Module Bay 5 Module Bay 2
48 49 50 51 56 57 58 59 48 49 50 51 56 57 58 59

0 2 0 2

1 3 1 3

HPE SN6600B 0 4 1 5 2 6 3 7 8 12 9 13 10 14 11 15 16 20 17 21 18 22 19 23 24 28 25 29 26 30 27 31 32 36 33 37 34 38 35 39 40 44 41 45 42 46 43 47 52 53 54 55 60 61 62 63 HPE SN6600B 0 4 1 5 2 6 3 7 8 12 9 13 10 14 11 15 16 20 17 21 18 22 19 23 24 28 25 29 26 30 27 31 32 36 33 37 34 38 35 39 40 44 41 45 42 46 43 47 52 53 54 55 60 61 62 63


FC Switch FC Switch

HPE StoreFabric SN6600B Fibre Channel Switch

HPE StoreFabric SN6600B Fibre Channel Switch

IN

OUT

IN
R

R
X

X
T

T
PORT 0

PORT 1

FC16

PORT 0

PORT 1

FC16

OUT

2*16Gb Fibre Channel 2 Port


Adapter Nimble Storage HF20 Adaptive Array

Figure 19. Cabling Details for HPE Nimble Storage Fabric Attach FC topology storage solution
Reference Architecture Page 32

HPE Nimble Storage Direct-Attach topology


Figure 20 shows the cabling of HPE Nimble Storage connected directly to HPE Synergy Frames. In a two frame HPE Synergy solution, the HPE
Nimble Storage is connected directly to port 3 of Interconnect Module (ICM) located in Bay 3 for HPE Synergy Frame 1 and Port 3 of
Interconnect Module located in Bay 6 for HPE Synergy Frame 2. In a single frame HPE Synergy solution, the Interconnect Module would be in
Bay 3 and Bay 6. The Interconnect Module (ICM) used is HPE Synergy Virtual Connect SE 40 Gb F8 Module.

Figure 20. HPE Nimble Storage Direct-Attached Fibre Channel (FC) topology
Reference Architecture Page 33

HPE Nimble Storage iSCSI topology


Figure 21 shows the cabling diagram of HPE Nimble Storage (iSCSI) with HPE Synergy. The cabling shows that the HPE Synergy 12000 Frame
is connected to VCF management network on HPE Synergy Frame through eth0a and eth0b on each HPE Nimble Storage controller via the HPE
Synergy Virtual Connect SE 40GB Interconnect Module. HPE Nimble Storage is configured to use two different networks, management network
of speed 1Gbps and data network of speed 10Gbps.

Figure 21. Cabling diagram of HPE Nimble Storage (iSCSI) with HPE Synergy

HPE OneView is used to provision, manage, and administer the HPE Nimble Storage.
For more information on the deployment of HPE Nimble Storage as an ancillary storage to VMware Cloud Foundation workload domain, refer to
http://h20195.www2.hpe.com/V2/GetDocument.aspx?docname=a50001414enw.
Refer the HPE Nimble Storage documentation to understand the best practices of configuring HPE Nimble Storage at
https://infosight.nimblestorage.com/InfoSight/app#documentation (requires HPE Nimble InfoSight login).
Reference Architecture Page 34

Demonstrate VMware Enterprise PKS as a cloud-native platform on HPE Synergy


The Reference Architecture illustrates the deployment of VMware Cloud Foundation NSX-T Workload Domain for VMware Enterprise PKS as the
platform for cloud-native applications. This document highlights the implementation of VMware NSX-T as software-defined networking
component on HPE Synergy Composable Infrastructure. VMware Cloud Foundation by default provides a software-defined data center (SDDC)
architecture by virtualizing compute, storage and network. Using the SDDC Manager, VMware Enterprise PKS is deployed as a platform on the
VMware Cloud Foundation NSX-T workload domain for organizations to deploy cloud-native applications.
HPE Synergy comfortably maps all its native components to the VMware SDDC model thus making it an ideal fit for VMware SDDC deployments.
HPE Synergy Storage Module D3940 hosts software-defined VMware vSAN storage. HPE Synergy Virtual Connect SE F8 Networking Module
along with HPE FlexFabric 5940 Switch provides logical networking components for VMware NSX-T that is a next-generation software–defined
networking solution. HPE Synergy powered by HPE OneView composes and recomposes Server Profiles comprising of compute, storage and
network for VMware vSphere environments.
The following are the deployment steps to configure VMware Cloud Foundation NSX-T workload domain and VMware Enterprise PKS on HPE
Synergy.
Deploy and configure a VMware NSX-T workload domain on HPE Synergy.
Deploy and configure a VMware NSX-T Edge on the VMware Cloud Foundation workload domain for north-south communication.
Configure the VMware NSX-T Routers for East-West and north-south communication from the data center.
Configure and create overlay networks and logical switches for the VMware Enterprise PKS environment.
Deploy and configure a VMware Enterprise PKS as a platform on the VMware Cloud Foundation workload domain.
Reference Architecture Page 35

Figure 22 shows the HPE Synergy master satellite connections for HPE Synergy Frame 1 and 2. HPE Synergy Frame 3 will comprise of two
satellite modules that will in turn connect to the master modules in Frame 1 and Frame 2 completing the network loop. For detailed
understanding, refer to https://techlibrary.hpe.com/docs/synergy/shared/cabling/GUID-322C6B82-3183-42E6-A01D-B52BE4CF77C7.html.

Figure 22. Network Layout for VMware Enterprise PKS and VMware NSX-T on HPE Synergy
Reference Architecture Page 36

VMware NSX-T architecture


VMware NSX-T is designed to address application frameworks and architectures that have heterogeneous endpoints and technology stacks. In
addition to vSphere, these environments may include other hypervisors, containers, bare metal, and public clouds. NSX-T allows IT and
development teams to choose the technologies best suited for their particular applications. NSX-T is also designed for management, operations,
and consumption by development organizations in addition use by IT. Figure 23 shows the VMware NSX-T architecture.

Figure 23. VMware NSX-T architecture


Reference Architecture Page 37

VMware Enterprise PKS Architecture


VMware Enterprise PKS builds on Kubernetes, BOSH, VMware NSX-T, and project Harbor to form a production-grade, highly-available container
runtime that operates on vSphere and public clouds. With built-in intelligence and integration, VMware Enterprise PKS ties all these open source
and commercial modules together, delivering a simple-to-use product for customers, ensuring the customers have the most efficient Kubernetes
deployment and management experience possible. Figure 24 shows VMware Enterprise PKS architecture.

Figure 24. VMware Enterprise PKS architecture

For more information on the deployment of VMware Enterprise PKS, refer to


https://h20195.www2.hpe.com/V2/GetDocument.aspx?docname=a50000685enw.
Reference Architecture Page 38

Demonstrate integrated composability with VCF and HPE Synergy using HPE OneView
connector for VCF
VMware introduced composability service component starting from VCF 3.5 within the SDDC Manager. This composability service communicates
with HPE OneView via the HPE OneView connector for VCF. This HPE OneView connector for VCF is developed and will be managed by Hewlett
Packard Enterprise and is powered by DMTF Redfish APIs, which is designed to deliver simple and secure management for SDDC. Technically, it
acts as a translator for VCF SDDC Manager and HPE OneView.
HPE OneView connector for VCF 0.3.3
HPE OneView connector for VCF 0.3.3 enables composability feature to provide a pure cloud experience in provisioning and deprovisioning of
composable resources (compute, storage, and networking resources of HPE Synergy). In this version, the support is extended for external
storages – HPE 3PAR StoreServ and HPE Nimble Storage operating on both FC and iSCSI storage protocols. The volume templates of these
storage are attached to the Server Profile templates in HPE OneView. Based on the need, the user may choose the required Server Profile
template while composing the composable resources from the SDDC Manager.

Figure 25: Workflow of HPE OneView connector for VCF


Reference Architecture Page 39

The connector is to be installed on a Linux instance (virtual or physical) out-of-band to VCF as a prerequisite within the management domain of
VMware Cloud Foundation. Details about step-by-step installation, prerequisites and other recommended configurations for HPE OneView
connector for VCF can be found at, http://h20195.www2.hpe.com/V2/GetDocument.aspx?docname=a50001374enw.

Note
If HPE D9340 (vSAN) storage module is part of the VCF workload domain deployment, compose the servers using HPE OneView connector
through SDDC Manager as shown in this section.

Once the installation is complete, we need to register HPE OneView connector for VCF within SDDC Manager. To know more about registering
HPE OneView connector for VCF with SDDC Manager and to compose servers on HPE Synergy using HPE OneView connector for VCF, reader
may refer to the installation document at https://h20195.www2.hpe.com/V2/GetDocument.aspx?docname=a50001374enw.

Demonstrate patching and upgrading in VMware Cloud Foundation


SDDC Manager internally has Lifecycle Management (LCM) enabled, which performs automated updates on VMware Cloud Foundation
components such as SDDC Manager and its internal services and VMware components such as NSX for vSphere, vCenter Server, VMware ESXi™,
Platform Services Controller (PSC), vRealize Suite, NSX-T, and VMware vRealize Suite Lifecycle Manager™. SDDC Manager is configured to
communicate with the VMware software repository, if the SDDC Manager VM has internet access and the VMware depot credentials are valid.
The high-level update workflow is as follows:
Receive notification of update availability.
Download the update bundle.
Select update targets and schedule the update.

However, the intent of this section is to help administrators to understand how to perform VMware Cloud Foundation upgrade when SDDC
Manager does not have access to the internet.
Offline VMware Cloud Foundation update
The intent of this section is to describe how to upgrade your VMware Cloud Foundation system if the SDDC Manager VM does not have internet
access. The document goes in detail of how to use the Bundle Transfer utility to manually download the bundles from the VMware depot on your
local computer with internet access and then upload them to an SDDC Manager VM, and update your VMware Cloud Foundation system.

When to perform offline update


VMware Cloud Foundation update needs to be performed only after verifying if the underlying HPE Infrastructure including drivers and firmware
are compatible with the version that is going to be installed. Refer to the HPE Synergy firmware and software compatibility matrix for VMware
Cloud Foundation guide at https://h20195.www2.hpe.com/V2/GetDocument.aspx?docname=a50001407enw to check if the VCF version is
listed as compatible along with the drivers and firmware.
Bundle types
Upgrade bundle
An upgrade bundle contains bits to update the appropriate Cloud Foundation software components in your management domain or VI workload
domain. In most cases, an upgrade bundle must be applied to the management domain before it can be applied to workload domain.
Some upgrade bundles are cumulative bundles. With a cumulative upgrade bundle, you can directly upgrade the appropriate software in your
workload domain to the version contained in the cumulative bundle rather than applying sequential upgrades to reach the target version.
Cumulative bundles are available only for vCenter Server, Platform Services Controller, and ESXi.

Note
You can apply a cumulative bundle to a workload domain only if the target release in the bundle is lower than or at the same version as the
management domain. If the cumulative bundle is available for both the management domain and VI workload domains, you must apply it to the
management domain before applying it to VI workload domains.
Reference Architecture Page 40

Install bundle
VMware Cloud Foundation includes the following install bundles:
• VI workload domain install bundle is used to deploy later versions of the software components instead of the versions in your original Cloud
Foundation installation. It includes software bits for vCenter Server and NSX for vSphere.
• Individual install bundles for vRealize products are used for deploying vRealize components.
• NSX-T install bundle is used for deploying an NSX-T based VI workload domain.
• Horizon 7 install bundle is used for creating a Horizon domain.

For more information on upgrading VMware Cloud Foundation (VCF) on HPE Synergy, refer to https://docs.vmware.com/en/VMware-Cloud-
Foundation/3.9/vcf-39-upgrade/GUID-7992A016-D5E3-4A31-8337-A0611B355C41.html.

Demonstrate stretching a vSAN cluster of a VMware Cloud Foundation management domain


VMware vSAN stretched clusters allow stretching of a single data site to two sites for a higher level of availability and inter-site load balancing.
This Reference Architecture tells about protecting VMware Cloud Foundation management domain, a special purpose workload domain
dedicated to management tasks, which is crucial to business. It contains the following management components:
• SDDC Manager
• vCenter Server and Platform Services Controllers
• vRealize Log Insight
• NSX Manager and controllers

VMware Cloud Foundation management domain is brought up on a single site with no protection or avoidance from disaster and a single point
of failure could disrupt application availability. Thus, it becomes important to protect the management domain to ensure business continuity.
To address this issue, we could take advantage of the disaster avoidance and protection solution that vSAN offers in the form of stretched
cluster. A vSAN stretched cluster is a specific configuration implemented when disaster/downtime has to be avoided in any situation where
business continuity is a critical requirement. The vSAN stretched cluster extends the cluster from one data site to two sites for high availability
and load balancing.
Use cases for implementing stretched cluster for VMware Cloud Foundation management domain are:
• Planned maintenance: Perform a planned maintenance on an availability zone without any downtime and then migrate the applications after
the maintenance is completed.
• Automated recovery: Stretching a cluster automatically initiates VM restart and recovery and has a low recovery time objective for most
unplanned failures.
• Disaster avoidance: With a stretched cluster, you can prevent service outages before an impending disaster.
Reference Architecture Page 41

VMware Cloud Foundation management domain is stretched across two sites (VCF Site1 and VCF Site2) within the region, so the network
latency between two sites should be minimum. A third site (vSAN witness Site3) contains a witness host separate from the location of the other
two data sites. Figure 35 shows architecture of VMware Cloud Foundation stretched management domain vSAN cluster across two sites in an
Active/Active configuration.

Figure 35. Stretched VMware Cloud Foundation management domain vSAN cluster

For more information on stretching a vSAN cluster of a VMware Cloud Foundation management domain, refer to
https://docs.vmware.com/en/VMware-Cloud-Foundation/3.9/com.vmware.vcf.admin.doc_39/GUID-7B4CC729-20BD-4CC9-B855-
B38F02F74D40.html.

Demonstrate ease of monitoring and reporting of VCF infrastructure using HPE OneView for
vRealize Operations
HPE OneView for VMware vRealize Operations provides an integrated monitoring and reporting tool for VCF infrastructure.
When the HPE OneView for VMware vRealize Operations is installed, the custom HPE OneView dashboards are added to the vRealize
Operations custom GUI. The HPE OneView dashboard allows you to proactively monitor HPE Synergy hardware resources and shows the object
relationship with other objects in the environments. Proactive monitoring of the HPE Synergy hardware used for VMware Cloud Foundation
helps improve productivity and facilitate efficient use of the resources resulting in minimized cost.

HPE OneView for vRealize Operations can be downloaded from Software Depot 1, please download
“HPE_OneView_for_VMware_vRealize_Ops_2.3_July_2019_Z7550-02544.zip” file and extract the “HPEOneViewAdapter_2.3.0.21_signed.pak”.
The details about the installation and configurations can be found at, https://support.hpe.com/hpsc/doc/public/display?docId=a00077762en_us.

1
HPE OneView for vRealize Operations: https://h20392.www2.hpe.com/portal/swdepot/displayProductInfo.do?productNumber=Z7500-63235
Reference Architecture Page 42

Following are the dashboards available for use:


• HPE OneView Infrastructure dashboard
• HPE OneView networking dashboard
• HPE OneView Servers overview dashboard
• HPE OneView Enclosure overview dashboard
• HPE OneView Uplink Port overview dashboard

The following sections show examples of four among above five dashboards.
HPE OneView Infrastructure dashboard
The HPE OneView Infrastructure dashboard provides an overview of the HPE Synergy Infrastructure managed by HPE OneView. It displays the
status of HPE OneView managed hardware and allows you to see how the hardware relates to your virtual environment. Figure 36 shows an
example of HPE OneView Infrastructure dashboard. It displays the physical and virtual resources that are part of the VCF domain and
interrelation between each object in the environments. Administrator gets a quick summary of the environment and relations between the
objects. It also shows metric charts demonstrating the performance and usage of the resources.

Figure 36. HPE OneView Infrastructure dashboard


Reference Architecture Page 43

HPE OneView Networking dashboard


The HPE OneView Networking dashboard provides an overview of the HPE OneView networking along with its connection to the virtual
environment. Selecting an object in the environment overview allows you to see how this object is related to the other objects and generates a
graph for each metric collected. Figure 37 shows an example of HPE OneView Networking dashboard with VCF-Node1 server selected and its
connections.

Figure 37. HPE OneView Networking dashboard


Reference Architecture Page 44

HPE OneView Servers Overview dashboard


The HPE OneView Servers Overview dashboard provides a summary and metrics information pertaining to CPU utilization, temperature, and
power utilization of physical servers in the VCF environment. The heatmaps allow you to quickly compare these metrics.

Figure 38. HPE OneView Servers Overview dashboard


Reference Architecture Page 45

HPE OneView Enclosures Overview dashboard


The HPE OneView Enclosures Overview dashboard provides summary and metrics information pertaining to the temperature and power
utilization of the HPE Synergy Enclosures. The Heatmaps allow you to quickly compare these metrics. Under each Heatmap, there is a
comparative representation between the two enclosures. Selecting an enclosure from this list generates a sparkline chart displaying the metric
history.
Figure 39 shows HPE OneView Enclosures Overview dashboard with the two enclosures as used in this Reference Architecture solution.

Figure 39. HPE OneView Enclosure Overview dashboard


Reference Architecture Page 46

Demonstrate HPE Synergy firmware upgrade using HPE OneView for VMware vRealize
Orchestrator
HPE OneView for VMware vRealize Orchestrator helps customers automate complex IT tasks in an extensible and repeatable manner. It provides
a predefined collection of HPE tasks and workflows that can be used in vRealize Orchestrator (vRO). The plugin can be added to vRealize
Orchestrator seamlessly and various IT tasks can be performed using the workflows available.
Among the available workflows, administrator can use the “Update Cluster Firmware“ workflow to update the HPE Synergy Compute Firmware
through HPE OneView and HPE Synergy Service Pack for ProLiant for Day 2 operations.
Following are the steps needed to be performed to update compute firmware using the “Update Cluster Firmware” workflow.
Install HPE iSUT on each of the HPE Synergy Nodes.
Configure the VMware vRealize Orchestrator (vRO).
Install and configure the HPE OneView for vRO plugin.
Configure the vRO Clients for the workflows.

HPE OneView for vRealize Orchestrator can be downloaded from the Software Depot 1, please download the
“HPE_OneView_for_VMware_vRealize_Orchestration_1.3_October_2019_Z7550-02585.zip” file and extract the “o11npluginhpeov4vro-
1.3.0.79.vmoapp” to add that as plug-in with VMware vRealize Orchestrator. Refer to
http://h20195.www2.hpe.com/V2/GetDocument.aspx?docname=a50000681enw for more detailed information.
We tested the “Update Cluster Firmware” workflow in the Hewlett Packard Enterprise Engineering lab to update the compute resources firmware
inside the HPE Synergy Frame with VMware Cloud Foundation installed. The following sections provides details about the test environment and
the process.
Prerequisites
• VMware vRealize Orchestrator is installed in the environment.
• VMware vRealize Orchestrator vSphere vCenter plug-in is configured with VMware vCenter(s).
• At least one instance of HPE OneView is installed and configured.
• An HPE OneView account is created and to be used by HPE OneView Management Instance in VMware vRealize Orchestrator.
• The VMware vRealize Orchestrator servers have network access to HPE OneView.
• VMware Cloud Foundation software is installed and configured on HPE Synergy.
• HPE OneView Server Profile template is configured for baseline Service Pack for ProLiant (SPP).
• The vRO workflows will be used to update new SPPs on the HPE Synergy Compute Nodes as available and configure them as baselines for
Server Profile template.

1
HPE OneView for vRealize Orchestrator, https://h20392.www2.hpe.com/portal/swdepot/displayProductInfo.do?productNumber=Z7500-63235.
Reference Architecture Page 47

Firmware and software versions


Table 15 shows the firmware and software used in the testing process.
Table 15. Firmware and software versions

Name Versions

VMware vRealize Orchestrator 7.6.0.317 Build Number -13020602

HPE OneView for VMware vRealize Orchestrator(OV4VRO) 1.3

HPE Synergy Composer 5.00.02

iSUT sut-esxi6.7-offline-bundle-2.4.5.0-16

VMware Cloud Foundation 3.9.1

vCenter 6.7 U3

Testing the workflow


In the Hewlett Packard Enterprise Engineering lab environment, the following two HPE Synergy Service Packs are downloaded, as shown in
Table 16.
Table 16: Firmware used in testing

Name Version Size

HPE Synergy Custom SPP 201912 2020 01 24 2020.01.24.00 4.28 GB

HPE Synergy Custom SPP 201912 2020 03 26 2020.03.26.00 4.29 GB

In the server profile template, the Firmware baseline was selected as " HPE Synergy Custom SPP 201912 2020 01 24” as shown in Figure 40.

Figure 40. Firmware baseline in HPE OneView


Reference Architecture Page 48

The firmware of the HPE Synergy Compute Nodes was updated to firmware version “HPE Synergy Custom SPP 201912 2020 03 26” using the
workflow “Update cluster Firmware” present in OV4VRO workflows. The following information was necessary to start the “Update cluster
Firmware” workflows:
• vCenter Cluster – The vCenter Cluster on which the HPE Synergy host firmware would be upgraded as shown in Figure 41.

Figure 41. vCenter Cluster selection


Reference Architecture Page 49

• HPE OneView Firmware Bundle –The version to which the firmware update would happen as shown in Figure 42.

Figure 42. HPE OneView Firmware Bundle selection


Reference Architecture Page 50

Once the required details are updated, the “Update cluster Firmware” workflow is ready to start as shown in Figure 43.

Figure 43. Start workflow

In HPE OneView, the Server Profile status for each compute resource shows the activity details for the firmware update process as shown in
Figure 44.

Figure 44. OneView Screen showing new firmware baseline being applied
Reference Architecture Page 51

Once the Firmware update is complete, it shows the new firmware version “HPE Synergy Custom SPP 201912 2020 03 26” as the firmware
baseline as shown in Figure 45. The firmware update state shows as Applied along with the time when it was installed.

Figure 45. HPE OneView showing the new firmware applied

Note
Certain firmware components need manual reboot for the complete installation of the firmware. In such scenario, a manual reboot of the host is
recommended.
Reference Architecture Page 52

Consolidated architecture deployment of VMware Cloud Foundation


VMware Cloud Foundation can be deployed as either a standard architecture model or consolidated architecture model. In the standard
architecture model, there is a dedicated management domain that hosts infrastructure virtual machines and at least one workload domain to host
user workloads. It also requires a minimum of seven (7) servers, four (4) for management domain and three (3) for workload domain. In the
consolidated architecture model, the management and customer workload VMs are part of the same domain or cluster and it leverages vSphere
resource pools to provide isolation between management and user workloads.
The consolidated architecture model targets smaller Cloud Foundation deployments, special use cases, and can be deployed starting with four
(4) hosts or higher. Management and user workload VMs run together in the same vSphere cluster and the environment is managed from a
single vCenter. vSphere resource pools provide isolation between the management and user workloads. Consolidated architecture model does
not support NSX-T or automated deployment of Horizon and Enterprise PKS.
Figure 46 shows the VMware Cloud Foundation consolidated architecture deployed on HPE Synergy Frame hosting four to six compute servers.

Figure 46. VMware Cloud Foundation consolidated architecture deployed on HPE Synergy Frame hosting 4 – 6 compute servers
Reference Architecture Page 53

Note
VCF consolidated deployment can support up to 64 hosts (to the limits of vCenter). But this solution is tested with four to six hosts. HPE
automation also enables four to six hosts for consolidated architecture (targeting Proof-of-Concept scenario and use cases for ROBO, SMB
deployments). However, this is not the limitation by VCF SDDC for consolidated deployment.

For more information on deploying VMware Cloud Foundation consolidated architecture on HPE Synergy, see
http://h20195.www2.hpe.com/V2/GetDocument.aspx?docname=a50001373enw.

Summary
Hewlett Packard Enterprise and VMware can deliver a software-defined solution running on modular infrastructure across compute, storage,
network, security, and cloud management. This Reference Architecture demonstrates a secured and scalable private cloud solutions built using
VMware Cloud Foundation on HPE Synergy. It showcases the ability to:
• Integrate composability with VCF and HPE Synergy using HPE OneView connector for VCF.
• Simplify deployment of VI workload domain and operations on VMware Cloud Foundation.
• Simplify firmware updates using HPE OneView for VMware vRealize Orchestrator.
• Build a VCF workload domain with a choice of either HPE 3PAR, HPE Nimble, or HPE Primera for SDDC managed principal storage or out of
band managed ancillary storage as secondary choice of storage.

HPE Synergy offers a unique design for running VMware private clouds, and for providing the right IT platform that matches VMware Cloud
Foundation characteristics—automated, software-driven, and flexible. HPE Synergy is the only modular infrastructure to run VMware Cloud
Foundation deployments, and it provides a foundation for supporting hybrid configurations.
The benefits of using VMware Cloud Foundation on HPE Synergy include:
• Reduced infrastructure complexity and cost.
• Elimination of top-of-rack switching and deployment.
• Availability of rack-scale fabric with HPE Virtual Connect.
• Efficiencies in scaling fabrics across multiple frames.
• Provision and management of the physical fluid resources for SDDC deployments through HPE OneView.
Reference Architecture Page 54

Appendix A: Bill of materials

Note
Part numbers are at time of publication and subject to change. The bill of materials does not include complete support options or complete rack
and power requirements. For questions regarding ordering, consult with your HPE Reseller or HPE Sales Representative for more details.
hpe.com/us/en/services/consulting.html.

Table 17. Bill of materials

Product Qty Product description

Rack and power

P9K10A 1 HPE 42U 600mmx1200mm G2 Enterprise Shock Rack

P9K40A 001 1 HP Factory Express Base Racking Service

H6J85A 1 HPE Rack Hardware Kit

120672-B21 1 HPE Rack Ballast Kit

BW932A 1 HPE 600mm Rack Stabilizer Kit

BW932A B01 1 HPE 600mm Rack include with Complete System Stabilizer Kit

P9S21A 2 HPE G2 Metered/Switched 3Ph 14.4kVA/CS8365C 40A/208V Outlets (12) C13 (12) C19/Vertical NA/JP PDU
HPE Synergy Frames

P06011-B21 2 HPE Synergy 12000 Configure-to-order Frame with 1x Frame Link Module 10x Fans

867796-B21 2 HPE Virtual Connect SE 100Gb F32 Module for Synergy

794502-B23 2 HPE Virtual Connect SE 40Gb F8 Module for Synergy

867793-B21 2 HPE Synergy 50Gb Interconnect Link Module

779218-B21 2 HPE Synergy 20Gb Interconnect Link Module

798096-B21 2 HPE 6x 2650W Performance Hot Plug Titanium Plus FIO Power Supply Kit

872957-B21 2 HPE Synergy Composer 2

876852-B21 2 HPE Synergy 4-port Frame Link Module

804943-B21 2 HPE Synergy Frame 4x Lift Handles

859493-B21 1 HPE Synergy Multi Frame Master1 FIO

804101-B21 4 HPE Synergy Interconnect Link 3m Active Optical Cable

876259-B21 4 HPE Virtual Connect SE 32Gb Fibre Channel Module for Synergy

P08477-B21 4 HPE Virtual Connect SE 16Gb Fibre Channel Module for Synergy

755985-B21 4 HPE Synergy 12Gb SAS Connection Module with 12 Internal Ports

871940-B21 16 HPE Synergy 480 Gen10 Configure-to-order Compute Module

P07351-B21 16 HPE Synergy 480/660 Gen10 Intel Xeon-Gold 6254 (3.1GHz/18-core/200W) Processor Kit

P07351-L21 16 HPE Synergy 480/660 Gen10 Intel Xeon-Gold 6254 (3.1GHz/18-core/200W) FIO Processor Kit

P00920-B21 384 HPE 16GB (1x16GB) Single Rank x4 DDR4-2933 CAS-21-21-21 Registered Smart Memory Kit

P01367-B21 16 HPE 96W Smart Storage Battery (up to 20 Devices) with 260mm Cable Kit

804424-B21 16 HPE Smart Array P204i-c SR Gen10 (4 Internal Lanes/1GB Cache) 12G SAS Modular Controller

804428-B21 16 HPE Smart Array P416ie-m SR Gen10 (8 Int 8 Ext Lanes/2GB Cache) 12G SAS Mezzanine Controller
Reference Architecture Page 55

Product Qty Product description

P02054-B21 16 HPE Synergy 6820C 25/50Gb Converged Network Adapter

777430-B21 16 HPE Synergy 3820C 10/20Gb Converged Network Adapter

777456-B21 12 HPE Synergy 5830C 32Gb Fibre Channel Host Bus Adapter

777452-B21 12 HPE Synergy 3830C 16Gb Fibre Channel Host Bus Adapter

872475-B21 32 HPE 300GB SAS 12G Enterprise 10K SFF (2.5in) SC 3yr Wty Digitally Signed Firmware HDD

845406-B21 10 HPE 100Gb QSFP28 to QSFP28 3m Direct Attach Copper Cable

721064-B21 4 HPE BladeSystem c-Class 40G QSFP+ to 4x10G SFP+ 3m Direct Attach Copper Splitter Cable

861413-B21 4 HPE Synergy Frame Link Module CAT6A 3m Cable


Storage

835386-B21 2 HPE Synergy D3940 12Gb SAS CTO Drive Enclosure with 40 SFF (2.5in) Drive Bays

P09098-B21 12 HPE 400GB SAS 12G Write Intensive SFF (2.5in) SC 3yr Wty Digitally Signed Firmware SSD

P09092-B21 48 HPE 1.6TB SAS 12G Mixed Use SFF (2.5in) SC 3yr Wty Digitally Signed Firmware SSD

757323-B21 2 HPE Synergy D3940 Redundant I/O Adapter

H6Z00A 2 HPE 3PAR StoreServ 8000 4-port 16Gb Fibre Channel Adapter

H6Z07B 1 HPE 3PAR 8440 2N+SW Storage Field Base

K2P90B 8 HPE 3PAR 8000 920GB+SW SFF FE SSD

K2P94B 28 HPE 3PAR 8000 1.8TB+SW 10K SFF HDD

E7Y71A 2 HPE 3PAR StoreServ 8000 SFF(2.5in) Field Integrated SAS Drive Enclosure

Q2S13A 1 HPE 3PAR StoreServ RPS Service Processor

Q8H72A 1 HPE Nimble Storage HF20 Adaptive Dual Controller 10GBASE-T 2-port Configure-to-order Base Array

Q8B69B 1 HPE Nimble Storage HF20/20C Adaptive Array 42TB (21x2TB) FIO HDD Bundle

Q8G27B 1 HPE Nimble Storage NOS Default FIO Software

Q8J30A 1 HPE Nimble Storage HF20 Adaptive Array R2 5.76TB (6x960GB) FIO Cache Bundle

R0N85A 1 HPE Nimble Storage 2x10GBASE-T 2-port and 2x16Gb Fibre Channel 4-port FIO Adapter Kit

R0P84A 2 HPE Nimble Storage IEC 60320 C14 to C19 250V 15Amp 1.8m FIO Power Cord

R3P91A 1 HPE NS AF/HF OEM Trk

N9Z46A 1 HPE Primera 600 2-way Storage Base

N9Z55A 1 HPE Primera A630 2-node Controller

581817-B21 1 HPE Configurator Defined Build Instruction Option

R1P32A 1 HPE Primera 600 without Installation Service

R3R39A 6 HPE Primera 600 1.92TB SAS SFF (2.5in) FIPS Encrypted SSD

N9Z39A 2 HPE Primera 600 32Gb 4-port Fibre Channel Host Bus Adapter

716195-B21 2 HPE External 1.0m (3ft) Mini-SAS HD 4x to Mini-SAS HD 4x Cable

N9Z50A 1 HPE Primera 600 2U 24-disk SFF Drive Enclosure

R3R39A 6 HPE Primera 600 1.92TB SAS SFF (2.5in) FIPS Encrypted SSD
Reference Architecture

Resources and additional links


VMware Cloud Foundation 3.9.1, https://docs.vmware.com/en/VMware-Cloud-Foundation/3.9.1/rn/VMware-Cloud-Foundation-391-Release-
Notes.html
VMware Cloud Foundation Planning and Preparation Guide, https://docs.vmware.com/en/VMware-Cloud-Foundation/3.9/vcf-39-planprep-
guide.pdf
HPE Reference Architecture, hpe.com/info/ra
HPE Synergy, hpe.com/info/synergy
HPE and VMware, hpe.com/partners/vmware
HPE Networking, hpe.com/networking
HPE Servers, hpe.com/servers
HPE Enterprise Information Library, hpe.com/info/convergedinfrastructure
HPE Technology Consulting Services, hpe.com/us/en/services/consulting.html
HPE OneView for VMware vCenter with operation Manager and Log Insight, hpe.com/hpeoneviewforvcenter

To help us improve our documents, please provide feedback at hpe.com/contact/feedback.

© Copyright 2020 Hewlett Packard Enterprise Development LP. The information contained herein is subject to change without notice.
The only warranties for Hewlett Packard Enterprise products and services are set forth in the express warranty statements accompanying
such products and services. Nothing herein should be construed as constituting an additional warranty. Hewlett Packard Enterprise shall
not be liable for technical or editorial errors or omissions contained herein.

Intel and Xeon are trademarks of Intel Corporation in the U.S. and other countries. Microsoft, Windows, Windows Server and SQL Server
are either registered trademarks or trademarks of Microsoft Corporation in the United States and/or other countries. VMware, vSphere,
VMware Cloud Foundation, VMware vRealize Automation, VMware NSX, vRealize Orchestrator, vRealize Operations Manager, vRealize
Suite Lifecycle Manager and vRealize Log insight are registered trademarks or trademarks of VMware, Inc. in the United States and/or
other jurisdictions. VMware Horizon is trademark of VMware, Inc. in the United States and/or other jurisdictions. Linux is the registered
trademark of Linus Torvalds in the U.S. and other countries. All other third-party trademark(s) is/are the property of their respective
owner(s).

a50001399enw, April 2020