Beruflich Dokumente
Kultur Dokumente
NSX Networking
and Security
HUMAIR AHMED
GILLES CHEKROUN
NICO VIBERT
Foreword by
Tom Gillis
Senior Vice President/General Manager
– NSBU, VMware
VMware Cloud on AWS:
NSX Networking
and Security
HUMAIR AHMED
GILLES CHEKROUN
NICO VIBERT
Foreword by
Tom Gillis
Senior Vice President/General Manager
– NSBU, VMware
VMWARE PRESS
Technical Writer
Rob Greanias
Design Agency
Mitchell Design
Foreword................................................................................................. XVII
Chapter 1 - Introduction..................................................................................1
| IV
Chapter 6 - NSX Security in VMware Cloud on AWS SDDC........................... 83
Role Based Access Control (RBAC)........................................................ 84
Grouping Objects.................................................................................. 85
Groups Based on IP Address................................................................ 85
Groups Based on VM Instance.............................................................. 86
Groups Based on VM Name.................................................................. 86
Groups Based on Security Tags..............................................................87
Management Groups and Workload Groups.......................................... 90
Edge Firewall........................................................................................ 92
Distributed Firewall (DFW).................................................................... 94
Distributed Firewall Concepts................................................................94
Distributed Firewall on VMware Cloud on AWS...................................... 96
Distributed Firewall Sections................................................................. 97
Sections and Rules..............................................................................100
V |
NSX-T Proxy URL................................................................................. 124
Python Example to Retrieve the NSX-T Reverse Proxy URL...................124
Network Segments Example................................................................ 125
Management Gateway Firewall Rules.................................................... 126
Compute Gateway Firewall Rules.......................................................... 127
SDDC vSphere APIs............................................................................. 128
Automation with PowerShell and PowerCLI.......................................... 129
What is PowerCLI?...............................................................................130
Install PowerCLI...................................................................................130
Install VMware Cloud on AWS Module..................................................130
Install the NSX-T Module.......................................................................131
VMware Cloud on AWS PowerCLI Functions..........................................131
Start Using PowerCLI for VMware Cloud on AWS.................................. 132
Summary...................................................................................................154
Index.........................................................................................................158
| VI
List of Figures
Figure 1 VMware Cloud on AWS Solution........................................................... 2
Figure 2 VMware Cloud on AWS Underlay and Infrastructure............................. 2
Figure 3 Deploying VMware Cloud on AWS SDDC............................................. 3
Figure 4 Deployed VMware Cloud on AWS SDDCs............................................ 3
Figure 5 VMware Cloud on AWS Connectivity.................................................... 4
Figure 6 Data Center Extension Use Case.......................................................... 8
Figure 7 VMware Cloud on AWS - Data Center Extension.................................. 9
Figure 8 Migration Use Case..............................................................................10
Figure 9 VMware Cloud on AWS - Migration......................................................11
Figure 10 Disaster Recovery Use Case................................................................11
Figure 11 VMware Cloud on AWS - Disaster Recovery....................................... 12
Figure 12 NSX Features in VMware Cloud on AWS............................................ 14
Figure 13 Deploying VMware Cloud on AWS SDDC - Subnet Selection.............18
Figure 14 VMware Cloud on AWS SDDC - MGW and CGW...............................19
Figure 15 vSphere Client Accessing a VMware Cloud on AWS SDDC...............20
Figure 16 VMware Cloud on AWS SDDC........................................................... 21
Figure 17 Deployed VMware Cloud on AWS SDDCs......................................... 23
Figure 18 Deployed VMware Cloud on AWS SDDC.......................................... 23
Figure 19 VMware Cloud on AWS SDDC - Networking & Security Tab.............. 24
Figure 20 VMware Cloud on AWS Solution Design and Capabilities................. 25
Figure 21 VMware Cloud on AWS SDDC - Settings Tab.................................... 26
Figure 22 Default Firewall Rules on MGW Firewall.............................................27
Figure 23 MGW Firewall Rule Allowing vCenter Access Over Internet................27
Figure 24 vCenter Access Over Internet............................................................ 28
Figure 25 vCenter Access Over Direct Connect Private VIF.............................. 28
Figure 26 Changing DNS Resolution to Private IP under Settings Tab.............. 29
Figure 27 MGW Firewall Rule Allowing vCenter Access for Specific Workloads
Over Direct Connect Private VIF................................................... 29
Figure 28 NSX Distributed Routing...................................................................30
Figure 29 Active/Standby NSX Edge Appliances............................................... 31
Figure 30 VMware Cloud on AWS SDDC.......................................................... 31
Figure 31 NSX Edge Connectivity...................................................................... 32
Figure 32 NSX Network Segments................................................................... 33
Figure 33 NSX Network Segment Types........................................................... 33
Figure 34 L2VPN Tunnel Needed Before Creation of Extended Network......... 34
Figure 35 Creating a Disconnected Network.................................................... 34
Figure 36 Enabling NSX DHCP......................................................................... 35
Figure 37 Creating NSX Network Segment Using NSX DHCP.......................... 35
Figure 38 NSX DHCP Relay.............................................................................. 36
VII |
Figure 39 NSX DHCP Relay Configuration........................................................ 36
Figure 40 NSX DNS...........................................................................................37
Figure 41 Multiple DNS Zones for CGW DNS.....................................................37
Figure 42 Changing DNS Resolution to Private IP under Settings Tab.............. 38
Figure 43 CGW Source NAT Public IP............................................................... 39
Figure 44 Created NSX Network Segments......................................................40
Figure 45 Compute Traffic Flow for Internet Access..........................................40
Figure 46 Requesting Public IP......................................................................... 41
Figure 47 Allocated Public IP............................................................................ 42
Figure 48 NSX NAT Configuration.................................................................... 42
Figure 49 Traffic Flow for Incoming Traffic from the Internet to Compute.......... 43
Figure 50 Traffic Flow for Outgoing Traffic from the Compute to Internet.......... 43
Figure 51 CGW Firewall Behavior Configuration................................................ 44
Figure 52 Applied CGW Firewall Rule on Internet Interface.............................. 44
Figure 53 Hybrid Cloud Design with Direct Connect......................................... 45
Figure 54 Hybrid Cloud Design with Direct Connect and Customer Router
at DX Location............................................................................... 45
Figure 55 Direct Connect Private VIF Design.................................................... 48
Figure 56 Direct Connect Private VIF Setup..................................................... 49
Figure 57 VMware Cloud on AWS SDDC - Direct Connect Private VIF
Configuration................................................................................50
Figure 58 Direct Connect Private VIF - Editing BGP Local ASN......................... 51
Figure 59 Encrypting Traffic with Route Based IPSEC VPN
Over Direct Connect Private VIF................................................... 52
Figure 60 Direct Connect Private VIF - Configuring VPN as Backup................ 53
Figure 61 Direct Connect Private VIF and VPN as Backup Design.................... 53
Figure 62 Direct Connect Private VIF - BGP Behavior...................................... 54
Figure 63 Direct Connect Private VIF - Advertised and Learned Routes........... 55
Figure 64 Direct Connect Private VIF - Advertising Default Route from
On Premises.................................................................................. 56
Figure 65 Route Based IPSEC VPN Design...................................................... 56
Figure 66 Public and Private IP Address for Route Based IPSEC VPN.............. 57
Figure 67 Route Based IPSEC VPN Configuration............................................ 57
Figure 68 Configuring BGP Local and Remote IP/Subnet................................ 58
Figure 69 Redundancy with Route Based IPSEC VPN Using Two BGP Local
IPs/VTIs........................................................................................ 59
Figure 70 Redundancy with Route Based IPSEC VPN Using Hardware
Cluster On Premises...................................................................... 59
Figure 71 Network Advertisements via Route Based IPSEC VPN from SDDC
to On Premises..............................................................................60
| VIII
Figure 72 Route Based IPSEC VPN Connectivity to Different
Endpoints/Devices........................................................................60
Figure 73 Route Based IPSEC VPN - DOWNLOAD CONFIG............................61
Figure 74 Route Based IPSEC VPN - VIEW STATISTICS....................................61
Figure 75 Route Based IPSEC VPN - ADVERTISED ROUTES.......................... 62
Figure 76 Route Based IPSEC VPN - LEARNED ROUTES................................ 62
Figure 77 Policy Based IPSEC Configuration.................................................... 63
Figure 78 AWS Transit Gateway Design............................................................ 64
Figure 79 AWS Transit Gateway Design - Shared Connectivity to
On Premises.................................................................................. 65
Figure 80 AWS Transit Gateway Design - Direct Connect Private VIF
Used by Spoke SDDC...................................................................66
Figure 81 AWS Console - Deployed Transit Gateway........................................ 67
Figure 82 AWS Console - Transit Gateway Attachments................................... 67
Figure 83 AWS Console - Transit Gateway VPN Established to VMware Cloud
on AWS SDDC 1............................................................................68
Figure 84 AWS Console - Transit Gateway VPN Established to VMware Cloud
on AWS SDDC 2...........................................................................68
Figure 85 AWS Console - AWS VPC 1 Route Table...........................................69
Figure 86 AWS Console - AWS VPC 2 Route Table..........................................69
Figure 87 AWS Console - Transit Gateway Route Table..................................... 70
Figure 88 VMware Cloud on AWS SDDC Route Based IPSEC Configuration
to TGW.......................................................................................... 70
Figure 89 VMware Cloud on AWS SDDC Learned Routes from TGW................ 71
Figure 90 Route Based IPSEC VPN ECMP Design with TGW............................ 71
Figure 91 AWS Console - AWS Resource Access Manager................................72
Figure 92 AWS Console - Accepting Resource Share....................................... 73
Figure 93 AWS Console - Transit Gateway Route Table Showing
New Learned Route...................................................................... 73
Figure 94 L2VPN over Direct Connect Private VIF Design.................................74
Figure 95 Extending Networks to VMware Cloud on AWS SDDC with L2VPN
and No NSX On Premises..............................................................74
Figure 96 Created L2VPN Server and Extended Network in VMware Cloud
on AWS SDDC.............................................................................. 75
Figure 97 Creating an Extended Network......................................................... 75
Figure 98 Deploying Unmanaged L2VPN Client OVF On Premises................. 76
Figure 99 Uploading the Unmanaged L2VPN Client OVF Files........................ 76
Figure 100 Selecting Networks During Unmanaged L2VPN
Client Deployment.........................................................................77
IX |
Figure 101 Configuring IP Address/Subnet and Peer Address During
Unmanaged L2VPN Client Deployment........................................ 78
Figure 102 Configuring TCP Setting and VLAN/Tunnel ID During
Unmanaged L2VPN Client Deployment........................................ 79
Figure 103 Extending Networks to VMware Cloud on AWS SDDC with L2VPN
and NSX-V On Premises............................................................... 79
Figure 104 Connected VPC via ENI Design.......................................................80
Figure 105 AWS Console - Connected VPC Active ENI.....................................81
Figure 106 AWS Console - Connected VPC Route Table...................................81
Figure 107 VMware Cloud ON AWS - Connected VPC Configuration............... 82
Figure 108 VMware Cloud on AWS SDDC - Networking & Security Tab........... 84
Figure 109 VMware Cloud on AWS RBAC - Identity and Access Management.85
Figure 110 NSX Grouping Object Based on IP Address..................................... 85
Figure 111 NSX Grouping Object Based on VM Instance....................................86
Figure 112 NSX Grouping Object Based on VM Name......................................86
Figure 113 Configuring NSX Grouping Object Based on VM Name................... 87
Figure 114 VMs and Associated NSX Security Tags............................................88
Figure 115 Configuring NSX Grouping Object Based on NSX Security Tag........88
Figure 116 Configuring Matching Criteria for NSX Grouping Object Based on
NSX Security Tag...........................................................................88
Figure 117 Feature to View Members of a NSX Grouping Object.......................89
Figure 118 Results of Viewing Members of a NSX Grouping Object...................89
Figure 119 Available Options for Defining NSX Grouping Objects......................90
Figure 120 Creating Security/Firewall Policies Using NSX Grouping Objects.....90
Figure 121 NSX Management Groups.................................................................91
Figure 122 NSX Workload Groups......................................................................91
Figure 123 Applying CGW and MGW Firewall Rules and Design....................... 92
Figure 124 Using ‘Apply To’ to Apply CGW Firewall Policies
on Specific Interfaces.................................................................... 93
Figure 125 MGW Firewall Rules Using Management Groups............................. 93
Figure 126 MGW System Defined Groups......................................................... 94
Figure 127 NSX DFW - Micro-segmentation and Design.................................. 95
Figure 128 NSX DFW - East/West Traffic Firewalling and Design..................... 95
Figure 129 NSX DFW GUI................................................................................. 97
Figure 130 NSX DFW Grouping Sections - Infrastructure Rules Example.........98
Figure 131 NSX DFW Grouping Sections - Environment Rules Example............98
Figure 132 NSX DFW Grouping Sections - Application Rules Example.............99
Figure 133 Created ‘Deny All’ DFW Rule for Whitelisting policy........................99
Figure 134 Created DFW Section.................................................................... 100
| X
Figure 135 Created NSX DFW Application Rule.............................................. 100
Figure 136 MGW Firewall Rule to Allow Communication from the ESXi hosts
in the SDDC to Wireshark.............................................................102
Figure 137 VMware Cloud on AWS SDDC Design for Port Mirroring
to Wireshark.................................................................................103
Figure 138 Configured Port Mirroring Session..................................................103
Figure 139 Examining Wireshark Capture.........................................................104
Figure 140 Configured Port Mirroring Session for Egress-Only Traffic..............104
Figure 141 VMware Log Intelligence Available for VMware Cloud on AWS........105
Figure 142 Enabling NSX-T Firewall Logs Collection.........................................105
Figure 143 Enabling VMware - NSX-T for VMware Cloud on AWS
Content Pack................................................................................106
Figure 144 Enabling Logging on MGW Firewall Rule........................................106
Figure 145 VMware Log Intelligence NSX Firewall Logs...................................106
Figure 146 Examining VMware Log Intelligence NSX Firewall Logs.................. 107
Figure 147 Configuring Splunk Data Inputs....................................................... 107
Figure 148 Configuring Splunk HTTP Event Collector.......................................108
Figure 149 VMware Log Intelligence Log Forwarding to Splunk.......................108
Figure 150 VMware Log Intelligence Log Forwarding Configuration................108
Figure 151 VMware Log Intelligence - Example NSX Firewall Logs...................109
Figure 152 IPFIX Collector Configuration...........................................................110
Figure 153 VMware Cloud on AWS Design for IPFIX with Collector in
Connected VPC............................................................................110
Figure 154 Configuring IPFIX Session................................................................111
Figure 155 Configured IPFIX Session.................................................................111
Figure 156 IPFIX Collector Example (FlowMon).................................................112
Figure 157 VMware Network Insight Example - Analyze and Display Traffic
Flows Between On-Premises Workloads and SDDC Workloads..113
Figure 158 VMware Network Insight Example - Analyze and Display Traffic
Flows Between Workloads in SDDC.............................................114
Figure 159 VMware Network Insight Example - Analyze application traffic
patterns and receive firewall rule recommendations......................115
Figure 160 VMware Cloud on AWS Solution and APIs...................................... 117
Figure 161 Navigating to VMware Cloud on AWS User Account........................119
Figure 162 Generating API Token.....................................................................120
Figure 163 VMware Cloud on AWS Developer Center..................................... 122
Figure 164 VMware Cloud on AWS Developer Center - API Explorer.............. 123
Figure 165 Org ID and SDDC ID in Support Tab............................................... 125
Figure 166 Configured MGW Rule for vCenter Access..................................... 127
Figure 167 CGW Firewall Rule Created from REST API Call.............................. 128
XI |
Figure 168 vSphere Web Client Displaying Compute Resource Pool............... 129
Figure 169 vSphere Web Client Displaying Datastore for Workloads............... 129
Figure 170 Search for PowerCLI Module..........................................................130
Figure 171 Check PowerCLI Installation............................................................130
Figure 172 Search for VMware Cloud on AWS Module.....................................130
Figure 173 List all VMware Cloud on AWS NSX-T specific functions..................131
Figure 174 Using PowerCLI, Connect to VMware Cloud on AWS..................... 132
Figure 175 Retrieve the NSX-T Reverse Proxy URL.......................................... 132
Figure 176 HCX Architecture............................................................................ 135
Figure 177 Gateway On Premises for HCX Extended Networks........................136
Figure 178 HCX Network Architecture..............................................................136
Figure 179 Load Balancing Use Cases.............................................................. 137
Figure 180 Load-Balancing with the AWS Load-Balancer................................138
Figure 181 Network Load-Balancer on AWS Pointing to a Target Group..........138
Figure 182 Target Group (aka Server Pool) Pointing to 3 Web Servers
in Healthy Status...........................................................................138
Figure 183 Virtual LB Needs to be Deployed in One-Arm mode.....................139
Figure 184 Load Balancing with a Virtual LB Appliance....................................139
Figure 185 AVI Controller - Application Performance and Visibility...................140
Figure 186 Connected VPC Design..................................................................141
Figure 187 Accessing Managed File Services from VMware Cloud on AWS
SDDC........................................................................................... 142
Figure 188 Leveraging S3 buckets for Back-ups and Snapshots...................... 143
Figure 189 Accessing a RDS Database from VMware Cloud on AWS SDDC.... 144
Figure 190 Communication from the EC2 Web Servers Back to the SDDC DB
Tier Using ENI.............................................................................. 145
Figure 191 AWS Directory Service....................................................................146
Figure 192 Accessing AWS Directory Service Using ENI..................................146
Figure 193 Inspecting VMware Cloud on AWS Traffic via On-Premises
Next-gen Firewall.........................................................................148
Figure 194 Using Next-gen Firewall On Premises and Exposing SDDC
Applications to Internet................................................................148
Figure 195 Next-gen Firewall Deployed within a Transit VPC in Native AWS.... 149
Figure 196 Connected VPC Design with Transit VPC.......................................150
Figure 197 Palo Alto Networks Appliances Deployed in a Transit VPC.............150
Figure 198 VMware Site Recovery Use Cases.................................................. 152
Figure 199 VMware Site Recovery Fan In Topology......................................... 152
Figure 200 VMware Site Recovery Fan Out Topology..................................... 153
Figure 201 AWS Data Charges Diagram.......................................................... 157
| XII
About the Authors
Humair Ahmed is a Senior Technical Product
Manager in VMware’s Network & Security Business
Unit (NSBU). Humair has expertise in network
architecture & design, multi-site and cloud solutions,
disaster recovery, security, and automation. Humair
is currently focused on networking and security
within VMware Cloud™ on AWS and providing cloud
solutions for customers.
Humair holds multiple certifications in development,
systems, networking, virtualization, and cloud
computing and has over 17 years of experience
across networking, systems, and development.
Previously at Force10 Networks and Dell Networking,
Humair specialized in automation, data center
networking, and software defined networking
(SDN). He has designed many reference architectures
for Dell’s new products and solutions, including Dell’s
first reference architecture with VMware NSX.®
Humair has authored many white papers, reference
architectures, deployment guides, training materials,
and technical/marketing videos while also speaking
at industry events like AWS re:Invent and VMworld.®
He is also the author of the VMware Press book,
VMware NSX Multi-site Solutions and Cross-vCenter
NSX Design – Day 1.
In his spare time, Humair writes on the VMware
Network Virtualization Blog and authors a popular
technology blog – http://www.humairahmed.com
– focused on networking, cloud, systems, and
development.
You can contact Humair on LinkedIn or Twitter
@Humair_Ahmed.
| XIII
About the Authors
Gilles Chekroun is a VMware Cloud on AWS Lead
Specialist in the European team. He joined VMware
in January 2015 after spending 20 years at Cisco in
the Data Centre Network and Virtualization team.
He has a strong background in networking
and storage.
Gilles started in the NSX team and quickly moved to
the European Solutions Engineering team to focus
on software-defined data center in the cloud with
AWS. He works closely with VMware customers in
designing, building and implementing their move
to the hybrid cloud model, helping them in their
digital transformation.
Gilles is VCP6-NV and AWS Solution Architect
certified. In his spare time, Gilles writes on the
VMware Cloud on AWS blog and authors a popular
blog at – http://www.gilles.cloud – focused on
VMware Cloud on AWS technologies.
You can contact Gilles on Twitter @twgilles.
XIV |
Acknowledgements
We would like to thank our family and friends for supporting us throughout
our careers, across many endeavors, and during the process of writing this
book. We also want to give a big thank you to Tom Gillis for writing the
foreword and opening this book with his perspective on NSX and VMware
Cloud on AWS. We would also like to thank Katie Holmes from the NSBU
marketing team for helping with some of the administrative details for
making sure this book gets published on time.
Additionally, thank you to all contributors and folks who provided feedback
from the VMware Network and Security Business Unit (NSBU) and VMware
Cloud Platform Business Unit (CPBU).
Finally, we would also like to thank all the VMware Cloud on AWS customers
out there for taking this journey to the cloud with us. We have been very
fortunate to have a great customer base to gain insight and feedback from
to help improve our solutions and services. This book is first and foremost
dedicated to our customers. Thank you!
Humair Ahmed
Gilles Chekroun
Nicolas Vibert
| XV
Preface
Enterprise customers are looking to extend and migrate their data center
solutions to the cloud; however, traditional cloud solutions hold many
challenges. These include application redesign, workload migration,
investment protection, and operational model consistency between on
premises and cloud.
Please note, this book is based on the VMware Cloud on AWS SDDC version
available at time of writing, SDDC version 1.7. As is with software, features/
capabilities change rapidly over time, especially for cloud/managed services.
Thus, if you are reading this book and have access to a VMware Cloud on
AWS SDDC on a later version, you may notice some discrepancies. Although
some differences may be seen in the GUI or there may be new features/
capabilities or different workflows, the general concepts, use cases, and
underlying core fundamentals will, for the most part, stay consistent.
XVI |
Foreword
The Hybrid Cloud is clearly becoming core to mainstream enterprise IT
infrastructure. Five years ago, there was public cloud and there was private
cloud – and the line between them was stark. VMware has been working
closely with the leading cloud providers to blur that line. As a result, today’s
hybrid cloud has many flavors. There are VMware private clouds, VMware
environments running on major public cloud vendors, VMware Cloud
Provider partners, and now public cloud providers bringing their hardware
into private cloud environments, like AWS Outposts. In all of these hybrid
cloud scenarios, NSX is the foundation that ties it all together.
Foreword | XVII
The Virtual Cloud Network is VMware’s vision of the future of networking
where NSX provides end-to-end connectivity, protection, and visibility of
workloads regardless of where they sit. Nothing exemplifies this vision
better than VMware Cloud on AWS. Whether your workloads are on
premises or in the cloud, the same NSX network and security stack can
be used to provide connectivity, security, and visibility.
In this book, VMware NSX and VMware Cloud on AWS experts Humair
Ahmed, Gilles Chekroun, and Nico Vibert take you through the most
fundamental NSX networking and security concepts in VMware Cloud on
AWS. Furthermore, you’ll walk away with a deep understanding of how
NSX is used in VMware Cloud on AWS and how it enables an unmatched
hybrid cloud solution where workloads can seamlessly be moved from on
premises to cloud and vice‑versa with no downtime.
So welcome to the cloud with VMware Cloud on AWS! I hope you are
excited as I am about this innovative new solution. As you read through
this book, I have no doubt you will start to see the NSX everywhere story
unfold in front of you and see how powerful the VMware Cloud on AWS
truly is and what it can do for your company.
Tom Gillis
Senior Vice President and General Manager,
NSBU, VMware
XVIII |
Chapter 1
Introduction
CHAPTER 1 - Introduction | 1
Additionally, customers can migrate and VMware vSphere® Storage
vMotion® workloads to the cloud with ease, as the same infrastructure is
used both on premises and in the cloud. Using Hybrid Linked Mode (HLM),
administrators can manage both the on-premises vCenter and VMware
Cloud on AWS SDDC vCenter from one centralized location, easily
migrating workloads between the different vCenters/sites.
The AWS network is a full layer 3 network and is itself an overlay. vSphere
ESXi VDS leverages port groups which use VLANs, thus VMware and AWS
jointly developed a solution where the VDS VLANs are mapped to AWS
Elastic Network Interfaces (ENIs). This VLAN-to-ENI mapping allows the
VDS and ESXi hosts to run in VMware Cloud on AWS on the AWS
infrastructure. Figure 2 diagrams this VLAN-to-ENI mapping.
2 |
Implementation details of the VMware Cloud on AWS SDDC underlay and
infrastructure are abstracted away; users can simply deploy an SDDC from
the VMware Cloud on AWS console as shown in Figure 3. vSphere ESXi
hosts, vCenter, vSAN, and NSX are automatically deployed and configured.
The administrator does not configure the underlay or vSphere infrastructure.
Users can deploy multiple SDDCs and can access them from a single
VMware Cloud on AWS console, as shown in Figure 4.
CHAPTER 1 - Introduction | 3
Once the VMware Cloud on AWS SDDC is deployed, users have multiple
connectivity options between on premises and the cloud SDDC. These
connectivity options include policy-based IPSEC VPN, route-based IPSEC
VPN, L2VPN, and Direct Connect. As mentioned prior, users also have
connectivity to native AWS resources (e.g., EC2, S3, and RDS) via ENI
connectivity. This connectivity is shown in Figure 5.
4 |
Chapter 2
There may be fewer issues when applications are deployed directly into
the cloud; however, challenges still exist, including:
6 |
Why VMware Cloud on AWS
VMware Cloud on AWS addresses the traditional challenges customers
face when moving to public cloud. In addition to enhanced functionality
and operational flexibility, VMware Cloud on AWS offers the following
benefits:
The next chapter looks at some popular uses cases before digging into the
available networking and security features and capabilities.
There are several factors that drive interest in leveraging cloud for data
center extension:
VMware Cloud on AWS provides the ability to build out or extend a data
center to the cloud with efficiency and ease, leveraging familiar technology
and tools. It enables customers to expand their data center footprint to the
cloud – either temporarily or permanently – with no hardware or CAPEX
investment. VMware Cloud on AWS is available globally, allowing
alignment of data center expansion with regional business requirements.
Figure 7 illustrates these global connectivity capabilities.
9 |
Migration
A top use case for VMware Cloud on AWS is migration of all applications
running in a data center to the VMware Cloud on AWS. This is sometimes
referred to as re-hosting.
Customers like the idea of accelerating their move to the cloud with no
disruption to their applications or operating model. Further, they can
continue leveraging their existing VMware expertise.
Some may want to move only specific workloads to the cloud, like dev/test
or VDI. Others are interested in moving all applications to VMware Cloud
on AWS ahead of decommissioning an on-premises data center. Still
others require a data center refresh, and instead of making additional
investments, decide to save on capital expenditure and associated
operational costs and migrate to VMware Cloud on AWS.
Disaster Recovery
Another common use case for VMware Cloud on AWS is disaster recovery
(DR). Setting up DR solutions can be a challenge due to the expense and
expertise required. Some customers are wary of running a dedicated data
center simply for the purposes of DR.
11 |
VMware Cloud on AWS Site Recovery is an add-on that can be connected
to an on-premises VMware Site Recovery Manager,™ leveraging vSphere
Replication and vSphere Recovery Manager to provide the DR solution.
When a customer enables the Site Recovery add-on as shown in Figure 11,
vSphere Recovery Manager and vSphere Replication are automatically
installed and configured on the management network in the SDDC. The
components and ecosystem lifecycle are managed by VMware while
customers manage the configuration and operations based on their DR
requirements. Customers can also download the on-premises components
to enable the DR solution between on-premises data centers and the
VMware Cloud on AWS SDDC.
About NSX
VMware NSX network virtualization provides a full set of logical network
and security services decoupled from the underlying physical infrastructure.
Distributed functions such as switching, routing, and firewalling not only
provide L2 extension capabilities, but also enhanced distributed
networking and security functions.
This book does not cover the basics of NSX. Rather, it discusses what
NSX features and capabilities are provided in VMware Cloud on AWS and
how to leverage these capabilities to provide robust cloud and hybrid
cloud solutions. The following chapter discusses these capabilities in
more detail. The goal of this chapter is to provide an overview of what
NSX enables within VMware Cloud on AWS and define the basic SDDC
network and connectivity architecture.
NSX Features
Figure 12 lists some of the core capabilities of NSX within VMware Cloud
on AWS, grouping them into three categories:
• Security
• Operations
14 |
Networking and Connectivity
Networking within the SDDC
• Deploy networks and define the respective subnet and gateway for
the workloads that will reside there. Users can create L2, L3, and
isolated networks.
• Create multiple DNS zones, allowing use of different DNS servers for
specific domains
• Direct Connect to carry all traffic between on premises and cloud over
high bandwidth, low latency connectivity, and optionally use VPN as
backup for compute traffic.
Security
For security, VMware NSX provides several features and capabilities:
• Edge firewall for both management (i.e., MGW firewall) and compute
(i.e., CGW firewall). Edge firewall is a stateful firewall that helps
protect north/south traffic in and out of the SDDC.
16 |
• For both Edge firewalls and DFW, NSX enables creation of Grouping
Objects used in the security rules. These Grouping Objects are defined
by the user; NSX automatically identifies and places workloads in the
group based on criteria like IP address, VM Instance, VM Name, and
Security Tag.
Operations
VMware NSX also provides important operational tools for troubleshooting
and managing within the SDDC.
• DFW IPFIX allows tools like VMware vRealize® Network Insight™ to map
traffic path specifics and identify traffic traversing the NSX Edge and
DFW firewalls.
• Public and private endpoints can use NSX APIs to automate anything
that’s supported by the VMware Cloud on AWS console.
When the SDDC is deployed, its respective ESXi hosts, vSAN, NSX,
and vCenter are automatically deployed and configured.
18 |
NSX-T is the networking and security platform used in VMware Cloud on
AWS. The VMware Cloud on AWS console in Figure 14 shows a deployed
Management Gateway (MGW) and a Compute Gateway (CGW) within a
SDDC. In NSX-T terminology, these are T1 routers.
Figure 16 shows that the MGW and CGW are connected via another
router. In NSX-T terminology, this is called a T0 router. This router allows
workloads on compute network segments connected to the CGW access
20 |
to components on the management network connected to the MGW.
The T0 router is also where connectivity to the external environment (e.g.,
IGW, Direct Connect, IPSEC VPN, ENI to connected VPC) takes place.
Note that in this architecture, both the compute network segments and
management network are overlays.
The T1 routers have a distributed component across all the hosts. The
distributed component of the CGW is the gateway for the workloads on
NSX network segments; it is essentially a distributed router. There is also a
T1 component on the Edge which provides access to services such as firewall
while also connecting to T0 for access to the external environment. The
Edge in VMware Cloud on AWS is a virtual appliance that runs in active/
standby mode. The standby is always placed on a different host from the
active appliance by leveraging anti-affinity rules.
• vCenter has both a public and a private IP address. User and location
access to vCenter can be restricted via the MGW firewall. This is
discussed in the coming sections.
22 |
Accessing VMware Cloud on AWS SDDC
Once the VMware Cloud on AWS SDDC is deployed, it is visible from the
central console (https://vmc.vmware.com/console). Figure 17 shows two
deployed SDDCs.
The Networking & Security tab is accessed by clicking the name of the
SDDC or the VIEW DETAILS link on the bottom left of the SDDC. This
menu offers access to additional configuration options for networking and
security configuration. Figure 18 presents the user view of the SDDC
screen offering access to the Networking and Security tab.
Figure 19 VMware Cloud on AWS SDDC - Networking & Security Tab
Once the SDDC is deployed, the customer can consume the rich NSX
networking and security capabilities to deploy robust solutions in VMware
Cloud on AWS SDDC. Figure 20 presents a design where the user:
24 |
• is accessing AWS EC2 resources in the connected VPC via ENI
connectivity
26 |
When a VMware Cloud on AWS SDDC is deployed, all access to vCenter is
blocked by default. This is because the MGW firewall protecting vCenter
has a Default Deny policy. Figure 22 shows the default firewall policies on
the MGW firewall when a new SDDC is deployed. In order to reach vCenter,
access must be explicitly permitted.
In Figure 23, a new firewall rule has been added allowing HTTPS access
from anywhere. This vCenter is now accessible over the Internet. Access
could be further restricted to a single IP address or subnet.
Figure 23 MGW Firewall Rule Allowing vCenter Access Over Internet
28 |
For access to vCenter via its private IP address, DNS resolution for
vCenter must be changed to Private IP as shown in Figure 26. DNS is
discussed in more detail in the next chapter.
Figure 27 MGW Firewall Rule Allowing vCenter Access for Specific Workloads Over
Direct Connect Private VIF
The next two chapters delve into greater details about the specific
features of NSX networking and security in VMware Cloud on AWS.
Routing
There are two different levels of routing available within VMware Cloud on
AWS – distributed and edge.
Distributed Routing
Distributed routing is east/west routing performed across all hosts within
the SDDC via kernel-level modules. Each host in the SDDC knows how to
reach all other workloads within other hosts in the SDDC, thus can route
directly without hair-pinning to an external router. This routing is very
efficient as it is performed at the kernel level.
As shown in Figure 29, the MGW and CGW routers are logical components
within the NSX Edge appliance.
The MGW and CGW provide north/south routing for management and
compute, respectively. Within the SDDC, there is a default route to the
AWS IGW for Internet connectivity.
31 |
It is important to understand the connectivity and traffic flow associated
with the NSX Edge appliance. All traffic entering and exiting the SDDC
must traverse the T0 router. This router, which also connects the MGW and
CGW, provides connectivity to the external world as presented in Figure 31.
• When the Direct Connect private VIF is connected, all ESXi and vMotion
traffic will use that connection. In that instance, it is not possible to set up
a separate IPSEC VPN connection for ESXi or vMotion traffic. If there is
no Direct Connect Private VIF connection, then the MGW route is used.
• If the same routes are learned over the Direct Connect private VIF and
route-based IPSEC VPN, the default setting prefers routes from the
route-based IPSEC VPN; however, ESXi and vMotion traffic will still
go over the Direct Connect private VIF connection.
• By enabling the Use VPN as backup to Direct Connect option under the
Direct Connect settings, all routes learned over Direct Connect Private
VIF will be preferred over routes learned over Route Based IPSEC VPN.
ESXi and VMotion traffic will always go over the Direct Connect
private VIF.
• VMware Cloud on AWS SDDC has a default route to the AWS IGW,
allowing workloads to have Internet connectivity if allowed through
the Edge FW. It is possible to also advertise a default route from
on premises over the Direct Connect private VIF or IPSEC VPN.
With this configuration, all traffic – including Internet-bound traffic
– will be routed back on premises.
33 |
Routed Network
A routed network is connected to the CGW. The distributed component
of the CGW is the gateway for the workloads on each respective network
segment. The network is routed to other networks attached to the CGW.
Extended Network
This network type is extended on premises via L2VPN. The gateway for
an extended network is always on premises as the extended network is
not connected to the CGW. An L2VPN server must be configured to
create an extended network. Creation of an extended network without an
L2VPN server configured will result in the alert shown in Figure 34. The
extended network is attached to the T0 router.
Disconnected Network
A disconnected network is not connected to the CGW. This isolated
segment is used by HCX – a VMware technology for extending layer 2 on
premises – so a field for Gateway / Prefix Length is present. This field
serves as a reminder of the IP address of the gateway used by the HCX
extended network; this gateway is always on premises.
The DHCP server sits on a network segment hidden from the GUI; this
allows users to set up DHCP for any network segment. If preferred, static
IPs can be used for workloads instead of DHCP. A combination of static
IPs and DHCP is also possible, in which case the static IPs should be
excluded from the DHCP IP range.
35 |
DHCP Relay
When using an external DHCP server, DHCP Relay can be configured
under the DHCP tab as shown in Figure 38.
Multiple DNS zones can be created for the CGW DNS. An example is
provided in Figure 41.
37 |
By default, the vCenter FQDN resolves to the public IP address for
vCenter. Resolution of the private vCenter IP address from the vCenter
FQDN may be desired when connecting to the SDDC via Direct Connect
private VIF or IPSEC VPN. Note, private vCenter IP address resolution is
required if you want to access vCenter over Direct Connect private VIF or
IPSEC VPN from on premises. This example was illustrated in the Access
to vCenter section in Chapter 4.
In this case, the default setting for vCenter FQDN under the Settings
must be changed as shown in Figure 42 to resolve to private IP address.
In this SDDC, the public IP used by the default SNAT rule for workloads is
35.160.216.100.
39 |
When deploying a workload with a private IP address on any of the network
segments in Figure 44, the default SNAT rule will be used. The public IP
address of the workload will be Source NAT Public IP, 35.160.216.100.
Figure 46 shows the interface used to request a new public IP address; the
Notes field can be used to help remember how the public IP address is used.
41 |
When a public IP address is requested from the SDDC console, the
respective public IP allocated will be shown as in Figure 47.
Please note, public IPs are not free. VMware passes the standard AWS
cost of a public IP on to customers. The public IP address fee is shown on
the VMware Cloud on AWS bill, not on the standard AWS bill. Please refer
to AWS documentation for fees related to public IPs.
Figure 49 Traffic Flow for Incoming Traffic from the Internet to Compute
Figure 50 Traffic Flow for Outgoing Traffic from the Compute to Internet
43 |
Under the Firewall column of the NAT rule, the default is Match Internal
Address. By clicking on Match Internal Address, as shown in Figure 51,
it is possible to change the default behavior so that the match is made on
the external address or public IP.
Figure 52 displays the rule created on the CGW firewall to allow access to
the web server from the Internet. The term Web in the destination field is
a Grouping Object with membership based on the IP address of web
server.
Figure 54 Hybrid Cloud Design with Direct Connect and Customer Router at DX Location
45 |
To start deploying Direct Connect, log into the AWS portal and request
a Direct Connect connection. Select the origin DX location/facility for the
Direct Connect connection and set the port speed. This will create a
Letter of Authorization (LOR) which can be provided to the DX location
facility to run the cross-connect.
• decreased latency
Details for each VIF Type are explained below. In this context, connection
means physical circuit and virtual interface or VIF means logical interface
on an AWS virtual router.
• AWS public endpoint services are not accessible over a private VIF
When using a Direct Connect public VIF to send traffic to VMware Cloud
on AWS SDDC, a VPN is required on the connection since it terminates at
the region level.
Direct Connect physical ports can either be 1G or 10G links, but this
bandwidth can be broken up or aggregated as desired. Two different
contract methods are supported:
Dedicated Ports
Hosted connections
It is also possible to use link aggregation groups (LAGs) with AWS; more
details on this are provided below.
47 |
Link Aggregation Group (LAG)
Direct Connect locations have resilient and diverse paths back to AWS
backbone. Using Direct Connect with VMware Cloud on AWS provides high
bandwidth, low latency connectivity to an SDDC deployed in VMware Cloud
on AWS. It can provide connectivity from on premises directly into a VMware
Cloud on AWS SDDC, delivering consistent connectivity and performance
while enabling live migration/vMotion from on premises to the cloud.
AWS Direct Connect can carry all traffic from on premises to a VMware
Cloud on AWS SDDC, and vice-versa. This includes ESXi management,
vMotion, cold migration, management appliance, and workload traffic.
• NSX network segments. These networks are the last three subnets in
the box at the bottom of the diagram.
Direct Connect uses BGP to exchange routes, using private and public
virtual interfaces for BGP peering. In the context of a Direct Connect
connection, there will be an external BGP (eBGP) session between two
different ASNs: one belonging to the customer and one belonging to AWS.
Figure 56 shows basic BGP configuration that is entered from the AWS
portal. This configuration is pushed down to the respective components
on AWS side, then must be propagated to complete the local router
connection to AWS Direct Connect. Once connectivity is established,
the entire VPC CIDR, management appliance network, and NSX network
segments are automatically advertised on premises. As of version 1.5, 16
network segments are supported for advertisement over a Direct Connect
private VIF to on premises. 100 networks can be learned from on premises
to VMware Cloud on AWS SDDC, and the actual range can be enhanced
with appropriate use of on-premises route aggregation.
49 |
Figure 57 is a screenshot of the Direct Connect tab in the VMware Cloud
on AWS console.
• The AWS Account ID. To use a Direct Connect private VIF in VMware
Cloud on AWS, create a hosted VIF in the AWS console and assign it
to the AWS account used by the associated VMware Cloud on AWS
SDDC. The only requirement in a VMware Cloud on AWS SDDC is to
accept the VIFs. Figure 57 shows two VIFs that were assigned to and
accepted by the respective AWS account used by the VMware Cloud
on AWS SDDC.
Figure 57 VMware Cloud on AWS SDDC - Direct Connect Private VIF Configuration
• For all new SDDCs, the default BGP Local ASN is 64512. It should not
overlap with what is being used on premises and can be changed as
necessary, as shown in Figure 58.
• An earlier version of the offering used the public ASN 7224. This
configuration remains valid, but once it is changed to a private ASN it
cannot be changed back. VMware recommends use of a private ASN.
Figure 58 Direct Connect Private VIF - Editing BGP Local ASN
51 |
• Multiple Direct Connect private VIFs can be deployed in active/active
or active/standby mode, agnostic to the VMware Cloud on AWS SDDC.
Figure 59 Encrypting Traffic with Route Based IPSEC VPN Over Direct Connect Private VIF
Figure 61 Direct Connect Private VIF and VPN as Backup Design
53 |
With a VMware Cloud on AWS SDDC and a Direct Connect private VIF,
Direct Connect terminates on an AWS Virtual Private Gateway (VGW), not
on the NSX Edge. The VGW learns the routes from on premises and the
NSX Edge route table is updated via API calls. This is illustrated in the
diagram in Figure 62. For this reason, it is not possible to advertise
different BGP metrics over a Direct Connect private VIF to influence
the routes; the metrics are never passed to the NSX Edge.
• when there are multiple routes, the route with longest prefix match is
always used
From the VMware Cloud on AWS console, both networks advertised from
the SDDC to on premises and those learned from on premises to the
SDDC are visible. This is shown in Figure 63.
Figure 63 Direct Connect Private VIF - Advertised and Learned Routes
55 |
Figure 64 Direct Connect Private VIF - Advertising Default Route from On Premises
Figure 66 Public and Private IP Address for Route Based IPSEC VPN
57 |
The informational button next to the BGP Remote IP displays the details
shown in Figure 68. If the IP address range is within the 169.254.0.0/16
subnet, the appropriate Edge/Gateway firewall policies are automatically
created. This eliminates the need for manual creation of firewall rules on
the compute gateway firewall permitting BGP communication over the
VPN interface.
There is only one public and one private IP address for VPN use in the
VMware Cloud on AWS SDDC. Even though there are two route-based
IPSEC VPN tunnels, the endpoint for the IPSEC VPN tunnel in the VMware
Cloud on AWS SDDC is the same. The two endpoints on premises are
different; the two tunnels each terminate on different hardware devices.
To create two tunnels to the on-premises data center, create two BGP
Local IPs or VTIs to peer with the two different on-premises neighbors.
Selection of the active and standby tunnels can be controlled via BGP
metrics from on premises.
Figure 70 Redundancy with Route Based IPSEC VPN Using Hardware Cluster On Premises
If both a route-based IPSEC VPN and a Direct Connect private VIF are
configured, routes learned over the route-based IPSEC VPN are given a
better administrative distance than routes learned over the Direct Connect
private VIF, so routes learned over route-based IPSEC VPN will be
preferred. The exception is that ESXi management and vMotion traffic
always go over the Direct Connect private VIF connection.
59 |
Figure 71 Network Advertisements via Route Based IPSEC VPN from SDDC to On Premises
The VIEW STATISTICS link, pictured in Figure 74, shows traffic statistics
and information useful in troubleshooting VPN connectivity issues.
61 |
There are also VIEW ROUTES and DOWNLOAD ROUTES links with
details on the advertised and learned routes. Figures 75 and 76 show
the advertised and learned routes from the VIEW ROUTES link. The
Advertised Routes tab displays the routes advertised from the SDDC.
The Learned Routes tab displays the routes learned by the SDDC.
• There are no entries in the route table for policy-based IPSEC VPN as
with a Direct Connect Private VIF and a route-based IPSEC VPN.
Route table decisions do not take into account the policy-based
IPSEC VPN.
63 |
AWS Transit Gateway
At AWS re:Invent 2018, AWS introduced a regional construct called a
Transit Gateway (TGW). The AWS Transit Gateway allows customers to
easily connect multiple Virtual Private Clouds (VPCs). The TGW can be
seen as a hub with the VPCs as the spokes in a hub and spoke-type
model. Any-to-any communication is made possible by traversing the
TGW. The TGW can replace the popular AWS Transit VPC design
previously used to connect multiple VPCs. The TGW can also be used
with VMware Cloud on AWS SDDC.
In this example, there is a VPN connection from the TGW to the on-premises
data center. Because the on-premises VPN connection connects to a TGW,
it can be shared among all the spoke VMware Cloud on AWS SDDCs and
native AWS VPCs. This is depicted more clearly in the Figure 79.
This design provides both shared connectivity on premises for all the
spokes while also supporting any-to-any configuration and communication.
65 |
Figure 80 AWS Transit Gateway Design - Direct Connect Private VIF Used by Spoke SDDC
Reflecting this design, below figures show the creation of an AWS Transit
Gateway in the console. ECMP is initially disabled as this design is using
active/standby VPN. When AWS TGW is deployed, ECMP is automatically
enabled. In the below case, as the VMware Cloud on AWS SDDC was
deployed before the 1.7 SDDC release, ECMP was manually disabled on
TGW. With the 1.7 SDDC release, if customers want to leverage ECMP
connectivity between TGW and VMware Cloud on AWS SDDC, they
should leave ECMP enabled.
Selecting the Resource ID in the first row shown displays the VPN
connections established to the first VMware Cloud on AWS SDDC. There
is a customer gateway in AWS which has the public IP address for the
VMware Cloud on AWS SDDC 1 VPN. In Figure 83, the Tunnel Details tab
displays two AWS-provided public IP addresses and two admin-selected
inside IP CIDR blocks used for virtual tunnel interfaces (VTI) interfaces.
67 |
BGP peers over these VTIs through an IPSEC VPN connection. This view
confirms that the tunnels are up and that the routes have been propagated
and learned automatically, both by the TGW and VMware Cloud on AWS
SDDC 1 via BGP.
Figure 83 AWS Console - Transit Gateway VPN Established to VMware Cloud on AWS SDDC 1
Figure 84 AWS Console - Transit Gateway VPN Established to VMware Cloud on AWS SDDC 2
With VPC attachments, the routes are not propagated from the TGW to
the spoke VPCs; however, the routes from the VPCs are propagated to the
TGW as propagation is enabled by default on the TGW.
In Figure 86, the subnet of VPC 2 is 172.33.0.0/16, with VPC 2’s route
table configured identically as before.
69 |
Figure 87 shows that all routes in the AWS Transit Gateway’s route table are
learned from the VMware Cloud on AWS SDDCs and native AWS VPCs.
Figure 88 VMware Cloud on AWS SDDC Route Based IPSEC Configuration to TGW
From the VMware Cloud on AWS console, users can also download or
view advertised routes and learned routes.
Figure 89 VMware Cloud on AWS SDDC Learned Routes from TGW
Figure 90 Route Based IPSEC VPN ECMP Design with TGW
71 |
It is easy to enable any-to-any communication by setting up connectivity
from a VMware Cloud on AWS SDDC to an AWS TGW. Further details and
a demo walkthrough of AWS Transit Gateway with VMware Cloud on AWS
are available at:
After creating this resource share, add other AWS accounts. Resource
share invitations can be accepted manually or automatically; this is a
parameter set at TGW creation.
At this stage, normal operations can proceed. New routes will be learned
from the newly-attached VPC and will appear in the TGW route table.
Figure 93 AWS Console - Transit Gateway Route Table Showing New Learned Route
73 |
L2VPN
VMware Cloud on AWS offers L2 extension via L2VPN. L2VPN
capabilities allow extension of L2 domains to the cloud to maintain
workload IP addresses and facilitate vMotion. Since IP addresses are
maintained via L2VPN, workloads can migrate to the cloud via vMotion
with no disruption. L2VPN can be used to extend L2 for on-premises
workloads that are on a VLAN or overlay; Figure 94 depicts an example.
Details of the specifics are explained in the remainder of this section.
Figure 95 Extending Networks to VMware Cloud on AWS SDDC with L2VPN
and No NSX On Premises
Figure 96 Created L2VPN Server and Extended Network in VMware Cloud on AWS SDDC
75 |
Figure 98 shows deployment of the unmanaged L2VPN client OVF. The
first step is to select an OVF deployment template.
Upload the unmanaged L2VPN client OVF files to the on-premises ESXi
server. These files can be downloaded from the VMware software
download website.
77 |
The next step of the OVF deployment configures additional settings
including default gateway, peer address, VLAN/Tunnel ID, and HA mode.
This configuration is shown in Figures 101 and 102.
Figure 101 Configuring IP Address/Subnet and Peer Address During Unmanaged L2VPN
Client Deployment
Figure 103 Extending Networks to VMware Cloud on AWS SDDC with L2VPN
and NSX-V On Premises
79 |
Connected VPC
During the SDDC deployment process, VMware creates an AWS account
for the customer and a dedicated VPC for the organization hosting the
SDDC. The customer must link its AWS account and provide a VPC with
one or more subnets to be linked to the VMware Cloud on AWS VPC. This
is referred to as the “connected VPC.” The connected VPC must meet a
few requirements:
• The AWS subnet(s) used for native AWS services should be associated
with the default route table.
• Every ESXi host is a bare metal EC2 instance with a 25 Gbps network
interface.
• Within a VMware Cloud on AWS cluster, one of the hosts runs the
active NSX Edge appliance; this is the host associated with the active ENI.
81 |
Connected VPC Information
In the main VMware Cloud on AWS console, there is a tab detailing
information on the connected VPC.
The prior chapter explored the networking aspects of NSX in the VMware
Cloud on AWS SDDC. This chapter focuses on the security capabilities
NSX brings to the VMware Cloud on AWS SDDC. These security
capabilities include Role-Based Access Control (RBAC), Grouping Objects,
Security Tags, Edge/Gateway Firewall for North/South traffic, and
Distributed Firewall (DFW) for East/West traffic.
Visibility of the Networking & Security tab, shown in Figure 108, requires
a user role of either NSX Cloud Auditor or NSX Cloud Admin.
Figure 108 VMware Cloud on AWS SDDC - Networking & Security Tab
Within Identity and Access Management, an admin can add users and
assign roles. NSX Cloud Auditor role and NSX Cloud Admin role are the
two roles specific to NSX networking and security; their permissions are
set as follows:
84 |
Figure 109 VMware Cloud on AWS RBAC - Identity and Access Management
Grouping Objects
Grouping Objects, also called Security Groups, allow for criteria-based
static or dynamic identification of workloads in an SDDC. These grouping
objects can also be used in security policies. This capability is beneficial
as it allows creation of security policies that are not simply dependent on
IP address; policies can be created around more meaningful concepts or
metadata like VM Name or Security Tag.
In this example, as the name includes TEST, there may be a rule stating that
this group cannot talk to VMs that have names including PRODUCTION.
Additionally, as the name contains SQL, there may be a rule specifying that
only port 1433 for SQL is opened for that VM.
Figure 112 shows a grouping object named webSG. If a VM has web in its
name, it will automatically become part of this group. A security policy can
later be created using this group and only allow traffic on ports 443 or 80.
86 |
Figure 113 shows that a group based on VM Name can have matching
criteria where the VM can either start with or simply contain a specific set
of characters.
Figure 115 shows creation of a group with matching criteria stating that any
VM that has a tag of nico is a member.
Figure 115 Configuring NSX Grouping Object Based on NSX Security Tag
Figure 116 Configuring Matching Criteria for NSX Grouping Object Based on NSX Security Tag
88 |
Once the group is created, membership visibility is provided so users can
easily determine which workloads are members.
Clicking the three dots to the left of the grouping object brings up the
menu in Figure 117.
View Members will display the VMs that are members of the group.
View References will show what security policies are using the group.
When the grouping objects are defined, they can then be used in security/
firewall policies, as shown in Figure 120.
90 |
Figure 121 NSX Management Groups
Workload Groups, shown in Figure 122, can use all the matching criteria
previously discussed such as IP address, VM Instance, VM Name, and
Security Tags.
The logical diagram in Figure 123 shows that CGW firewall rules can be
selectively applied to specific interfaces, providing connectivity to the external
environment. These interfaces include Internet, Direct Connect private VIF,
customer-native AWS or connected VPC, and VPN. The MGW firewall rules
control who and what can access vCenter and the management network.
Figure 123 Applying CGW and MGW Firewall Rules and Design
Figure 124 presents rules implemented on the CGW. CGW firewall rules
can use Grouping Objects based on the different matching criteria discussed
earlier in the chapter: IP address, VM Instance, VM Name, and Security
Tag. Workload Groups are used in CGW firewall rules.
The screen shot also contains a Security Group called Connected VPC
Prefixes. This Security Group is automatically created and has the Connected
VPC prefixes as members. This allows easy creation of security policies for
access to AWS services running in the Connected VPC.
92 |
The Applied To field identifies which specific interfaces the policies are
applied to as described in Figure 123’s logical diagram.
Figure 125 shows MGW firewall rules using Management Groups. MGW
firewall rules only use groups based on IP addresses. These rules are
typically based on infrastructure or access to management components
like vCenter. As mentioned earlier, predefined management groups
already exist for vCenter, ESXi hosts, and NSX Manager. Users can also
create groups based on IP address for on-premises ESXi hosts, vCenter,
and other management appliances.
94 |
allowing for segmentation within the same L2 network or across separate
L3 networks. A picture of this is shown in Figure 127 along with several
benefits of the practice.
The NSX DFW presents a contextual view of the virtual data center. It
can secure workloads based on meaningful metadata rather than simply
source and destination IP addresses. This can be done using Grouping
Objects discussed earlier in this chapter. These Grouping Objects are
called Workload Groups and can be based off of IP address, VM Instance,
VM Name, or Security Tag.
Isolation
Isolation of different workloads is achieved by leveraging micro-segmentation
capabilities and preventing communication between different environments
(e.g., production, test).
Multi-tenancy
NSX DFW allows for migration of different tenants to the cloud while
preventing communication between them. Since NSX DFW permits
applying policies at the vNIC level of workloads, tenants can easily be
migrated to the cloud while maintaining separation. As of SDDC version
1.7, overlapping IP addresses are not supported due to the common
connection to a single CGW.
DMZ anywhere
Since security policies can be applied granularly with the DFW, this allows for
creation of DMZs anywhere by applying the respective policies to workloads
in specific Grouping Objects. It is straightforward to control what is accessible
by the Internet and enforce rules on communication between zones.
96 |
This might be acceptable for customers using a VMware Cloud on AWS
SDDC to host a specific application or for test/dev workloads, but as
customers evacuate entire data centers and migrate workloads to VMware
Cloud on AWS or want to run a DMZ within a VMware Cloud on AWS
SDDC, they often require more advanced security and some additional
level of segmentation.
The GUI presented in Figure 129, displaying the Distributed Firewall tab,
is available under the Networking and Security tab within the VMware
Cloud on AWS console.
Users do not have direct access to the NSX Manager on VMware Cloud on
AWS. All networking and security configuration is provided via the VMware
Cloud on AWS console using an NSX Policy Appliance and respective APIs.
These high-level sections offer a way to build security logic and organize
rules/policies. Users do not have to create security rules to fit this model.
They may want to define all rules in one of these predefined sections;
however, these pre-defined high-level sections offer a standardized
approach to laying out policy.
98 |
Application Rules: Applies to specific application rules. An example is
allowing app-1 to talk to app-2 or block traffic between app-3 and app-4.
The rules are processed in top-down order, meaning traffic will hit the
emergency rules first and the application-specific rules last; therefore,
rules should be implemented from least specific to most specific. If no
rules are hit, the default rule is applied; this will allow all traffic.
The default rule – allow all traffic - cannot be changed from allow to deny.
With this ‘allow all traffic’ rule at the end, users need to blacklist (i.e., specify
which traffic is specifically blocked) traffic prior to reaching that rule.
Figure 133 Created ‘Deny All’ DFW Rule for Whitelisting policy
Figure 135 shows that each DFW rule contains a name, a source, a
destination, services (e.g., ICMP, http, https, etc.), an action (e.g., allow,
drop, or reject), and an option for logging via VMware Log Intelligence,
VMware’s cloud log platform.
100 |
Chapter 7
This chapter focuses on some of the operational tools that can be leveraged
with NSX in VMware Cloud on AWS. Some of these tools like port mirroring
and IPFIX are included with NSX, while other applications/tools like VMware
Log Intelligence and vRealize Network Insight leverage NSX features/
capabilities to provide additional useful information and services.
Port Mirroring
Port Mirroring is a feature on a virtual or physical switch that allows
capture of packets from a port and redirection to a destination device or
appliance. This capability is useful to analyze or monitor specific traffic. It
is typically used for these reasons:
On the VMC console, select the Networking & Security tab and then
Security -> Edge Firewall -> Management Gateway to get to the MGW
firewall policies.
Figure 136 shows a rule called To Wireshark that allows communication from
the ESXi hosts in the SDDC to Wireshark. This rule is needed to allow port
mirroring traffic to be sent from the ESXi hosts directly to Wireshark. The
ESXi in the source field is a system-created group that identifies all the ESXi
hosts in the VMware Cloud on AWS SDDC. The Wireshark in the destination
is a user-created group that identifies the workload where Wireshark is
running. Chapter 6 provides additional detail on Grouping Objects.
Figure 136 MGW Firewall Rule to Allow Communication from the ESXi hosts
in the SDDC to Wireshark
102 |
In the example from Figure 137, the user is monitoring traffic to the web
VMs. All the traffic to the web VMs is being copied and sent to Wireshark
running on the VM with the 192.168.1.134/24 IP address.
Figure 137 VMware Cloud on AWS SDDC Design for Port Mirroring
to Wireshark
In this screen shot, groups have been selected for Source and Destination.
For the Direction field, there are three options: Ingress, Egress, and
Bi-directional:
• Egress is the inbound network traffic from the logical network to the VM.
The outer header has a source IP of 10.56.32.4 (the ESXi host) and a
destination IP of 192.168.1.134 (the Wireshark VM) and is using GRE. The
inside header has a source IP of 172.31.26.221 (traffic coming from the
Internet over to the connected AWS VPC and over the ENI to the webserver)
and a destination IP of 172.30.120.18 (one of the web servers) and a
destination port of 80 (HTTP).
104 |
VMware Log Intelligence provides unified visibility into VMware Cloud on
AWS NSX-T network packet logs. With this capability, customers can analyze
and troubleshoot application flows through visibility into packets matching
specific NSX firewall rules. Benefits of this feature include the ability to:
• Maintain security
Figure 141 VMware Log Intelligence Available for VMware Cloud on AWS
For additional insight into the NSX-T firewall logs, enable the VMware -
NSX-T for VMware Cloud on AWS content pack. These dashboards provide
visibility into packet count and application flows in a single pane of glass.
As shown in Figure 144, to monitor the traffic to the SDDC NSX Manager,
enable logging on the rule on the right-hand side. In this example, the
source is Any, but a specific source IP can be specified if desired.
106 |
When drilling down on these logs, specific packet-level detail is available.
Figure 146 shows a packet sent from 172.30.0.177 (vRealize Network
Insight proxy on prem) to the VMware Cloud on AWS NSX Manager on
10.56.224.4 over HTTPS (443). As indicated by the firewall action (PASS),
traffic is allowed to go through.
For Splunk to accept logs over HTTP/HTTPS from VMC, set up the data
inputs and configure the HTTP Event Collector.
Once set up, the system will show how many events and logs are forwarded.
108 |
In the example from Figure 151, Splunk is running on native AWS in an EC2
instance from the AWS Marketplace. Ensure traffic is allowed on port 8008
to that instance in the AWS Security Group or the traffic will be dropped.
Table 1
Once the collector is created, IPFIX can be set up on the VMware Cloud
on AWS console. IPFIX data can be sent to up to four different collectors
at a time.
In the example in Figure 152, IPFIX data will be sent to IPFIX collector
software with an IP of 172.31.38.7 over port 2055.
Figure 153 VMware Cloud on AWS Design for IPFIX with Collector in Connected VPC
Once the collector has been created, an IPFIX session can be created as
shown in Figure 154.
110 |
Figure 154 Configuring IPFIX Session
Individual network segments can be selected for flow monitoring. All the
flows from the VMs connected to those network segments are captured
and sent to the IPFIX collector.
Users can also specify the Active Timeout – the length of time after which a
flow will time out, even if more packets associated with the flow are received
– and the Idle Timeout – the length of time after which a flow will time out if
no more packets associated with the flow are received. Both values can be
between 60 and 3600 seconds for VMware Cloud on AWS.
Figure 155 shows a configuration with an IPFIX session called IPFIX Export.
It has a 60-second Active Timeout, a 60-second Idle Timeout, and is
capturing flows on several network segments.
With this configuration complete, flows will start appearing on the IPFIX
Collector and can be used for analysis, troubleshooting, and reporting.
The example in Figure 156 leverages the FlowMon software, but other
IPFIX platforms can also be used. In this example, traffic statistics are
stacked up by service.
VMware also has its own network and security monitoring platform –
vRealize Network Insight. The next section will walk through its integration
with VMware Cloud on AWS.
112 |
With these two solutions, VMware offers customers a choice of software
consumption model. Customers can consume VMware Network Insight as
a service or deploy vRealize Network Insight within a data center to manage
their multi-cloud environment.
The following three examples are the most common VMware Network
Insight use cases for customers of VMware Cloud on AWS.
Figure 157 VMware Network Insight Example - Analyze and Display Traffic Flows Between
On-Premises Workloads and SDDC Workloads
Figure 158 VMware Network Insight Example - Analyze and Display Traffic Flows Between
Workloads in SDDC
114 |
Micro-segmentation Planning
Analyze application traffic patterns and receive firewall rule recommendations
– based upon traffic observed – to implement on the DFW. VMware
Network Insight analytics can be applied before or after migrating
applications to the cloud.
Figure 159 VMware Network Insight Example - Analyze application traffic patterns and receive
firewall rule recommendations
• vSphere APIs
The following list of tools and programming languages will be used with
the VMware Cloud on AWS REST APIs for examples in this chapter.
• cURL
–– cURL means command URL. It sends commands to a URL, in this
case sending REST commands with header, body, and parameters.
One downside of using cURL with VMware Cloud on AWS APIs
is that each cURL command must include a long authorization
token. The cURL command is available on Linux and Mac OS X
machines. It can be downloaded from https://curl.haxx.se
117 |
• Python
–– Python programming is a good way to use the REST interfaces
with VVMware Cloud on AWS. The program can store the
authorization token, minimizing copy and paste overhead. The
requests package is required for REST transmission, and the json
or simplejson package to form JSON headers. Python program
examples are available on the https://code.vmware.com , in the
code samples section.
• PowerCLI
–– It is also possible to control VMware Cloud on AWS using PowerCLI
commandlets in the Windows PowerShell. More information is
provided later in this book. Examples are available at the VMware
Cloud on AWS console under Developer Center > Downloads.
• Postman
–– Postman REST client is available as a free stand-alone application
for Windows, Mac OS X, and Linux. It simplifies making REST API
calls in general and to VMware Cloud on AWS SDDC.
The following section reviews the four different API families, providing
examples using different tools and programming languages.
119 |
In My Account page, select API Tokens and generate one. Tokens are
valid for six months.
Once the API token is available, it is used for generating a session token
called access token
https://console.cloud.vmware.com/csp/gateway/am/api/auth/api-tokens/
authorize
def getAccessToken(myKey):
params = {‘refresh_token’: myKey}
headers = {‘Content-Type’: ‘application/json’}
response = requests.post(
CSPURL +
‘/csp/gateway/am/api/auth/api-tokens/authorize’,
params=params,
headers=headers)
json_response = response.json()
access_token = json_response[‘access_token’]
return access_token
The python function above returns the access token; it makes a POST
request with the API token as an input parameter.
The base URL for the VMware Cloud on AWS APIs is https://vmc.vmware.
com/vmc/api. Full documentation of the available VMware Cloud on AWS
NSX Networking and Security APIs can be found by selecting VMware
Cloud on AWS at the following link https://code.vmware.com/apis.
121 |
Developer Center
In every VMware Cloud on AWS organization, there is a Developer
Center page.
This is a generic page where code samples, downloads, and the API
Explorer can be found. The API Explorer is available for both Cloud
Services Platform and VMware Cloud on AWS APIs.
Execution is performed online and will apply to the selected SDDC. Use
this with caution as API calls will be applied to a live environment and not
to a simulation.
With the 1.7 SDDC release, NSX-T APIs are now available with API
Explorer. Two NSX-T APIs are available: NSX VMC Policy API and NSX
VMC AWS Integrations API. These are shown in Figure 164.
There are two methods for interacting with NSX-T policy APIs on VMware
Cloud on AWS: VMC Console (via NSX-T Proxy URL) or NSX Manager (via
NSX Manager private IP address).
Connections performed through the NSX-T Manager follow the same process
as on premises. This approach is only possible after creating a VPN or Direct
Connect link and an initial firewall rule that allows incoming connection to the
NSX-T Manager in VMware Cloud on AWS SDDC. Setting this up can be
problematic since the NSX firewall blocks all access by default, including
access to the NSX-T Manager.
123 |
NSX-T Manager
Clients interact with the NSX-T Manager policy API using RESTful web
service calls over the HTTPS protocol. Once the proper firewall rule
provides access to the NSX-T Manager, API calls can use its IP address
with the correct https-based or session-based authentication.
For example, the base URL for the API call is:
https://<nsx-mgr-ip>/policy/api/v1/
An example NSX-T Policy API call listing all network segments is:
https:// <nsx-mgr-ip>/policy/api/v1/infra/tier-1s/cgw/segments
https://vmc.vmware.com/vmc/api/orgs/<orgId>/sddcs/<sddcId>
This parameter is the base NSX-T reverse proxy URL. The URL looks like:
https://nsx-A-B-C-D.rp.vmwarevmc.com/vmc/reverse-proxy/api/
orgs/<orgId>/sddcs/<sddcId>/sks-nsxt-manager
Once the reverse proxy URL is retrieved, it will be used in every API call.
For any operation using the NSX-T policy API, the access token is needed
with the reverse proxy URL appended in front of the specific API.
This python function returns the reverse proxy URL and receives as input
the Org ID, the SDDC ID, and the session token.
The NSX-T proxy URL allows the use of all VMware Cloud on AWS
networking and security functions via API.
/policy/api/v1/infra/tier-1s/<networkId>/segments
125 |
The reverse proxy URL is used and appended in front of the base link in
the following way:
https://nsx-A-B-C-D.rp.vmwarevmc.com/vmc/reverse-proxy/api/
orgs/<orgId>/sddcs/<sddcId>/sks-nsxt-manager/policy/api/v1/infra/
tier-1s/cgw/segments
Note that <network-Id> is replaced by cgw since this is where all network
segments are connected.
https://nsx-A-B-C-D.rp.vmwarevmc.com/vmc/reverse-proxy/api/
orgs/<orgId>/sddcs/<sddcId>/sks-nsxt-manager/policy/api/v1/infra/
domains/mgw/gateway-policies/default/rules/ID
The CGW firewall rules can also use predefined groups. The source/
destination can be a predefined group like Connected VPC or S3
prefixes. The Applied to field defines the scope to VPC interface,
Internet Interface, VPN Tunnel, Direct Connect, or All Uplinks.
https://nsx-A-B-C-D.rp.vmwarevmc.com/vmc/reverse-proxy/api/
orgs/<orgId>/sddcs/<sddcId>/sks-nsxt-manager/policy/api/v1/infra/
domains/cgw/gateway-policies/default/rules/LS1
127 |
“resource_type”: “CommunicationEntry”,
“scope”: [
“/infra/labels/cgw-public”
],
“destination_groups”: [
“ANY”
]
}
Figure 167 CGW Firewall Rule Created from REST API Call
The SDDC vSphere APIs are the same ones consumed on premises.
Since this SDDC is now a managed environment, not all vSphere APIs are
applicable and users should note the managed model and structure with
VMware Cloud on AWS SDDC. For example, virtual machines should be
deployed in the compute resource pool, as shown in Figure 168.
129 |
What is PowerCLI?
PowerCLI is the VMware PowerShell module that enables interaction with
VMware environments. When PowerCLI is installed, a series of commandlets
are deployed. These can be used in a Windows or Mac-OS PowerShell
environment.
Install PowerCLI
In a PowerShell window, run the command Find-Module -Name VMware.
PowerCLI, then Install-Module -Name VMware.PowerCLI.
https://github.com/lamw/PowerCLI-Example-Scripts/tree/master/
Modules/VMware.VMC.NSXT
Figure 173 List all VMware Cloud on AWS NSX-T specific functions
131 |
Start Using PowerCLI for VMware Cloud on AWS
At this stage the VMware Cloud on AWS PowerCLI is ready to automate
the SDDC. To do this, there are three mandatory parameters:
• $APIToken = “2fa2ef9b-xxxx-xxxx-xxxx-d9457ebae5c8”
# Create Groups
New-NSXTGroup -GatewayType CGW -Name LS1 -IPAddress @(“192.168.1.0/24”)
New-NSXTGroup -GatewayType CGW -Name LS2 -IPAddress @(“192.168.2.0/24”)
New-NSXTGroup -GatewayType CGW -Name VPC1 -IPAddress @(“172.201.0.0/16”)
New-NSXTGroup -GatewayType CGW -Name VPC2 -IPAddress @(“172.202.0.0/16”)
New-NSXTGroup -GatewayType CGW -Name VPC3 -IPAddress @(“172.203.0.0/16”)
133 |
Chapter 9
As discussed in Chapter 3, there are many use cases for VMware Cloud on
AWS. This chapter discusses some specific use cases and solutions including
hybrid cloud, disaster recovery, workload mobility/migration, access to native
AWS services, load balancing applications, and advanced 3rd party security.
HCX provides the ability to extend networks. Through its incorporated WAN
optimization engine, it enables customers to vMotion workloads over Internet.
HCX has retro-compatibility with vSphere 5.0 versions. Customers do not need
to upgrade their VMware estate to connect to VMware Cloud on AWS; they can
simply lift and shift applications, migrating hundreds of VMs out of their data
center to the cloud – and back if necessary – in a matter of days, if not hours.
The HCX Manager is installed on the cloud side when HCX is activated on
the VMware Cloud on AWS Add-Ons tab.
Currently with the 1.7 SDDC release, with HCX Network Extension, the
default gateway for the extended network only exists at the origin site.
Routed traffic from VMs on remote extended networks returns to the
origin site gateway.
135 |
This applies also to a network stretched with a NSX L2VPN, as previously
described in Chapter 5.
Migration can begin once the secure tunnel has been established between
the HCX source and destination sites. Using the HCX Network Extension
appliance is not compulsory; HCX can still be used – along with its benefits
such as retro-compatibility to vSphere 5.0 and its WAN optimization engine
– to migrate workloads when maintaining the IP address is not a strict
requirement.
The example in Figure 180 shows a simple web farm with three Windows
web servers in VMware Cloud on AWS.
The first design leverages AWS Elastic Load Balancing to balance traffic
towards the web farm through the Elastic Network Interface. The health
of the web farm can be monitored with health checks monitoring the
responsiveness of each real server over TCP 80.
137 |
Figure 180 Load-Balancing with the AWS Load-Balancer
Figure 182 Target Group (aka Server Pool) Pointing to 3 Web Servers in Healthy Status
One such load balancer is available from AVI Networks. This solution
not only provides enterprise-grade load balancing but also the ability to
perform multi-cloud load-balancing across VMs, native cloud instances,
and containers
Leveraging AVI requires installing the AVI Controller – where all the
intelligence lives – and the AVI Service Engine – essentially the load-
balancer that does all the work – within VMware Cloud on AWS.
139 |
In Figure 184, a virtual AVI load balancer was deployed with one of its
network interfaces attached to the web segment where the web farm is
based. Additionally, the management network interface of the load balancer
is connected to a dedicated management segment. The AVI load balancer
connects to the AVI Controller to receive its configuration. A pool of servers
and a virtual service – designated by the virtual IP – that monitors and
load balances the traffic are also depicted.
The shadow AWS account is linked to the customer’s own AWS account.
At creation of the SDDC, an AWS identity and access management (IAM)
role is created in the customer’s AWS account to allow the shadow AWS
account to update its connected VPC routing table. This enables
connectivity between the SDDC and the connected VPC.
141 |
While VMware Cloud on AWS makes cloud migration easier than ever
before, it is still worthwhile to consider how best to move, especially for
those with a significant volume of data. It is worth breaking down terabytes
or petabytes of data into different buckets, then transferring that data to
where it is most appropriate.
• Migrate VMs and their associated virtual hard drives to VMware Cloud
on AWS SDDC.
• Migrate file services to EFS for Linux VMs or FSx for Windows VMs.
EFS is an AWS-managed file server for Linux VMs. FSx is the AWS-
managed file server for Windows VMs. As these services reside on EC2
within a VPC, they can easily be attached and connected to over the
connected VPC.
Figure 187 Accessing Managed File Services from VMware Cloud on AWS SDDC
AWS Simple Storage Service (S3) was the first AWS service launched in
2006. S3 provides the following benefits:
143 |
Direct Access to S3 buckets from VMware Cloud on AWS VMs
VMware Cloud on AWS administrators may notice improvements in
performances when accessing S3 buckets. If on-premises VMs are already
accessing AWS S3 buckets to store and access objects, migrating them to
the VMware Cloud on AWS SDDC should improve performance. The
VMware Cloud on AWS SDDC will be able to connect to the buckets
across the private AWS network via the VPC endpoint.
While there are other criteria to consider – including licensing, cost, and
performance – the benefits of managed database administration and
maintenance remain attractive.
Figure 189 Accessing a RDS Database from VMware Cloud on AWS SDDC
Communications from the EC2 web servers back to the SDDC DB tier
would be across the ENI.
Figure 190 Communication from the EC2 Web Servers Back to the SDDC DB Tier Using ENI
145 |
Accessing Managed Directory Services from VMware Cloud
on AWS
AWS offers a managed Microsoft Active Directory (AD) platform called
AWS Directory Service.
The compute domain is protected by the CGW FW. The CGW FW provides
north/south network connectivity for VMs running in the SDDC.
Both the CGW and MGW provide firewall capabilities. The CGW FW and
MGW FW offer layer 4 (L4) firewalling – inspecting traffic up to layer 4 of
the OSI model. They look at IP addresses – both source and destination
– as well as TCP/UDP ports, filtering traffic based upon these criteria.
To accomplish this, advertise the default route over the VPN or Direct
Connect. All Internet-bound traffic from the VMware Cloud on AWS VMs
will transit the on-premises L7 appliance.
147 |
Figure 193 Inspecting VMware Cloud on AWS Traffic via On-Premises Next-gen Firewall
Inbound traffic from the Internet will go through the on-premises Internet
firewall, where the destination IP will be NATted to the private IP of
VMC-VM, then transferred across DX/VPN to VMC-VM.
Figure 194 Using Next-gen Firewall On Premises and Exposing SDDC Applications to Internet
This is a common option which works well. For customers who have no
presence on premises, a different option is required – a firewall hosted in
the cloud.
This option leverages the concept of an AWS transit VPC. A transit VPC is
a common strategy for connecting multiple, geographically-dispersed
VPCs and remote networks to create a global network transit center.
Figure 195 Next-gen Firewall Deployed within a Transit VPC in Native AWS
The AWS transit VPC serves as a hub VPC that connects to spoke VPCs
via a VPN. A next-gen firewall would be deployed within the transit VPC
as an EC2 instance. All traffic leaving the spoke VPCs is directed to the
hub/transit VPC for inspection by the next-gen firewall. In this setup,
VMware Cloud on AWS SDDC is just another spoke VPC.
149 |
The ENI-connected VPC is accessed via the ENI when the SDDC is
deployed. It is typically used for services such as Active Directory,
File Services (FSx) or backups. The ENI-connected VPC would not be
connected to the transit VPC – it would remain reachable from the
VMware Cloud on AWS SDDC via the ENI.
Disaster Recovery
As introduced in the use case section in Chapter 3, VMware Cloud on
AWS can offer a highly-reliable disaster recovery (DR) solution.
VMware Site Recovery uses public cloud to help enterprises solve the DR
problem. VMware Site Recovery is a disaster recovery as-a-Service (DRaaS)
offering on VMware Cloud on AWS. It is built on three key components:
151 |
Figure 198 VMware Site Recovery Use Cases
Migration of protected inventory and services from one site to the other
is controlled by a recovery plan that specifies the order in which virtual
machines are shut down and started up, the resource pools to which they
are allocated, and the networks they can access.
153 |
Summary
VMware Cloud on AWS enables several uses cases around data center
extension, migration, and disaster recovery. VMware Cloud on AWS
with NSX networking and security features provides robust capabilities
and solutions to existing data center problems. Customers can easily
deploy a cloud or hybrid cloud solution leveraging many of the technologies
they are already familiar with using on premises. VMware Cloud on AWS
offers a consistent look-and-feel experience on premises and in the cloud
while following similar operational and management models. Customers
can also leverage existing investments in technology and training. It
is extremely easy to get started using VMware Cloud on AWS and the
respective networking and security features; more information is
available at: https://cloud.vmware.com/vmc-aws.
| 154
Appendix A: Account Structure and
Data Charges
With the success of the VMware Cloud on AWS service, there are often
questions about the AWS account structure and data charges. The charges
discussed below are in addition to the cost of deploying a VMware Cloud
on AWS SDDC – which is available as on-demand, 1-year, and 3-year
terms. The latest pricing and details are available at https://cloud.vmware.
com/vmc-aws/pricing.
It is important to understand who owns what and how AWS charges will
be presented.
155 |
The customer will receive two bills: one from VMware and one from AWS.
The AWS bill is paid directly by the customer. It includes charges related
to the customer VPC like Internet out, cross-region, cross-AZ, EC2, S3,
RDS, etc.
The VMware bill includes charges related to the AWS VPC used by
sthe VMware Cloud on AWS SDDC. This bill includes egress charges,
cross-AZ charges, Elastic IPs etc. as described at the following link:
https://cloud.vmware.com/vmc-aws/pricing
• For S3 storage, data out to S3 is free and data in from S3 in the same
region is also free. The AWS S3 service itself is not free.
157 |
Index
A G
APIs 7, 17, 97, 116, 117, 118, 119, 121, Grouping Objects 17, 27, 29, 44,
122, 123, 124, 128 83, 85, 86, 87, 88, 89, 90, 92,
ASN 49, 51 95, 96, 102
AWS EC2 25, 145
H
AWS S3 144, 156
AWS Transit Gateway 16, 64, 65, 66, HCX 11, 16, 34, 134, 135, 136
70, 71, 72 Hybrid Linked Mode 2, 29
B I
BGP 16, 46, 49, 51, 54, 56, 57, 58, IGW 21, 22, 31, 32
59, 60, 63, 68, 71 IPFIX 17, 101, 109, 110, 111, 112
C L
CGW Firewall 16, 21, 22, 39, 40, 44, L2VPN 4, 16, 34, 74, 75, 76, 77, 78,
90, 92, 93, 127, 128 79, 136
Connected VPC 21, 22, 24, 25, 80, LAG 47, 48
81, 82, 92, 110, 127, 133, 141, 142, Load Balancing 134, 137, 138, 139,
150, 155, 156 145
D
M
DHCP 15, 33, 35, 36, 133
MGW Firewall 16, 21, 22, 26, 27, 29,
Direct Connect 4, 15, 21, 22, 25, 28, 90, 92, 93, 94, 102, 106
32, 45, 46, 47, 48, 49, 50, 52, Micro-segmentation XVII, 16, 21,
53, 54, 55, 60, 65, 123, 127, 147 25, 94, 95, 96, 115
Direct Connect Private VIF 24, 28,
29, 32, 38, 39, 45, 47, 48, 49, N
50, 51, 52, 53, 54, 55, 56, 57,
59, 60, 63, 66, 74, 92, 156 NAT 13, 15, 39, 41, 42, 44, 148
Direct Connect Public VIF 47 Network Segments 15, 19, 20, 21,
Disaster Recovery XIII, 8, 11, 12, 134, 22, 33, 34, 35, 36, 40, 48, 49,
151, 154 59, 63, 111, 114, 124, 126
Distributed Firewall (DFW) 16, 83, NSX XIII, XIV, XV, XVI, XVII, XVIII,
90, 94, 95, 96, 97, 98, 99, 100, 1, 3, 4, 7, 13, 14, 15, 16, 17, 18, 19,
106, 115, 137 20, 21, 22, 24, 25, 29, 30, 31,
32, 33, 35, 36, 37, 40, 42, 45,
DNS 15, 29, 37, 38, 55, 98
48, 49, 54, 59, 63, 64, 74, 79,
DX 45, 46, 148 80, 83, 84, 85, 86, 87, 88, 89,
90, 91, 92, 93, 94, 95, 96, 97,
E 98, 99, 100, 101, 105, 106, 107,
Edge Firewall 13, 15, 16, 17, 22, 25, 109, 116, 121, 122, 123, 124, 125,
92, 102, 137, 147 131, 132, 136, 137, 147, 154
Index | 158
P T
Policy-Based IPSEC VPN 4, 15, 56, Transit VPC 64, 149, 150, 151
63
Port Mirroring 17, 25, 101, 102, 103, V
104 vCenter XIII, XVI, 2, 3, 7, 19, 20, 21,
Public IPs 15, 26, 38, 39, 40, 41, 42, 22, 24, 26, 27, 28, 29, 38, 39,
44, 46, 55, 57, 67, 68, 148 55, 87, 90, 92, 93, 94, 126, 127,
133, 147
R
VIF 46, 47, 50
Role Based Access Control VPC 15, 16, 18, 46, 47, 49, 56, 59,
(RBAC) 17, 24, 83, 84, 85 64, 65, 67, 69, 70, 73, 80, 82,
Route-Based IPSEC VPN 4, 16, 32, 104, 110, 112, 127, 133, 141, 142,
52, 53, 54, 56, 57, 58, 59, 60, 143, 144, 146, 149, 151, 155, 156
61, 62, 63, 64, 66, 71 vRealize Network Insight 101, 107,
Routing 13, 15, 25, 30, 31, 54, 55, 112, 113
102, 135, 141 vSphere ESXi 1, 2, 3, 7
S
SDDC XVI, 1, 2, 3, 4, 12, 13, 14, 15,
16, 17, 18, 19, 20, 21, 22, 23, 24,
25, 26, 27, 28, 30, 31, 32, 33, 37,
38, 39, 42, 47, 48, 49, 50, 51, 52,
54, 55, 56, 58, 60, 62, 64, 65,
66, 67, 68, 69, 70, 71, 72, 74, 75,
77, 79, 80, 81, 82, 83, 84, 85, 92,
96, 97, 101, 102, 103, 106, 110,
113, 114, 117, 118, 121, 122, 123, 124,
125, 128, 132, 134, 135, 137, 139,
140, 141, 142, 143, 144, 145, 147,
148, 149, 150, 151, 152, 155, 156
Security Groups 85, 92, 109, 110, 146
Security Tags 17, 83, 85, 87, 88, 89,
91, 92, 95
Site Recovery Manager (SRM) 12, 151
SNAT 15, 22, 39, 40
159 |
Companies are under tremendous pressure to implement cloud strategies –
ranging from establishing cloud-based development environments to
completely evacuating data centers and moving to the cloud. Traditional
cloud solutions hold many challenges; these include application redesign,
workload migration, investment protection, and operational model
consistency between on premises and cloud. VMware Cloud on AWS –
an offering jointly developed by VMware and AWS – addresses these
challenges with comprehensive solutions. VMware Cloud on AWS
simplifies the experience of enterprise customers looking to implement
a cloud strategy, allowing them to extend and migrate their data center
solutions to the cloud with ease.
This book dives into the features and capabilities of VMware NSX
networking and security in VMware Cloud on AWS. NSX is VMware’s
network virtualization and security stack that provides a full set of logical
network and security services decoupled from the underlying physical
infrastructure. These enhanced NSX distributed networking and security
functions enable much of the functionality in VMware Cloud on AWS,
providing a robust cloud and hybrid cloud solution. Customers can now
immediately utilize VMware Cloud on AWS SDDCs, deploying and
migrating workloads to the cloud while leveraging technologies they
already use on premises like VMware ESXi™, VMware vSAN,™ VMware
NSX® networking and security, and VMware vCenter.®
ISBN-13: 978-0-9986104-9-8
ISBN-10: 0-9986104-9-6
www.vmware.com/go/run-nsx
www.cloud.vmware.com/vmc-aws