HCX(Hybrid Cloud Extension) Explained

Hybrid Cloud Extension (HCX) is an all in one solution for workload mobility. Customers can freely move workloads between multiple on-premises environments as well as VMware Cloud on GCP (GCVE).

Think about what’s usually involved when migrating workloads from one location to another. The network is at the top of the list, ensuring adequate bandwidth and routing is in place. Not only from data center to data center or data center to cloud, but also across regions and remote sites. Validating vSphere version compatibility between the source and destination is also important. Other workload migration considerations could include:

• Moving across different vSphere SSO domains independent of enhanced linked mode
• Mapping workload dependencies
• Older hardware preventing from upgrading to vSphere 6.x
• Bi-directional migrations independent of vSphere SSO domains, vSphere versions, hardware, or networks
• Validation of resources (compute and storage) on the destination side
• SLA and downtime involved
• Mobility options correlating to SLAs (downtime, low downtime, no downtime)
• No replatforming of the workloads or applications including no changes to IP or MAC Addresses, VM UUID, certs

HCX Overview

HCX is the swiss army knife of workload mobility. It abstracts and removes the boundaries of underlying infrastructure focusing on the workloads. A HCX vMotion, for example, requires no direct connectivity to ESXi hosts in either direction compared to a vSphere vMotion. All HCX vMotion traffic gets managed through the HCX vMotion Proxy at each location. The HCX vMotion Proxy resembles an ESXi host within the vCenter Server inventory. It’s deployed at the data center level by default, no intervention is necessary. One thing to mention is the HCX vMotion proxy gets added to the vCenter Server host count by default. The HCX team is aware and will be changing this in the future, but this has no impact on your vSphere licensing.

Another boundary HCX removes is it supports several versions of vSphere going back to vSphere 5.0 to the most current release of vSphere 6.7 Update 1. This provides flexibility in moving workloads across vSphere versions, on-premises locations, and vSphere SSO domains. For on-premises to on-premises migrations, a NSX Hybrid Connect license is required per HCX site pairing. Migrating workloads from on-premises to GCVE does not require a separate HCX license. By default when deploying a GCVE SDDC HCX is included as an add-on and is enabled by default.

In order to start migrating workloads, network connectivity between the source and destination needs to be in place. The good news is it’s all built-in to the product. HCX has WAN optimization, deduplication, and compression to increase efficiency while decreasing the time it takes to perform migrations. The minimum network bandwidth required to migrate workloads with HCX is 100 Mbps. HCX can leverage your internet connection as well as direct connect. The established network tunnel is secured using suite B encryption. The on-premises workloads being migrated with no downtime will need to reside on a vSphere Distributed Switch (VDS). It also supports a 3rd party switch in the Nexus 1000v. Cold and bulk HCX migration types are currently the only two options which support the use of a vSphere standard switch but implies downtime for the workload. To minimize migration downtime, HCX has a single click option to extend on-premises networks (L2 stretch) to other on-premises sites or VMware Cloud on GCP. Once the workloads have been migrated there is also an option to migrate the extended network, if you choose. Other built-in functionality includes:

• Native scheduler for migrations
• Per-VM EVC
• Upgrade VM Tools / Compatibility (hardware)
• Retain mac address
• Remove snapshots
• Force Unmount ISO images
• Bi-directional migration support

Migration Types

HCX supports four different migration types:

• Cold Migration — offline
• vMotion (live) — no downtime
• Bulk Migration — low downtime
• Cloud Motion with vSphere Replication — no downtime

Cold migration and vMotion should be familiar to those using the built-in vSphere mobility options. Bulk Migration and Cloud Motion with vSphere Replication are new options and only available with HCX. Let’s take a look at how each option works and what is the impact to the workloads.

Cloud Motion with vSphere Replication

The new kid on the block announced during the VMworld 2018 US keynote. This new option provides zero downtime for workload mobility from source to destination. First, the workload disk(s) (VMDK) get replicated to the destination site. The replication is handled using the HCX built-in vSphere replication. This process is dependent on the amount of data and available network bandwidth. Once the data sync is complete the HCX switchover initiates a vMotion. The vMotion migrates the workload to the destination site and synchronizes only the remaining data (delta) and workload memory state. There is an option to schedule a maintenance window for the vMotion data sync switchover otherwise, it happens immediately.

When using the scheduling option, there are a few things to be aware of. First, the switchover can be re-scheduled any time prior to the actual scheduled time. Secondly, if the initial vSphere Replication data sync has not completed prior to the scheduled window, the workload(s) being migrated will fail. In this case, these workload migrations must be restarted in their entirety. Finally, if the vSphere Replication data sync has completed prior to the scheduled switchover window, HCX will continue to sync delta information to keep the migrated data as current as possible prior to the vMotion switchover. The vMotion switchover happens in a serialized fashion, only one at a time.

Bulk Migration

Bulk Migration creates a new virtual machine on the destination site. This can either be on-premises or GCVE and retains the workload UUID. Then it uses vSphere Replication to copy a snapshot of the workload from source to destination site while the workload is still powered on. In this case, a snapshot is a point of the time of the workload disk state, but not a traditional vSphere snapshot. The Bulk Migration is managed by the HCX interconnect cloud gateway proxy. During the data sync, there is no interruption to the workloads. The data sync is dependent on the amount of data and available bandwidth. There is an option to schedule a maintenance window for the switchover (similar to the schedule options described in the Cloud Motion section above) otherwise, the switchover happens immediately. Once the initial data sync completes, a switchover takes place (unless scheduled). The source site workloads are quiesced and shut down leveraging VMware Tools. If VMware Tools is not available, HCX will prompt you to force power off the workload(s) to initiate the switchover. During the switchover process, a delta sync occurs based on changed block tracking (CBT) to sync the changes since the original snapshot. The workloads on the destination site will begin to power on once the data sync is complete (including delta data changes). There are checks in place to ensure resources are available to power on the workloads. If a destination workload cannot power on due to resources, the source workload will get powered back on.

Unlike cold migration, the downtime incurred after the bulk migration cut over is low. The HCX manager renames the original source workloads by appending a binary timestamp. A migrated VMs folder gets created in the VM and templates view. Finally, the original source workloads get placed in the migrated VMs folder. This also provides an option for easy rollback and quick data seeding since bulk migration supports reverse migrations. One thing worth mentioning is HCX has built-in network resiliency and will pick up where it left off during a data sync if there is any network interruption. On the flip side, if there is an issue powering on workloads on the destination site or missed scheduling a cut over, the entire data sync process will need to be started over. This technology allows a workload migration between different chipset versions (e.g. Sandy Bridge to Skylake) and across different CPU families (e.g. AMD to Intel).

vMotion “Live Migration”

HCX supports the vMotion we know and love today. The workloads are migrated live with no downtime similar to Cloud Motion with vSphere Replication. vMotion should not be used to migrate hundreds of workloads or workloads with large amounts of data. Instead, use Cloud Motion with vSphere Replication or Bulk Migration. Usually, a vMotion network needs to be configured and routed to the target vSphere host, in this case, the vMotion traffic is handled by the HCX Interconnect cloud gateway for cross-cloud vMotion. vMotion through HCX encapsulates and encrypts all traffic from source to destination removing network complexity of routing to cloud. The vMotion captures workload:

• Active Memory
• Execution State
• IP Address
• MAC address

HCX has a built-in option to retain the workloads MAC address. If this option is not checked, the workloads will have a different MAC on the destination site. Workloads must be at compatibility (hardware) version 9 or greater and 100 Mbps or above of bandwidth must be available. With vMotion and bi-directional migration, it’s important to consider Enhanced vMotion Compatibility (EVC). The good news here is HCX also handles EVC. The workloads can be migrated seamlessly and once rebooted will inherit the CPU features from the target cluster. This allows a cross-cloud vMotion between different chipset versions (e.g. Sandy Bridge to Skylake) but within the same CPU family (e.g. Intel). Also, an important thing to note is vMotion is done in a serialized manner. Only one vMotion occurs at a time and queues the remaining workloads until the current vMotion is complete.

Cold Migration

As the name states, a cold migration occurs when a workload is offline. CPU compatibility checks do not apply during a cold migration. During a cold migration, VMware’s Network File Copy (NFC) protocol is used to transfer the workloads from source to destination which includes:

• Configuration files including BIOS settings (NVRAM)
• Disks associated with the workload
• Log files

Usually, cold migration traffic takes place over the host management network. This traffic type is known as provisioning traffic. Provisioning traffic is not encrypted but uses run-length encoding. Run-length encoding is a simple form of lossless data compression. In the case of HCX, cold migration traffic will get encrypted using Suite B encryption. It leverages the same host management network to migrate the data through the HCX vMotion proxy. HCX only displays the cold migration option if the selected workload is not powered on. Although HCX provides the option to multi-select it is still limited. vCenter Server only handles 8 concurrent vMotions (cold and live) at a time. It places the remaining selected workloads in a queue until ready to migrate.

Migrate from On Prem VMWare to GCVE

In the basic HCX architecture based on GCVE, there are two major components known as HCX Cloud and HCX Connector that establish the communication between on-premises infrastructure and the GCVE environment. As the name states, HCX Cloud is the HCX deployment on the GCVE side (also referred to as destination HCX) and the HCX Connector is installed on the source site. Depending on the requirements, we can also deploy the HCX Connector per site to connect to GCVE in a multi-site service mesh topology, where multiple source sites connect to a single destination site. The basic configuration of HCX also requires it to connect with vSphere management components such as vCenter Server, vCloud Director and NSX.

Let’s work through the steps to deploy and configure HCX Connector for a source HCX site.

VMware HCX installation and configuration

In GCVE, HCX Cloud is pre-installed as a part of the initial Private Cloud initialization and configuration. So, let’s start with the installation and configuration of the HCX Connector.

Gathering the resources required to install and configure HCX Connector

Before we begin the installation and configuration of HCX Connector, we will need the following resources:

• Access and permissions to the GCVE Console
• HCX Connector OVA appliance
• HCX activation key

1. Start collecting the resources. We’ll begin with the HCX Connector OVA appliance. For that, log in to the GCVE console, navigate to Resources > [Private Cloud Name]. Under HCX Manager Cloud version, click on 4.2.4 (or the available version, if different). Use the CloudOwner user credentials or any other user that is configured with similar permissions.

2. Once logged in to the HCX Cloud Manager, navigate to the Support section and then click on Request Download Link.

3. Two web buttons will be generated. Click on VMWARE HCX to download the OVA file (OVA is the zipped version of OVF files) immediately on the machine from which you are accessing HCX manager console, or click COPY LINK to share or download the file on another machine.

4. Next, generate the HCX Connect Activation key from the GCVE console, navigate to Resources > [Private Cloud Name]. Under the HCX Manager Cloud version, click the Generate HCX Activation Key icon. Repeat this step to generate multiple keys.

5. Once you receive the notification “Successfully generated HCX connector activation key”, click the Download HCX Activation Key icon to download the generated keys.

Deploying HCX Appliance

Now we’ll start the deployment of HCX Connector OVA. This is a simple, straightforward deployment, like the other OVA appliances.

1. At the source HCX site, in the vCenter Server select the OVA appliance.

2. Enter the appliance name and select the destination folder.

3. Select the destination cluster.

4. Review the appliance details.

5. Read and accept the License Agreement, then select the datastore.

6. Select the VM network which has access to the management components, such as vCenter Server.

7. Enter the appliance-specific configuration information, including:

• Admin and root password
• Network properties
• Static routes (optional)
• DNS and service configuration, such as NTP and enabling SSH (optional)

8. Review the configuration carefully and click FINISH to start the deployment process.

Configure HCX Appliance

1. Once the deployment completes, power-on the HCX appliance (if not already started) and log in to the HCX Manager management interface from https://<hcx manager IP/FQDN>:9443 with username admin and the password defined at the time of deployment.
2. From the login screen you will be redirected to the HCX Manager configuration wizard. On the first step enter the HCX Connector Activation key that was downloaded previously.

3. Enter the exact location of the source site (or as close as possible).

4. Enter the system name to be used by the HCX Connector, then click CONTINUE.

5. On the confirmation screen, click YES, CONTINUE to configure vSphere integration.

6. Enter the fully qualified domain name (FQDN) or IP address of the vCenter server as https://<vCenter Server IP/FQDN> and a service or user account who is a member of the vSphere Administrators group or that has the vSphere Administrator role assigned. Optionally, you can also integrate the HCX with NSX. To integrate this, we need a service or user account with the Enterprise Admin (for NSX-T) or Enterprise Administrator (for NSX-V) role assigned.

7. Enter the IP or the FQDN of the vCenter Server or the Platform Services Controller (PSC) as https://<vCenter Server IP/FQDN>

8. Verify the information is correct, then restart the system.

9. Once the restart is finished and the HCX manager is back, review the status of the services from the dashboard.

10. After the initial configuration we are ready to connect the HCX Connector with HCX Cloud. Log in to the HCX Manager console from https://<HCX manager IP/FQDN> using a user account that is a member of the vSphere Administrators Group or has the vSphere Administrator role assigned. By default, vSphere Administrators have the permissions to perform HCX operations. You can also assign the permissions to perform the HCX operations to a user that is not a member of vSphere Administrators Group or does not have the vSphere Administrator role assigned by using HCX Role Mapping.

11. Create a site pairing to connect to the remote HCX site (GCVE).

12. Enter the HCX Cloud Manager (GCVE) IP/FQDN as https://<HCX Cloud Manager IP/FQDN> and credentials for a user that is a member of the Cloud-Owner-Role group. By default, the CloudOwner user has the required permissions.

13. Once connected, the Site Pairing will show the pairing status, as shown below:

Congratulations, you have successfully configured the HCX Connector and connected the HCX Connector to the HCX Cloud!

Summary

HCX offers customers several different workload mobility options covering different SLAs and requirements. No reconfiguration of the source or destination sites is necessary to get started. The platform offers built-in encryption, WAN optimization, and cross-version migration through the HCX interconnects. The HCX service is evolving at a rapid rate, the team is making improvements and adding new functionality on almost a monthly basis.