Self-Managed VCF 9.x on Google Cloud Bare Metal

Google Cloud now offers self-managed VMware Cloud Foundation 9.x on bare-metal instances, granting full root access and license portability across 22 regions.

TL;DR

  • Google Cloud has announced general availability for self-managed VMware Cloud Foundation (VCF) 9.x on dedicated bare metal across 22 initial regions.
  • The bare-metal model grants root-level access to ESXi, vCenter Server, and NSX components, shifting day-2 maintenance and operational burdens entirely to the customer.
  • Deployments run on z3-highmem-192-highlssd-metal hardware instances featuring 96 physical Intel Xeon cores, 1.5 TB RAM, 72 TB local NVMe storage, and dual 100 Gbps (200 Gbps aggregate) networking.
  • Infrastructure prerequisites mandate enabling VLAN-based networking in Google Cloud VPC, configuring an 8896-byte MTU for jumbo frames, and provisioning dedicated subnets for management, vSAN, vMotion, and twin NSX networks.
  • Customers bring portable Broadcom VCF 9.x subscriptions, utilizing local NVMe with vSAN Express Storage Architecture (ESA) or external NetApp Volumes for primary storage.
  • No direct, automated migration path currently exists between Google Cloud VMware Engine (GCVE running VCF 5.2) and the self-managed VCF 9.x bare-metal environment.

Google Cloud has launched a self-managed deployment option for vmware cloud foundation 9.x running directly on dedicated bare-metal infrastructure across 22 initial regions. The release introduces an operational model that diverges from the existing Google Cloud VMware Engine (GCVE) service. Rather than delegating hypervisor lifecycle management and control plane maintenance to Google, organisations receive root-level privileges across the entire VMware software stack, including ESXi, vCenter Server, and NSX. This shift provides infrastructure teams with unconstrained administrative privileges while requiring them to take direct ownership of system patching, minor and major version upgrades, and architecture compliance.

The bare-metal deployment addresses enterprise demand for license portability and version parity with on-premises environments. According to Google, the infrastructure relies on high-density compute nodes that run VMware’s latest release stack. However, gaining complete control of the hypervisor layer removes the operational abstractions provided by managed cloud services. Infrastructure architects evaluating this platform must balance the architectural flexibility of root access against the overhead of self-managed day-2 operations, cluster bootstrapping requirements, and manual lifecycle upkeep.

Google Cloud Bare-Metal VMware Architecture

The core distinction between Google Cloud VMware Engine (GCVE) and the new bare-metal offering lies in the management boundary and administrative permissions. GCVE operates as a provider-managed service running VCF 5.2 on ve1 and ve2 node instances. In that managed environment, Google retains ESXi root access, manages control plane availability, automates patching, and limits customer permissions to role-based vCenter access. In contrast, the self-managed bare-metal model deploys VCF 9.x on z3-highmem-192-highlssd-metal instances, granting complete administrative control to the customer.

This division of responsibilities places physical infrastructure maintenance on Google while handing the complete software stack over to the customer. Google monitors, repairs, and replaces the underlying bare-metal hardware, power systems, rack top-of-rack (ToR) switches, and physical server components. The customer assumes operational responsibility for the hypervisor, storage policy configuration, software-defined networking, identity integration, and VCF lifecycle operations.

+-------------------------------------------------------------------------+
|                  CUSTOMER OPERATIONAL BOUNDARY                          |
|                                                                         |
|  +-------------------------------------------------------------------+  |
|  |                 VCF Operations & Management Stack                 |  |
|  +-------------------------------------------------------------------+  |
|  |  vCenter Server 9.x  |   NSX Manager 9.x   |  vSAN ESA Manager    |  |
|  +-------------------------------------------------------------------+  |
|  |                Root Access: ESXi 9.x Hypervisor Layer             |  |
|  +-------------------------------------------------------------------+  |
+-------------------------------------------------------------------------+
|                    GOOGLE CLOUD INFRASTRUCTURE                          |
|  +-------------------------------------------------------------------+  |
|  | z3-highmem-192-highlssd-metal (96-core Xeon, 1.5TB RAM, 72TB NVMe)   |  |
|  +-------------------------------------------------------------------+  |
|  | Physical Network (200 Gbps NICs) | Power & Cooling | Rack Placement |  |
|  +-------------------------------------------------------------------+  |
+-------------------------------------------------------------------------+

A primary driver behind this launch is Broadcom’s subscription licensing model. As Broadcom transitioned customers toward portable VMware Cloud Foundation subscriptions, enterprise IT organisations sought cloud targets capable of consuming these entitlement keys directly without middleware translation layers. With self-managed VCF 9.x on Google Cloud bare metal, organisations apply their existing portable Broadcom VCF enterprise licensing directly to the deployed control plane. This allows workloads to run without modifications to hypervisor configuration or operational tooling.

Operational Responsibility Shift

Obtaining root access on ESXi hosts and NSX Edge nodes voids provider-managed lifecycle automation. If a security vulnerability impacts ESXi or vCenter, your internal operations team or designated managed services partner is responsible for testing, scheduling, and applying patches using VCF Operations.

z3-highmem Hardware and Networking Prerequisites

The underlying compute host for this deployment is the z3-highmem-192-highlssd-metal bare-metal instance. Each node is engineered for resource-intensive virtual machines and storage-heavy enterprise applications. The hardware specifications per physical host include:

  • Compute: 96 physical cores powered by 4th-generation Intel Xeon Scalable processors.
  • Memory: 1.5 TB of physical RAM.
  • Storage: 72 TB of local, high-performance NVMe flash storage.
  • Network Interfaces: Dual physical NICs supplying an aggregate network bandwidth of 200 Gbps.

Deploying VCF 9.x on Google Cloud bare metal requires strict network pre-configuration within the Google Cloud Virtual Private Cloud (VPC) environment. Standard VPC default configurations are insufficient for hypervisor-level software-defined networking. The project must explicitly have VLAN-based networking enabled prior to provisioning bare-metal hardware. This allows VMware NSX to map overlay networks directly into the Google Cloud network architecture.

Furthermore, the target VPC must be configured with a Maximum Transmission Unit (MTU) of 8896 bytes. Standard 1500-byte MTUs will cause encapsulation failure. The elevated MTU accommodates jumbo frames required for high-throughput vSAN ESA mirror traffic and Geneve (Generic Network Virtualization Encapsulation) overlay headers used by NSX Data Center. Without an 8896-byte MTU, packet fragmentation at the hypervisor layer will severely degrade network performance and drop NSX tunnel endpoints (TEPs).

Subnet Name Network Purpose Required MTU Architectural Requirement
Management Subnet ESXi Host Management, vCenter, VCF Operations 8896 Bytes Dedicated Layer 2/3 CIDR routed to Google Cloud VPC
vSAN Subnet vSAN ESA Distributed Storage Traffic 8896 Bytes Isolated, high-bandwidth non-routed or private routed CIDR
vMotion Subnet Hot VM Migration Between Bare-Metal Hosts 8896 Bytes Low-latency dedicated subnet across host network interfaces
NSX Overlay Subnet 1 Geneve TEP Tunnel Communication (Segment A) 8896 Bytes Supports distributed routing and East-West traffic overlay
NSX Overlay Subnet 2 Geneve TEP Tunnel Communication (Segment B) 8896 Bytes Multi-TEP redundancy and North-South Edge transport uplink

Terraform and Toolkit Provisioning Workflow

Provisioning a bare-metal VCF 9.x cluster relies on automated deployment orchestration provided through a specialized VMware solution toolkit. Infrastructure engineers can execute this workflow using the GCP Console interface or programmatically through HashiCorp Terraform modules provided for Google Cloud.

The provisioning sequence executes in five key stages:

  1. VPC and Subnet Validation: The toolkit checks that the target Google Cloud VPC has VLAN attachment capabilities, an 8896-byte MTU, and five allocated subnets.
  2. Node Reservation and Spread Placement: The deployment engine reserves z3-highmem-192-highlssd-metal instances. It applies spread placement policies across distinct physical server racks within the Google Cloud zone to minimize single-rack hardware failure risks.
  3. ESXi Hypervisor Bootstrapping: The bare-metal instances are imaged with the customer-selected ESXi 9.x release image, assigning management IP addresses and binding physical NICs to VMkernel ports.
  4. Cloud DNS and Routing Integration: The toolkit automatically registers forward and reverse DNS records within Google Cloud DNS for ESXi hosts, vCenter Server, NSX Managers, and VCF management components.
  5. VCF Installer Deployment: A pre-validated VCF Installer virtual appliance is deployed into the bootstrap management cluster, initiating the automated installation of vCenter Server, NSX Data Center, and vSAN ESA.
# Example Terraform configuration for Google Cloud VPC Network Prerequisites
# Ensure MTU is explicitly set to 8896 bytes for VCF Bare-Metal compatibility
# Test in staging environments prior to production rollout

resource "google_compute_network" "vcf_vpc" {
  name                    = "vcf-baremetal-vpc"
  auto_create_subnetworks = false
  mtu                     = 8896
}

resource "google_compute_subnetwork" "vcf_management" {
  name          = "vcf-mgmt-subnet"
  ip_cidr_range = "10.100.10.0/24"
  region        = "europe-west1"
  network       = google_compute_network.vcf_vpc.id
}

resource "google_compute_subnetwork" "vcf_vsan" {
  name          = "vcf-vsan-subnet"
  ip_cidr_range = "10.100.20.0/24"
  region        = "europe-west1"
  network       = google_compute_network.vcf_vpc.id
}

resource "google_compute_subnetwork" "vcf_vmotion" {
  name          = "vcf-vmotion-subnet"
  ip_cidr_range = "10.100.30.0/24"
  region        = "europe-west1"
  network       = google_compute_network.vcf_vpc.id
}

vSAN ESA and NetApp Storage Architecture

Primary local storage for the z3-highmem-192-highlssd-metal nodes relies on VMware vSAN Express Storage Architecture (ESA). vSAN ESA replaces traditional disk group designs (which separated caching and capacity tiers) with a single-tier architecture optimized for high-throughput NVMe drives. Each node contributes up to 72 TB of raw local NVMe flash storage to the cluster-wide datastore pool.

vSAN ESA leverages the 200 Gbps network interfaces to handle RAID-5/6 erasure coding, RAID-1 mirroring, and inline compression without creating CPU bottlenecks. Because vSAN ESA processes write paths directly against high-speed NVMe storage pools, it reduces latency for database workloads and large enterprise applications compared to older vSAN architecture models.

For workloads requiring external storage scale independent of compute growth, Google Cloud supports NetApp Volumes as external NFS datastores. Network File System (NFS) mounts connect over private Google Cloud interconnects directly to the ESXi host cluster, enabling capacity expansion without forcing the addition of physical compute nodes.

+-------------------------------------------------------------------------+
|                       VCF 9.x STORAGE OPTIONS                           |
+-------------------------------------------------------------------------+
|                                                                         |
|  +----------------------------------+  +-----------------------------+  |
|  |     PRIMARY LOCAL STORAGE        |  |  SECONDARY/EXTERNAL STORAGE |  |
|  |     vSAN ESA Datastore Pool      |  |     Google Cloud NetApp     |  |
|  |                                  |  |           Volumes           |  |
|  |  * 72 TB NVMe Flash per Node     |  |                             |  |
|  |  * Single-tier Storage Pool      |  |  * NFS v3 / v4.1 Datastores |  |
|  |  * RAID-5/6 Erasure Coding       |  |  * Independent Scaling      |  |
|  |  * Inline Dedupe & Compression   |  |  * Cross-Region Replication |  |
|  +----------------------------------+  +-----------------------------+  |
|                                                                         |
+-------------------------------------------------------------------------+

According to reported vendor claims cited by InfoQ, Broadcom and Google state that this architecture achieves up to 40% lower server TCO through NVMe memory tiering and up to 39% lower storage TCO via vSAN ESA efficiency. However, the specific baseline workloads, hardware pricing references, and testing methodology backing these TCO figures have not been publicly disclosed. Enterprise architects must benchmark their own IOPS profiles, compression ratios, and node growth requirements rather than assuming baseline cost savings.

In terms of failure modes, drive replacements and hardware faults follow a shared operational path. When a local NVMe disk fails on a bare-metal host, vSAN ESA initiates automatic component object rebuilds across surviving cluster nodes. Google’s hardware infrastructure team handles physical disk swapping at the datacentre facility level, while the customer’s storage administrators monitor vSAN object repairs and cluster capacity via VCF Operations.

Managed vs Self-Managed VMware Hyperscaler Matrix

The enterprise hyperscaler ecosystem now offers two distinct operational tracks for running VMware Cloud Foundation: provider-managed and self-managed (customer-operated). Google Cloud’s addition of self-managed bare metal brings it into direct alignment with AWS and Oracle, while contrasting with Microsoft’s managed approach.

Hyperscaler Platform Product Offering Operational Model VCF Version Support Root Access Granted
Google Cloud VCF on Bare Metal Self-Managed (Customer/Partner) VCF 9.x Yes (ESXi, vCenter, NSX)
Google Cloud Google Cloud VMware Engine (GCVE) Provider-Managed (Google) VCF 5.2 (ve1/ve2 nodes) No (Role-Based vCenter)
Amazon Web Services Elastic VMware Service (EVS) Self-Managed (Customer/Partner) VCF 9.0 / 9.1 Yes (Full Control Plane)
Oracle Cloud Oracle Cloud VMware Solution (OCVS) Self-Managed (Customer/Partner) VCF 5.x / 9.x supported Yes (Full Administrative Access)
Microsoft Azure Azure VMware Solution (AVS) Provider-Managed (Microsoft) Provider Managed Releases No (Role-Based Access)

Operating a self-managed environment forces infrastructure teams to absorb Broadcom’s software release cycles directly. Managed services like GCVE or Azure VMware Solution (AVS) validate, test, and apply updates on behalf of the customer, often delaying minor feature access to maintain operational stability. In contrast, self-managed environments allow immediate deployment of current Broadcom patch sets and major releases, but require dedicated engineering resources to perform operational upgrades.

To mitigate operational overhead, organisations frequently contract system integrators and managed service providers. System integrators including Accenture, HCL, IBM, Infosys, TCS, Wipro, and Xtravirt offer managed services for day-2 operations on top of customer-owned cloud infrastructure. As consulting firm Xtravirt highlighted regarding this operational model, full control over a VCF estate is only valuable if an internal team is willing to accept operational accountability—otherwise, third-party management becomes necessary to handle off-hours maintenance and patching.

Provider Pricing and Regional Visibility

Google Cloud has not publicly published specific hourly or monthly pricing rates for z3-highmem-192-highlssd-metal bare-metal instances, nor has it released the complete list of all 22 initial launch regions. Service level agreements (SLAs) covering bare-metal hardware replacement speeds and physical availability are delivered through direct account representative engagements.

Operational Responsibility and License Portability

Transitioning from a managed cloud service to a self-managed bare-metal model shifts software operational maintenance entirely to internal virtualization engineers. Managing day-2 operations involves handling routine health checks, security patching, cluster expansions, and configuration drift management through the VCF Operations platform.

When high-severity zero-day security vulnerabilities affect hypervisor or control plane components, customer operations teams must execute emergency patching workflows. In a provider-managed environment, the cloud vendor schedules and applies mitigations. On bare metal, the engineering team must stage, validate, and execute host remediation cycles—frequently during off-peak maintenance windows at 2 AM—to maintain infrastructure integrity.

+-------------------------------------------------------------------------+
|                  OPERATIONAL RESPONSIBILITY MATRIX                      |
+---------------------------------------------------+---------------------+
| Task / Responsibility                             | Owning Party        |
+---------------------------------------------------+---------------------+
| Physical Datacentre Power, Cooling, & Racks       | Google Cloud        |
| Hardware Fault Diagnostics & Physical NIC Swaps   | Google Cloud        |
| ESXi Hypervisor Patching & Major Upgrades         | Customer / Partner  |
| vCenter Server & NSX Manager Maintenance          | Customer / Partner  |
| Storage Policy & vSAN ESA Rebalance Tuning        | Customer / Partner  |
| Broadcom VCF License Subscription Compliance      | Customer            |
| Guest OS Patching & Application Backups           | Customer            |
+---------------------------------------------------+---------------------+

Licensing operates on a Bring-Your-Own-License (BYOL) structure based on Broadcom’s portable VCF subscription model. Customers purchase subscriptions from Broadcom or authorized distributors and import subscription keys directly into their Google Cloud-hosted VCF Manager. This subscription model allows core entitlements to move between on-premises datacentres and public cloud bare-metal hosts without re-purchasing licenses.

For existing Google Cloud VMware Engine (GCVE) customers looking to adopt VCF 9.x on bare metal, migration requires planning. Google Cloud has not disclosed or provided automated or native migration pathways from GCVE (which runs VCF 5.2) to self-managed VCF 9.x. Migrating production workloads requires manual cross-vCenter vMotion configurations, cold migrations, or executing application-level re-platforming using tools such as VMware HCX.

Implementation Guide for Infrastructure Teams

The following deployment blueprint outlines the technical sequence required to provision a self-managed VCF 9.x cluster within Google Cloud.

Phase 1: VPC Network and Subnet Provisioning

Before launching compute nodes, configure the target Google Cloud project for VLAN-based networking and create custom subnets with an 8896-byte MTU.

# Step 1: Create a custom VPC network with an 8896-byte MTU (test in staging)
gcloud compute networks create vcf-baremetal-vpc \
    --subnet-mode=custom \
    --mtu=8896

# Step 2: Create the primary Management Subnet
gcloud compute networks subnets create vcf-mgmt-subnet \
    --network=vcf-baremetal-vpc \
    --region=europe-west1 \
    --range=10.200.10.0/24

# Step 3: Create the dedicated vSAN Subnet
gcloud compute networks subnets create vcf-vsan-subnet \
    --network=vcf-baremetal-vpc \
    --region=europe-west1 \
    --range=10.200.20.0/24

# Step 4: Validate VPC MTU configuration settings
gcloud compute networks describe vcf-baremetal-vpc --format="get(mtu)"

Phase 2: Executing Provisioning via the VMware Solution Toolkit

Initiate cluster deployment using the Google Cloud Console VMware Solution Toolkit interface or by applying validated Terraform manifests. Specify the target region, project ID, subnet references, and the selected quantity of z3-highmem-192-highlssd-metal instances (minimum cluster node count requirements apply based on vSAN ESA policies).

Phase 3: Host Placement and Cloud DNS Validation

Once the toolkit completes host deployment, verify that spread placement rules have successfully distributed physical nodes across distinct rack failure domains. Verify forward and reverse DNS resolution across all VMkernel and management interfaces.

# Validate Cloud DNS entry resolution for ESXi management nodes
nslookup vcf-esxi-node-01.internal.gcp.domain
nslookup 10.200.10.15

Phase 4: License Key Activation and VCF Operations Bootstrapping

Access the newly deployed VCF Installer interface, input portable Broadcom VCF 9.x license keys, and complete cluster bootstrapping. Register the deployment with internal identity providers (such as Active Directory or Okta via SAML/OIDC) and configure backup targets for vCenter and NSX control plane state.

Common Implementation Mistake

Deploying subnets with default 1500-byte MTUs is the most frequent cause of deployment failure. If the MTU is misconfigured prior to toolkit execution, NSX Overlay interfaces will fail to establish TEP communication, requiring a complete tear-down and re-provisioning of the bare-metal environment.

Key Takeaways for Cloud Architects

Selecting self-managed VCF 9.x on Google Cloud bare metal requires choosing between full platform administrative control and vendor-managed operational convenience. The following criteria should guide architecture review boards when evaluating this solution:

  • Control Plane Requirements: If your security policies, custom hypervisor agents, or compliance frameworks mandate root-level ESXi and NSX access, the bare-metal model fulfills requirements that managed offerings like GCVE cannot.
  • Operational Capability: Ensure internal operations teams possess the capacity to execute lifecycle management, emergency security patching, and vSAN ESA storage optimization. If internal staffing is constrained, factor the cost of third-party managed services partners (e.g., IBM, Accenture, Xtravirt) into total cost calculations.
  • Licensing Strategy Alignment: Organisations with substantial investments in portable Broadcom VCF subscription entitlements can use this platform to move workloads to cloud infrastructure without forfeiting existing software investments.
  • Infrastructure Dependencies: Network architecture prerequisites—specifically VLAN integration and 8896-byte MTU VPC configurations—must be established prior to cluster deployment to avoid deployment failures.

Frequently asked questions

How does self-managed VCF 9.x on Google Cloud differ from Google Cloud VMware Engine (GCVE)?

Google Cloud VMware Engine (GCVE) is a provider-managed service running VCF 5.2 on ve1 and ve2 instances with role-based vCenter permissions and automated patching handled by Google. Self-managed VCF 9.x runs on dedicated z3-highmem-192-highlssd-metal bare-metal nodes, granting customers root access across ESXi, vCenter, and NSX. The customer assumes full responsibility for Day-2 upgrades, lifecycle maintenance, and security patching using portable Broadcom licensing.

What are the mandatory network configuration requirements before deploying bare-metal VCF 9.x?

Before deployment, your Google Cloud project must have VLAN-based networking enabled, and the target VPC must be configured with an 8896-byte MTU to support jumbo frames. Furthermore, infrastructure teams must provision five distinct subnets dedicated to ESXi Management, vSAN storage traffic, vMotion host migrations, and two separate NSX Geneve overlay transport segments.

Can existing VMware Engine VCF 5.2 workloads be migrated directly to self-managed VCF 9.x?

Google has not released automated or native migration tools to transition workloads directly from GCVE (VCF 5.2) to self-managed VCF 9.x on bare metal. Organizations migrating between these environments must plan manual cross-vCenter vMotion operations, cold migrations, or utilize migration tools such as VMware HCX to move virtual machine workloads across clusters.

How is licensing managed for self-managed VMware Cloud Foundation on Google Cloud?

Licensing relies on a Bring-Your-Own-License (BYOL) model. Customers purchase portable Broadcom VCF 9.x subscription licenses directly from Broadcom or authorized partners and apply those keys directly to their Google Cloud bare-metal VCF control plane. This allows organizations to transfer software entitlement keys between on-premises datacentres and public cloud environments without purchasing separate cloud platform software licenses.

Sources and further reading

This article was drafted with AI assistance from the publicly cited sources and passed automated checks for structure and sourcing. Verify critical details against the vendor advisories linked above before acting.