#Upgrading Clusters on VMware vSphere

This document explains how to upgrade Kubernetes clusters on VMware vSphere after the platform-side distribution upgrade is complete. The documented workflow focuses on updating the control plane and worker nodes through Cluster API resources.

#Upgrade Sequence

Upgrade VMware vSphere clusters in the following order:

1. (Prerequisite) Upgrade the ACP platform on the management cluster first. This brings the cluster-api-provider-vsphere controller and the related CAPI components to versions that understand the new schema. Trigger workload-cluster upgrades only after the management-side controllers have rolled out and become Ready.
2. Complete the distribution-version upgrade described in Upgrading Clusters.
3. Verify that the control plane is healthy and the current cluster is stable.
4. Upgrade Kube-OVN to the chart version required by the target ACP release and wait for the AppRelease to reach Success.
5. Upgrade the control plane Kubernetes version.
6. Upgrade worker nodes to the target Kubernetes version.

#Prerequisites

Before you begin, ensure the following conditions are met:

• The distribution-version upgrade is complete.
• The control plane is healthy and reachable.
• All nodes are in the Ready state.
• A current etcd backup has been taken and verified by using the supported ACP backup procedure.
• The target VM template is present in the vSphere environment under the same name as the Alauda OS Image Version value in the OS Support Matrix row. The upgrade fails if the template is not present when the new VSphereMachineTemplate is applied.
• For cross-version upgrades that span more than one Kubernetes minor, the intermediate-version Core images and VM templates are pre-staged. See Cross-Version Upgrade Preparation.
• The target Kubernetes version is compatible with your workloads and add-ons.
• The machine config pools have enough capacity for rolling updates.
• If you rely on pool-managed persistent disks, keep KubeadmControlPlane.spec.rolloutStrategy.rollingUpdate.maxSurge: 0 and each MachineDeployment.spec.strategy.rollingUpdate.maxSurge: 0 so the replacement can reuse the same slot and disk identity.
• Review the Kubernetes upgrade path and version skew policy.

#Required Values From the OS Support Matrix

The authoritative mapping between an ACP release, its VM template, the Kubernetes version, the matching CoreDNS, etcd, and Kube-OVN versions lives in OS Support Matrix. Locate the row that corresponds to the target ACP version before you start; the row supplies every value the steps below need.

The cells you read from that row map to the upgrade manifests as follows:

The CoreDNS and etcd image tags are control-plane-only because clusterConfiguration is a KubeadmControlPlane field. Worker nodes inherit container image versions from the new VM template; the MachineDeployment does not carry its own dns/etcd tags. The Kube-OVN annotation lives on the Cluster resource, not on KubeadmControlPlane, because the vSphere provider watches it independently of the Kubernetes control plane rollout.

#Steps

kubectl patch machinedeployment <cluster_name>-md-0 -n <namespace> \
  --type='merge' -p='{
    "spec": {
      "template": {
        "spec": {
          "version": "<target_kubernetes_version>",
          "infrastructureRef": {
            "name": "<new-worker-template-name>"
          },
          "bootstrap": {
            "configRef": {
              "name": "<new-bootstrap-template-name>"
            }
          }
        }
      }
    }
  }'

kubectl -n <namespace> get machinedeployment <cluster_name>-md-0 -w
kubectl -n <namespace> get machine
kubectl --kubeconfig=/tmp/<cluster_name>.kubeconfig get nodes -o wide

#Recovering From a Failed Phase 2 Upgrade

Do not treat a Kubernetes minor downgrade as an ordinary rollback. Choose the recovery path from the rollout stage:

1. No target-version control-plane Machine has been created: restore the previous Kube-OVN annotation and the previous KubeadmControlPlane and MachineDeployment manifest values. This cancels the target rollout before a new control-plane data format is introduced.
2. Only the machine template or OS image changed, and the Kubernetes minor did not change: point the controlling resource back to the previous template. Cluster API performs another replacement rollout. Keep the Kubernetes minor unchanged.
3. A control-plane Machine on the target Kubernetes minor has joined the cluster: do not patch Kubernetes, CoreDNS, or etcd back to the previous minor. Stop further rollout, repair forward on the target minor, or restore the cluster from the verified pre-upgrade backup by using the supported ACP recovery procedure.

If a target-minor control-plane Machine was created but never joined, first restore healthy etcd quorum and determine whether the failed replacement can be removed safely. Do not assume that changing the version fields alone is sufficient.

Keep these infrastructure facts in mind during any recovery:

• The old VMs are gone. They were destroyed during the upgrade. Template recovery builds a fresh set of replacement machines; it does not restore the original VMs.
• The old VSphereMachineTemplate resource must still exist. Do not delete the previous template until the new rollout is healthy. If you already deleted it, recreate it from version control or backup before attempting same-minor template recovery.
• Pool-managed disk identity is preserved, but data state is not. Disks declared in VSphereMachineConfigPool.spec.configs[].persistentDisks[] reattach to the replacement machines at the same slot, but data written during the upgrade window remains on those disks.

For stage 1, use the vSphere rule in Restore Kube-OVN During Stage-1 Recovery. Both current and legacy vSphere providers restore from the annotation, but the shared rule preserves the chart-version and provider-version boundary. Wait until the restored AppRelease passes the shared verification before changing control-plane manifests.

The KubeadmControlPlane controller can block replacement while etcd is unhealthy. Recover quorum before retrying any safe replacement action.

#Verification

Confirm the following results after the upgrade:

• KubeadmControlPlane reaches the target version and desired replica count.
• MachineDeployment reaches the target version and desired replica count.
• Control plane and worker nodes return to the Ready state.
• The vSphere CPI daemonset remains available in the workload cluster.

#Next Steps

After the Kubernetes upgrade is complete, continue with routine node operations in Managing Nodes on VMware vSphere.