Upgrade a self-hosted data plane

The order and the rules for keeping a self-hosted data plane current: what upgrades first, what keeps working while the data plane lags a release, how to read the release notes before upgrading, and when a patch is worth taking off-schedule. The command itself is covered by the tare upgrade reference; this page is the part around the command.

The order: the management plane leads, the data plane follows

The Tetrate-hosted management plane is always on the current release; Tetrate upgrades it as part of the weekly cut. A data plane is therefore never newer than its management plane, and the operator's half of every upgrade is the data plane and the tare CLI, whose version matches the release tag.

Because the management plane is always current, "upgrade the management plane first" is not a step an operator performs: it has already happened by the time a release appears on the release notes page. The operator's decision is only when the data plane follows.

Nothing forces an immediate upgrade

Each release is supported for six months from the day it ships, and a supported data plane keeps working against the current management plane for that whole window. The end-of-support date for every release is on its release notes entry and on Release compatibility. Since 0.6.0 the Admin Console also warns in place: a data plane whose release is within 30 days of its end of support, or past it, is marked on the data plane list under Directory → Data Planes, and its detail page shows a notice. The warning means the data plane's release is leaving the support window, so its upgrade should be planned; it reads the data plane's reported version against the release archive, and a release whose end date is not yet published shows nothing.

Fixes are not backported: getting a fix means upgrading to a release that contains it. That, rather than compatibility, is what usually decides the timing.

What lags while the data plane is behind

A management plane feature can need a newer data plane. The feature stays off for that data plane until it is upgraded; nothing breaks, and the rest of the release keeps working. Each such feature names its version in the release notes ("available once the data plane is on X or later"), and the Minimum data plane column on Release compatibility carries the highest such version per release: a data plane at or above it has every feature of that release.

Skipping releases: an upgrade goes direct

A data plane several releases behind upgrades in one move, not release by release. The upgrade re-renders the deployment at the target tag, whichever path installed it, so there is no chain of intermediate upgrades to walk. No release to date has restricted which release an upgrade may start from or required a stepping-stone release in between; the release contract reserves fields for both, so a release that ever introduces such a requirement will carry it in its Upgrade and rollback block, which is the thing to read before any jump.

What a multi-release jump does change is the reading list. Everything the skipped releases shipped arrives at once: their behavior changes, their deprecations, and any per-release required action all land in the same upgrade, so the notes of every release between the current one and the target are part of the pre-upgrade read, not just the target's. The release notes index lists them in order.

Read the release before upgrading

Every entry on the release notes page carries an Upgrade and rollback block. Four things in it decide what the upgrade needs:

1. The breaking-change line. When operator action is required before upgrading, the entry carries a warning naming the action. A release can also carry a breaking change that requires no action; the line says which.
2. The traffic line. Most upgrades roll through without dropping traffic. When a release does not, the line says so, and a maintenance window is needed.
3. The rollback line. Rollback to the previous release is unconditional unless the entry carries a rollback condition; a conditional rollback names exactly what stops working after rolling back.
4. The Kubernetes range. The range is fixed per release and enforced at install and upgrade time, and managed Kubernetes cannot be downgraded in place, so a cluster below a new release's floor must be recreated before the data plane can follow. The range per release is on Release compatibility.

Deprecations is the fifth check, on a slower clock: anything deprecated keeps working for twelve months, and the running list shows what is on the clock and its replacement.

When to take a patch off-schedule

Releases are cut every Wednesday, and a patch ships when it is needed rather than waiting for the next cut. A patch that carries only security updates, the shape v0.2.1 had, changes no source, API, configuration, or behavior: it rolls through without dropping traffic and rolls back unconditionally, so the cost of taking it immediately is low and the reason it shipped early is the reason not to wait.

Who can run an upgrade

Since 0.5.0, the upgrade operations sit behind their own authorization boundary: they require the dataplane_operator scope on the platform credential, and an organization admin credential can no longer run them. On a Tetrate-hosted management plane the grant is prepared by Tetrate; on a deployment that runs its own management plane, the upgrade prepares it, and the release is deployed in a fixed order. The boundary, the order, and what fails when a step is skipped are on Data plane upgrade authorization.

Upgrading to 0.5.0

The 0.5.0 upgrade carries two required actions of its own. Both are checked up front, so missing one fails the upgrade rather than half-applying it:

1. Set RBAC_AUTH_DB_MODE in the auth values. The setting declares whether the deployment owns its auth database or shares a hosted one. An installation that never set it was running as shared, so keeping shared preserves the behavior it has today; the render fails when the setting is absent rather than the upgrade half-applying.
2. Make sure the platform credential gains the dataplane_operator scope. The upgrade's bootstrapper job adds it with a database migration; when the bootstrapper is disabled, the migration is run by hand first, per Data plane upgrade authorization. A deployment upgraded without it keeps serving traffic but loses the ability to upgrade its data plane, and the resulting permission_denied does not name the cause.

Upgrading to 0.6.0

Before 0.6.0, a proxy setting changed on the management plane did not reach a data plane that serves project gateways: the change was stored and shown as applied, while those gateways kept the values rendered at their last gateway change, often from when the gateway was created. From 0.6.0 the change is delivered. The consequence for the upgrade is that any proxy setting changed while running an earlier release may apply for the first time once the data plane is on 0.6.0: cost and fee handling, overrun thresholds, cache sizing, and the request-log mode set from the Admin Console are all carried in those settings. Cost figures a project-gateway data plane recorded before the upgrade reflect the older values. Check each such setting after upgrading and confirm the value in effect is the intended one.

Upgrading to 0.7.0

0.7.0 moves the data plane to Envoy Gateway v1.9.1, which raises two cluster requirements. Meet both before the data plane upgrade starts. tare checks both before anything is applied, so a cluster that misses either is refused rather than half-upgraded. The Helm and GitOps paths check only the Gateway API CRDs, so confirm the Kubernetes version there yourself:

1. Kubernetes 1.33.0 to 1.36.x. The range was 1.32.0 to 1.35.x. A cluster on 1.32 is upgraded first. Kubernetes 1.37 is outside the range.
2. Gateway API CRDs at bundle v1.6.0 or later. The data plane is tested against v1.6.1. v1.6.0 has the same CRD schemas and is accepted.

The cluster's Kubernetes version, Gateway API bundle, and release channel:

kubectl version
kubectl get crd gateways.gateway.networking.k8s.io \
  -o go-template='{{ index .metadata.annotations "gateway.networking.k8s.io/bundle-version" }} {{ index .metadata.annotations "gateway.networking.k8s.io/channel" }}{{ "\n" }}'

Who moves the Gateway API CRDs depends on who installed them. The data plane never takes them over from a managed add-on:

Cluster	Owner of the Gateway API CRDs	Action before upgrading the data plane
GKE with the Gateway API add-on (--gateway-api=standard, as in the GCP installation guide)	GKE	Upgrade the cluster to GKE 1.36.4-gke.1391000 or a later 1.36 patch, available on the Regular channel, where the add-on installs v1.6.0. Then hold it on 1.36.
CRDs installed by tare install (channel experimental, or no channel shown)	tare	None beforehand: run tare upgrade with --force-gateway-api-crds. On the Helm path, pass the same flag to tare install --crds-only.
Standard-channel CRDs installed by the operator (EKS, AKS as in the installation guides, or any other cluster)	The operator	Apply the v1.6.1 standard-channel manifest, then upgrade.
GitOps, with dataplane-crds-helm set to crds.gatewayAPI: true	The CRDs Application	Sync the CRDs Application at 0.7.0, then the workload Application.

The standard-channel manifest for the third row:

kubectl apply --server-side \
  -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.6.1/standard-install.yaml

--force-gateway-api-crds is refused on a standard-channel cluster: it applies the experimental CRDs the data plane bundles, which a managed add-on or the Gateway API admission policy rejects.

What else changes in the 0.7.0 upgrade

Three things happen during or after the upgrade that an operator plans around, none of which stops it:

• Upgrade the management plane before the data planes. On a Tetrate-hosted management plane this has already happened. On a deployment that runs its own, the order matters from 0.7.0 on: the data plane now removes the identity headers only the gateway sets, and every x-ai-eg-* and x-internal-traces-* header, from every request on arrival. A data plane upgraded ahead of its management plane removes x-tars-idp-token before the gateway can verify it, on any gateway that already has a gateway auth policy. Callers that send one of those headers for a model provider or MCP server to read need to move the value to a header the data plane forwards before the upgrade, because the upstream no longer receives it. The full list is under Headers removed from every request.
• Every gateway pod restarts once. The data plane moves to Envoy AI Gateway v1.1.0, and the guardrails filter is now built with the rest of the data plane, so the gateway image tag changes and the upgrade rolls every gateway proxy. The rollout keeps traffic flowing under the pod disruption budget, as any gateway image change does.
• The router_* metrics on the Envoy Prometheus endpoint are renamed. On :19001/stats/prometheus each one now carries an envoy_dynamicmodulescustom_ prefix: envoy_dynamicmodulescustom_router_requests_total where router_requests_total was exposed before. Anyone who scrapes that endpoint directly updates the dashboards and alerts that query router_* names. Metrics sent to the management plane, metrics shipped through an OpenTelemetry Collector that strips the dynamicmodulescustom. scope, and tare doctor are unaffected. See the data plane metrics.

Hold GKE on 1.36

GKE 1.37 is outside the range 0.7.0 supports, and a cluster on a release channel moves to it automatically when the channel does. A cluster cannot be downgraded in place, so stop minor upgrades before that happens. A no_minor_upgrades maintenance exclusion keeps the cluster on 1.36 and still takes 1.36 patches:

gcloud container clusters update "${CLUSTER_NAME}" --zone "${ZONE}" \
  --add-maintenance-exclusion-name=hold-gke-1-36 \
  --add-maintenance-exclusion-scope=no_minor_upgrades \
  --add-maintenance-exclusion-until-end-of-support

Remove the exclusion with --remove-maintenance-exclusion=hold-gke-1-36 once the data plane runs a release that supports 1.37.

AKS Managed Gateway API is not covered

A cluster that runs AKS Managed Gateway API (--enable-gateway-api) is outside the Azure installation guide, which does not enable it. On AKS 1.36 that feature installs Gateway API v1.5.1, below the requirement, and blocks installing the CRDs by hand. Contact Tetrate support before upgrading such a cluster to 0.7.0.

Run the upgrade

• tare CLI path: tare upgrade with the data plane's identity file. The command upgrades atomically and rolls back on failure; flags, the doctor rows, and the drain behavior are in the tare upgrade reference. Since 0.5.0 the command also stops when the cluster's Gateway API CRDs are older than the release requires; when tare owns the CRDs, --force-gateway-api-crds (or TARE_FORCE_GATEWAY_API_CRDS=1) proceeds anyway.
• Helm and GitOps paths: the upgrade is a re-render at the new tag through the same path that installed: pull the new chart version for a Helm install, or re-render and commit the manifests for a GitOps install, CRDs first where they sync as their own Application. Since 0.6.0 these paths get the same Gateway API CRD check as the CLI, run inside the cluster: an install job stops the rollout when the cluster's CRDs are older than the release requires, and the job's log names the version to move to. Move the CRDs there first, through the cloud's managed add-on where one owns them, or by applying the upstream Gateway API release manifest.

Afterward, tare doctor confirms component state, and the data plane's reported version appears in the Admin Console data plane list's Version column. See Check the management plane version and release notes.

Rolling back

Rollback to the previous release is unconditional unless that release's notes carry a rollback condition. A conditional rollback never blocks the rollback itself: the condition names what stops working until the data plane is rolled forward again, so the decision can be made with the blast radius known.

A rollback reverts the data plane workloads, not the CRDs. Every path installs the CRDs outside the workload release (the tare CRD phase, tare install --crds-only, or the CRDs Application), so the CRDs stay at the newer release. Re-applying the previous release's CRDs is not a rollback step: CRDs move forward only.

Rolling back from 0.7.0

0.7.0 carries a rollback condition: the workloads go back to 0.6.x, and the Gateway API, Envoy Gateway and Envoy AI Gateway CRDs stay at 0.7.0's versions. 0.6.x runs on them unchanged: Gateway API v1.6 still serves every API version 0.6.x reads, and the Envoy Gateway and Envoy AI Gateway CRDs only gained fields, which 0.6.x ignores.

The CRDs cannot follow the workloads back. After the upgrade, TCPRoute, UDPRoute and TLSRoute are stored at v1, which the previous Gateway API bundle does not define, so the API server refuses an older CRD that drops it. Where the data plane installed the Gateway API CRDs, the safe-upgrades.gateway.networking.k8s.io admission policy also refuses any bundle older than v1.5.

Path	Roll back with	Do not
tare CLI	helm rollback tars -n tars-system	Run a 0.6.x tare upgrade or tare install to go back.
Helm	helm rollback tars -n tars-system	Re-run tare install --crds-only with a 0.6.x tare.
GitOps	Point the workload Application back at the 0.6.x render.	Move the CRDs Application back; keep it at 0.7.0.
GKE with the Gateway API add-on	Roll back the data plane as above.	Downgrade the cluster; it stays on GKE 1.36.

A 0.6.x tare re-applies 0.6.x's CRDs: the Gateway API ones, where it applies them, are refused, and the Envoy Gateway and Envoy AI Gateway ones would drop the fields 0.7.0 added. helm rollback restarts the gateway pods once more, onto the 0.6.x images.

helm history tars -n tars-system
helm rollback tars -n tars-system --wait --timeout 10m

---

Where to go next

tare upgrade

Flags, atomic rollback, drain behavior, and the doctor rows.

Release notes

What changed in each release, with upgrade and rollback notes.

Release compatibility

Kubernetes range, Envoy Gateway, and minimum data plane per release.

Deprecations

The removal policy and what is on the clock today.