Agent Router data plane installation using Helm

The data plane is installed with plain Helm from a private container registry (Azure Container Registry, Harbor, JFrog Artifactory, Nexus, or Google Artifact Registry), for platform teams that own the Helm release lifecycle.

---

In this guide:

• tetrate syncs Tetrate Agent Router release artifacts into the private registry.
• tetrate applies CRDs outside Helm.
• tetrate renders Helm values.
• Helm installs or upgrades the data plane workload from the private registry.

Plan for 30 to 60 minutes, plus the time to mirror artifacts into the private registry.

info

This procedure installs the data plane only: it does not create a public gateway (Kubernetes Gateway API resource), ingress, DNS record, or TLS certificate. Exposure is covered in Step 10.

Architecture overview

Agent Router Enterprise uses a split-plane model. The management plane, hosted by Tetrate, holds configuration and serves the Developer Console and the Admin Console. The data plane runs in customer-managed cluster infrastructure and handles all AI traffic. The two planes communicate over a single outbound HTTPS connection initiated by the data plane; no inbound connections from the internet are required.

Overview

• Prepare: obtain the credential and set install variables
  • Prerequisites
  • Step 1: obtain the data plane credential
  • Step 2: install the tetrate CLI
  • Step 3: set variables
• Mirror: copy Agent Router artifacts to the private registry
  • Step 4: sync release artifacts
• Install: apply CRDs and deploy with Helm
  • Step 5: apply CRDs
  • Step 6: render Helm values, with the optional egress NetworkPolicy
  • Step 7: create namespaces and pull secrets
  • Step 8: install or upgrade with Helm
• Verify and expose: confirm health, expose the service, and smoke test
  • Step 9: verify
  • Step 10: expose the data plane
  • Step 11: validation and smoke tests
• Operate
  • Upgrade and rollback
  • If artifacts are already mirrored
  • Troubleshooting

Prerequisites

Cluster requirements

Already running Envoy Gateway cluster-wide?

The data plane installs a GatewayClass named tars-egress whose controllerName is the Envoy Gateway default, gateway.envoyproxy.io/gatewayclass-controller. An Envoy Gateway that already watches the whole cluster claims tars-egress too, and the two controllers contend over it. There is no override: install the data plane in a cluster of its own. See A cluster already running Envoy Gateway.

Cluster sizing

The cluster is dedicated to the data plane and needs at least three nodes. The default chart installs several always-on components: the AI Gateway proxy (Envoy-based, shown as egress envoy in manifests and logs) with a minimum of 2 replicas, the AI gateway controller and external processor, the controller and worker, Redis, and the rate-limit service. A single-node footprint is not sufficient.

The egress envoy is the dominant consumer. Its CPU and memory grow roughly linearly with the number of AIGatewayRoute and AIServiceBackend resources, header-mutation rules, and per-route features it holds in memory, so capacity has to follow the number of providers, models, and projects.

Size	Use case	Approximate allocatable target, across three nodes
Small	Dev, test, and low traffic	At least 6 vCPU and 20 GiB RAM
Medium	Staging and light production	At least 12 vCPU and 40 GiB RAM
High	Production with burst headroom	At least 24 vCPU and 80 GiB RAM

Demo installs can start at Small; production installs should start at Medium.

Required access and tools

• An Agent Router data plane credential from the Admin Console, saved as data-plane-credentials.json.
• helm and kubectl installed locally. tetrate is installed in Step 2.
• The release of the management plane, shown at the bottom of the popup that opens from the profile icon in the Admin Console.
• Access to the target Kubernetes cluster.
• Push access to the private registry mirror.
• Pull access from the target cluster to the same private registry.

Conventions

Later steps reuse the shell variables defined in Step 2 and Step 3: set variables, such as TARE_VERSION, DP_CREDENTIAL, and PRIVATE_IMAGE_REGISTRY. The steps assume a single shell session. If the session changes, re-export the variables before continuing.

Step 1: obtain the data plane credential

In the Admin Console, go to Directory → Data Planes.

If this data plane is not registered yet:

1. Click Register Data plane.
2. Set Data plane ID to a stable ID such as acme-prod.
3. Optionally set Name, Description, and Labels.
4. Click Register Data plane.

Then generate the credential from the data plane row:

1. Click the row's edit action to open the Data plane drawer.
2. In the Credential section, click Generate credential.
3. Save the downloaded file as data-plane-credentials.json on the machine the install runs from.

Each data plane uses its own credential, and only one credential is active per data plane. The file downloaded in Step 5 of management plane onboarding is the credential of the default data plane and is named data-plane-credentials-default.json. Either rename it to data-plane-credentials.json, or set DP_CREDENTIAL to its real path in Step 3. A credential can be rotated or revoked from the same Credential section in the data plane edit drawer.

Step 2: install the tetrate CLI

The version of tetrate decides the version of the data plane. Mirroring, CRDs, and rendered values all come from the release the CLI belongs to, and the Helm chart in Step 8 has to be the same release. The CLI is therefore installed at the release of the management plane, not at the latest release.

Set TARE_VERSION to the management plane release, keeping the leading v, and run the installer at that version:

TARE_VERSION=v<management-plane-release>
curl -fsSL https://tare.tetrate.ai/tools/install.sh | bash -s -- --version "${TARE_VERSION}"

The installer verifies the signed release manifest and installs tetrate into ~/.tetrate/bin. Add that directory to PATH when the installer says it is missing, then verify the version:

export PATH="$PATH:$HOME/.tetrate/bin"
tetrate --version

The reported version has to match TARE_VERSION. When the management plane moves to a new release, reinstall the CLI at that release before upgrading, as described in Upgrade and rollback.

Step 3: set variables

For Azure Container Registry (ACR):

DP_CREDENTIAL=./data-plane-credentials.json
PRIVATE_REGISTRY_HOST=${ACR_NAME}.azurecr.io
PRIVATE_IMAGE_REGISTRY=${ACR_NAME}.azurecr.io/tare
PULL_SECRET=tare-registry-pull

For another private registry:

DP_CREDENTIAL=./data-plane-credentials.json
PRIVATE_REGISTRY_HOST=registry.acme.example.com
PRIVATE_IMAGE_REGISTRY=registry.acme.example.com/tare
PULL_SECRET=tare-registry-pull
REGISTRY_USERNAME=<registry-user>
REGISTRY_PASSWORD=<registry-password-or-token>

Step 4: sync release artifacts

Log in locally with credentials that can push to the destination registry.

For ACR:

az acr login --name "${ACR_NAME}"

For a generic private registry:

printf '%s' "${REGISTRY_PASSWORD}" | \
  docker login "${PRIVATE_REGISTRY_HOST}" \
    --username "${REGISTRY_USERNAME}" \
    --password-stdin

Copy the pinned Agent Router images and the serve-helm OCI chart. The sync mirrors the release of the installed tetrate, which is why Step 2 comes first:

tetrate agentrouter dataplane install "${DP_CREDENTIAL}" \
  --image-sync "${PRIVATE_IMAGE_REGISTRY}" \
  --sync-only \
  --parallel 2 \
  --stall-threshold 5m

tetrate uses the data plane credential to pull from Tetrate's source registry. The local registry login is used only to push to the private registry.

Step 5: apply CRDs

Apply CRDs outside Helm:

tetrate agentrouter dataplane install "${DP_CREDENTIAL}" \
  --crds-only \
  --image-registry "${PRIVATE_IMAGE_REGISTRY}"

CRDs are intentionally managed outside the Helm release so helm uninstall does not delete cluster-scoped APIs and cascade-delete custom resources.

Step 6: render Helm values

Render values for the Helm release:

tetrate agentrouter dataplane install "${DP_CREDENTIAL}" \
  --image-registry "${PRIVATE_IMAGE_REGISTRY}" \
  --gateway-hostname=ai-gateway.acme.com \
  --print-helm-values > values.yaml

--gateway-hostname declares a hostname client applications call, and the rendered values carry it under global.gateway.hostnames. The flag is repeatable; put the name callers should use first. The hostname does not need to resolve yet. When it is not known yet, leave the flag out and add the list to values.yaml later, as described in Step 10. Every later render that regenerates values.yaml needs the same flags, or the list is lost.

Behind a corporate proxy, pass the proxy flags explicitly: --http-proxy, --https-proxy, and --no-proxy. The installer asks about a proxy only when it runs in a terminal, so a scripted render, or one whose standard input is already used by --image-pull-secret-stdin, configures no proxy, silently. Every later render that regenerates values.yaml needs the same flags, or the proxy configuration is lost. The flags, the proxy topologies, and how to verify the result are in Outbound proxy.

Treat values.yaml as sensitive because it contains the data plane identity secret. Do not commit it as plaintext. Use SOPS, Sealed Secrets, External Secrets, or the organization's approved secret workflow.

Make sure the rendered values do not set the legacy serveUrl to the management-plane API URL. Keep these fields empty:

global:
  serveUrl: ""
controller:
  config:
    proxyUrl: ""

If needed, override them during Helm install:

--set-string global.serveUrl= \
--set-string controller.config.proxyUrl=

Optional: restrict the gateway pods' egress with a NetworkPolicy

From 0.7.0 the chart can install a NetworkPolicy for the gateway proxy pods. It is off by default. When enabled, the pods can reach DNS, every pod in the cluster, and any off-cluster address outside the internal and reserved ranges (cloud metadata, RFC 1918, carrier-grade NAT, link-local, loopback, multicast, and the other reserved IPv4 and IPv6 ranges). Every other connection is refused, which stops the gateway fetching a gateway auth policy's jwks_uri from an internal address. The policy takes effect only on a CNI that enforces NetworkPolicy, such as Calico or Cilium, and does not match an internal IPv4 address reached through NAT64.

Value	Default	What it does
egressResources.egress.networkPolicy.enabled	false	Renders the policy
egressResources.egress.networkPolicy.extraEgress	[]	Egress rules appended as written, for each off-cluster private destination the pods still need
egressResources.egress.networkPolicy.inClusterNamespaceSelector	{}	Narrows which namespaces' pods stay reachable; empty keeps every namespace, which the gateway needs for its external processor, guardrails filter, OTLP receiver, and any in-cluster model service
egressResources.egress.networkPolicy.allowIPv6	true	Allows off-cluster IPv6 outside the internal ranges; false allows no off-cluster IPv6 at all
egressResources.egress.networkPolicy.extraExceptCIDRs, extraExceptCIDRsV6	[]	More off-cluster ranges to refuse, such as an on-premises management subnet

Before setting enabled: true, list in extraEgress every off-cluster private destination the gateway still reaches. The ones commonly missed:

• identity providers and their key endpoints
• outbound proxies
• the management plane, where it is reached over a private network
• telemetry collectors
• model endpoints on a private network or a private link
• a node-local DNS cache on a link-local address, such as 169.254.20.10

egressResources:
  egress:
    networkPolicy:
      enabled: true
      extraEgress:
        - to:
            - ipBlock:
                cidr: 10.20.0.0/16
          ports:
            - protocol: TCP
              port: 443

Add the block to values.yaml after rendering it in Step 6, and keep it there for every later helm upgrade.

Step 7: create namespaces and pull secrets

Create the data plane namespaces:

kubectl create namespace tars-system --dry-run=client -o yaml | kubectl apply -f -
kubectl create namespace tars-dataplane --dry-run=client -o yaml | kubectl apply -f -

For Azure Kubernetes Service (AKS) attached to ACR, no image pull secret is usually needed after running:

az aks update \
  --resource-group "${RESOURCE_GROUP}" \
  --name "${AKS_CLUSTER_NAME}" \
  --attach-acr "${ACR_NAME}"

For a private registry that requires Kubernetes pull credentials, create the same secret in both namespaces:

kubectl create secret docker-registry "${PULL_SECRET}" \
  --docker-server="${PRIVATE_REGISTRY_HOST}" \
  --docker-username="${REGISTRY_USERNAME}" \
  --docker-password="${REGISTRY_PASSWORD}" \
  --namespace tars-system \
  --dry-run=client -o yaml | kubectl apply -f -

kubectl create secret docker-registry "${PULL_SECRET}" \
  --docker-server="${PRIVATE_REGISTRY_HOST}" \
  --docker-username="${REGISTRY_USERNAME}" \
  --docker-password="${REGISTRY_PASSWORD}" \
  --namespace tars-dataplane \
  --dry-run=client -o yaml | kubectl apply -f -

When using a pull secret, render values with:

tetrate agentrouter dataplane install "${DP_CREDENTIAL}" \
  --image-registry "${PRIVATE_IMAGE_REGISTRY}" \
  --image-pull-secret-name "${PULL_SECRET}" \
  --gateway-hostname=ai-gateway.acme.com \
  --print-helm-values > values.yaml

Step 8: install or upgrade with Helm

The serve-helm chart is published at the Agent Router release version, without the leading v. Derive it from the CLI version set in Step 2, so the chart always matches the CLI, the CRDs, and the rendered values:

: "${TARE_VERSION:?set TARE_VERSION as in Step 2}"
CHART_VERSION="${TARE_VERSION#v}"

An unset TARE_VERSION would leave the chart version empty, and Helm would then pull the newest chart in the registry, so the line above stops with an error instead.

Install or upgrade the release:

helm upgrade --install tars "oci://${PRIVATE_IMAGE_REGISTRY}/serve-helm" \
  --version "${CHART_VERSION}" \
  -f values.yaml \
  -n tars-system \
  --skip-crds \
  --set ai-gateway-crds.enabled=false \
  --set-string global.serveUrl= \
  --set-string controller.config.proxyUrl=

Pass the same values.yaml on every later helm upgrade. An upgrade that passes other values without the file that carries global.gateway.hostnames, such as --set flags alone, drops the key, and the readiness probe falls back to the address stored for the data plane.

Use --skip-crds and --set ai-gateway-crds.enabled=false together. The first skips chart crds/ entries; the second disables AI Gateway CRDs rendered as normal Helm templates.

A release chart carries the release version, such as 0.7.0, and since 0.7.0 a prerelease chart reads 0.7.0-rc.1.gabc1234, one string that is valid as both a SemVer version and an OCI tag. A chart from before 0.7.0 carried its commit as + build metadata, such as 0.6.0-rc.3+8d71fcc: OCI stores that tag with _, but helm --version still takes the + form.

Step 9: verify

Check the Helm release:

helm list -n tars-system
helm status tars -n tars-system

Wait for workloads:

kubectl rollout status deployment/controller -n tars-system --timeout=300s
kubectl rollout status deployment/egress -n tars-dataplane --timeout=300s

Confirm images come from the private registry:

kubectl get deploy egress -n tars-dataplane \
  -o jsonpath='{range .spec.template.spec.containers[*]}{.name}={.image}{"\n"}{end}'

Run doctor:

tetrate agentrouter doctor "${DP_CREDENTIAL}" --verbose

A fresh data plane with no routes can report warnings for missing RouteDeployments, AIGatewayRoutes, HTTPRoutes, or EnvoyPatchPolicies. The core install is healthy when pods roll out and images pull from the private registry.

Step 10: expose the data plane

For gateway exposure, follow the published cloud install guide for the target platform. On a self-managed cluster (k3s, kubeadm, OpenShift), follow Expose the gateway on a generic cluster, which uses the Service below. Any other ingress or Gateway stack can target the same Service.

The data plane serves traffic through the in-cluster egress service:

• Service: egress
• Namespace: tars-dataplane
• Port: 10080
• Paths: /v1/*, /mcp/*, and /.well-known/*

The hostname client applications call is declared in the chart, not in the Admin Console. If values.yaml does not carry it yet, add the list and re-run the helm upgrade from Step 8:

global:
  gateway:
    hostnames:
      - ai-gateway.acme.com   # first: the address the consoles print

Confirm the chart rendered it. The Admin Console lists the same name on the data plane's page under Directory → Data Planes after the next configuration poll:

kubectl -n tars-system get configmap tars-gateway-declaration -o jsonpath='{.data.gateway-hostnames}'

Declaring the hostname creates no DNS record, certificate, or route; the gateway stack above provides those. A data plane older than 0.7.0 cannot declare a hostname, and its address is set through the deprecated SetDataplaneURL admin API. The rules for the list and the certificate settings are in Declare the hostnames a data plane serves.

Step 11: validation and smoke tests

A healthy install can still read Degraded in the Console

Enabling a provider or model in the organization catalog makes it available to grant, not routable. Nothing routes through this data plane until a model is granted to a project attached to it, and until then the cluster reads healthy while the Console shows Degraded and the gateway's model list stays empty. That state is configuration, not a failed install: see Add providers and models to projects.

Validate the Helm install first:

helm status tars -n tars-system
kubectl rollout status deployment/controller -n tars-system --timeout=300s
kubectl rollout status deployment/egress -n tars-dataplane --timeout=300s
kubectl get deploy egress -n tars-dataplane \
  -o jsonpath='{range .spec.template.spec.containers[*]}{.name}={.image}{"\n"}{end}'
tetrate agentrouter doctor "${DP_CREDENTIAL}" --verbose

Then run traffic smoke tests after onboarding has configured provider keys and routes. Until the gateway is exposed, use a local port-forward to the in-cluster egress service:

kubectl port-forward -n tars-dataplane svc/egress 18080:10080

In another terminal, set the local endpoint and an API key from the Console:

export DP_SCHEME=http
export DP_HOST=127.0.0.1:18080
export TETRATE_API_KEY=<your-api-key-from-router-app>

At minimum, verify model listing with and without authentication:

curl -s -o /tmp/tare-models.json -w "%{http_code}\n" \
  "${DP_SCHEME}://${DP_HOST}/v1/models" \
  -H "Authorization: Bearer ${TETRATE_API_KEY}"

curl -s -o /tmp/tare-models-no-auth.json -w "%{http_code}\n" \
  "${DP_SCHEME}://${DP_HOST}/v1/models"

Expected result: the authenticated request returns 200; the unauthenticated request returns 401.

Also smoke test every API shape enabled during onboarding, such as /v1/chat/completions, /v1/responses, /v1/messages, /v1/embeddings, /v1/images/generations, /v1/audio/speech, and Model Context Protocol (MCP) profile paths.

Upgrade and rollback

Use the same split workflow for upgrades. The tetrate CLI moves first: it is reinstalled at the release of the management plane, then the artifacts are re-synced, the CRDs are updated, and the Helm workload release is upgraded.

Reinstall the CLI at the new release and verify it:

TARE_VERSION=v<management-plane-release>
curl -fsSL https://tare.tetrate.ai/tools/install.sh | bash -s -- --version "${TARE_VERSION}"
tetrate --version

Mirror the artifacts of the new release with the sync command from Step 4. The chart for the new release is not in the registry until this is done.

If the new Agent Router release includes CRD changes, re-run CRD apply:

tetrate agentrouter dataplane install "${DP_CREDENTIAL}" \
  --crds-only \
  --image-registry "${PRIVATE_IMAGE_REGISTRY}"

Run it with the tetrate of the release being upgraded to. When that release raises the Gateway API requirement, as 0.7.0 does, the command stops on a cluster below it: move the CRDs first through the managed add-on that owns them, or, where tetrate installed them, add --force-gateway-api-crds. See Upgrading to 0.7.0.

Render values for the new release:

tetrate agentrouter dataplane install "${DP_CREDENTIAL}" \
  --image-registry "${PRIVATE_IMAGE_REGISTRY}" \
  --gateway-hostname=ai-gateway.acme.com \
  --print-helm-values > values.yaml

If a pull secret is used, include it when rendering values:

tetrate agentrouter dataplane install "${DP_CREDENTIAL}" \
  --image-registry "${PRIVATE_IMAGE_REGISTRY}" \
  --image-pull-secret-name "${PULL_SECRET}" \
  --gateway-hostname=ai-gateway.acme.com \
  --print-helm-values > values.yaml

Upgrade the workload release:

CHART_VERSION="${TARE_VERSION#v}"

helm upgrade tars "oci://${PRIVATE_IMAGE_REGISTRY}/serve-helm" \
  --version "${CHART_VERSION}" \
  -f values.yaml \
  -n tars-system \
  --skip-crds \
  --set ai-gateway-crds.enabled=false \
  --set-string global.serveUrl= \
  --set-string controller.config.proxyUrl=

Rollback uses Helm's normal rollback flow:

helm rollback tars -n tars-system

CRDs are not affected by rollback because they are not part of the Helm release manifest. Existing custom resources remain in the cluster. Leave the CRDs at the newer release: do not re-run tetrate agentrouter dataplane install --crds-only with the previous tetrate. CRDs move forward only, and from 0.7.0 on the cluster refuses the previous Gateway API CRDs. See Rolling back.

If artifacts are already mirrored

Skip the --image-sync step and start at CRD apply. The installed tetrate and TARE_VERSION still have to match the release that was mirrored:

tetrate agentrouter dataplane install "${DP_CREDENTIAL}" \
  --crds-only \
  --image-registry "${PRIVATE_IMAGE_REGISTRY}"

Then render values and install with Helm as above.

Troubleshooting

Destination registry reports repository not found

Create the destination repository before syncing. For example, with Google Artifact Registry:

gcloud artifacts repositories create tare \
  --repository-format docker \
  --location <region>

ACR and many enterprise registries create repositories on first push, but some registries require pre-creation.

Helm reports invalid ownership metadata

This usually means Helm is trying to adopt CRDs already applied by tetrate agentrouter dataplane install --crds-only.

Example:

CustomResourceDefinition "aigatewayroutes.aigateway.envoyproxy.io" exists and cannot be imported into the current release: invalid ownership metadata

Install or upgrade with both:

--skip-crds \
--set ai-gateway-crds.enabled=false

Helm reports improper constraint

helm --version expects a SemVer constraint, not a bare commit SHA. Use the chart's full SemVer, such as 0.7.0 for a release or 0.7.0-rc.1.gabc1234 for a prerelease (a chart from before 0.7.0 reads 0.6.0-rc.3+8d71fcc).

Bare commit SHAs are rejected before Helm contacts the registry. The registry stores SemVer build metadata with _, but helm --version should use the SemVer form with +.

Image sync fails with 401 Unauthorized

A 401 Unauthorized on HEAD https://registry.tetrate.ai/v2/... during the sync means the credential is not authorized to pull from registry.tetrate.ai. The credential file downloads normally even when its registration with registry.tetrate.ai did not complete, so the failure only shows at the sync.

Before the data plane is installed, generate a new credential in the Admin Console under Directory → Data Planes, from the Credential section of the data plane's edit drawer, and retry the sync. On a data plane that is already running, generating a credential is a rotation that revokes the one in use; follow Data plane credentials instead.

Pods are stuck in ImagePullBackOff

Check that:

• the private registry contains every image tag for the Agent Router release
• the cluster identity can pull from the private registry
• pull secrets exist in both tars-system and tars-dataplane when required
• the pull secret is referenced from values.yaml; if not, re-render values with --image-pull-secret-name
• values.yaml points global.imageRegistry at the intended registry

Helm cannot pull the chart from the mirror

If helm install or helm upgrade fails locally with pull access denied, log in to the OCI registry from the operator machine:

helm registry login "${PRIVATE_REGISTRY_HOST}"

This is separate from Kubernetes image pull access. Helm pulls the chart artifact from the local machine; the cluster pulls workload images using node identity or image pull secrets.

---

Where to go next

Gateway installation

Install the data plane gateway components that manage inbound access and configure request routing.

Self-Hosted Data Plane

See the full install sequence: gateway, SSO, then developer handoff.