# Installation guide for Helm > Install the Agent Router data plane with Helm for teams that own the release lifecycle and run a private container registry. # Agent Router data plane installation using Helm This guide is for platform teams that own the Helm release lifecycle and already have a private container registry, such as Azure Container Registry, Harbor, JFrog Artifactory, Nexus, or Google Artifact Registry. In this guide: - `tare` syncs TARE release artifacts into the private registry. - `tare` applies CRDs outside Helm. - `tare` renders Helm values. - Helm installs or upgrades the data plane workload from the private registry. Plan for 30 to 60 minutes, plus the time to mirror artifacts into the private registry. :::info This guide covers data plane installation only. It does not create a public Gateway, ingress, DNS record, or TLS certificate. For gateway exposure, follow the published cloud install guide for the target platform, or use the organization's preferred ingress or Gateway stack. ::: ## Architecture overview Agent Router Enterprise uses a split-plane model. A **Management Plane** hosted by **Tetrate** holds configuration and serves the web UI. A **Data Plane** runs in customer-managed cluster infrastructure and handles all AI traffic. The two planes communicate over a single outbound HTTPS connection initiated by the data plane; no inbound connections from the internet reach the management plane. ```mermaid architecture-beta group customer(cloud)[Customer hosted] group tetrate(cloud)[Tetrate hosted] service mp(server)[Management Plane] in tetrate service dp(server)[Data Plane] in customer mp:R -- L:dp ``` ## Overview - **Prepare**: obtain the credential and set install variables - [Prerequisites](#prerequisites) - [Step 1 - Obtain the data plane credential](#step-1---obtain-the-data-plane-credential) - [Step 2 - Set variables](#step-2---set-variables) - **Mirror**: copy TARE artifacts to the private registry - [Step 3 - Sync release artifacts](#step-3---sync-release-artifacts) - **Install**: apply CRDs and deploy with Helm - [Step 4 - Apply CRDs](#step-4---apply-crds) - [Step 5 - Render Helm values](#step-5---render-helm-values) - [Step 6 - Create namespaces and pull secrets](#step-6---create-namespaces-and-pull-secrets) - [Step 7 - Install or upgrade with Helm](#step-7---install-or-upgrade-with-helm) - **Verify and expose**: confirm health, expose the service, and smoke test - [Step 8 - Verify](#step-8---verify) - [Step 9 - Expose the data plane](#step-9---expose-the-data-plane) - [Step 10 - Validation and smoke tests](#step-10---validation-and-smoke-tests) - **Operate** - [Upgrade and rollback](#upgrade-and-rollback) - [If artifacts are already mirrored](#if-artifacts-are-already-mirrored) - [Troubleshooting](#troubleshooting) --- ## Prerequisites - A TARE data plane credential from the dashboard, saved as `data-plane-credentials.json`. - `tare`, `helm`, and `kubectl` installed locally. - Access to the target Kubernetes cluster. - Push access to the private registry mirror. - Pull access from the target cluster to the same private registry. ## Conventions Later steps reuse the shell variables defined in [Step 2 - Set variables](#step-2---set-variables), such as `DP_CREDENTIAL` and `PRIVATE_IMAGE_REGISTRY`. The steps assume a single shell session. If the session changes, re-export the variables before continuing. ## Step 1 - Obtain the data plane credential In the dashboard, go to **Settings -> Data planes**. If this data plane is not registered yet: 1. Click **Register Data plane**. 1. Set **Data plane ID** to a stable ID such as `acme-prod`. 1. Optionally set **Name**, **Description**, and **Labels**. 1. Leave **URL** blank for now unless the final gateway hostname is already known. 1. Click **Register Data plane**. Then generate the credential from the data plane row: 1. Click the row's edit action to open the **Data plane** drawer. 1. In the **Credential** section, click **Generate credential**. 1. Save the downloaded file as `data-plane-credentials.json` on the machine the install runs from. Each data plane uses its own credential. A credential can be rotated or revoked from the same **Credential** section in the data plane edit drawer. ## Step 2 - Set variables For Azure Container Registry: ```bash DP_CREDENTIAL=./data-plane-credentials.json PRIVATE_REGISTRY_HOST=${ACR_NAME}.azurecr.io PRIVATE_IMAGE_REGISTRY=${ACR_NAME}.azurecr.io/tare PULL_SECRET=tare-registry-pull ``` For another private registry: ```bash DP_CREDENTIAL=./data-plane-credentials.json PRIVATE_REGISTRY_HOST=registry.acme.example.com PRIVATE_IMAGE_REGISTRY=registry.acme.example.com/tare PULL_SECRET=tare-registry-pull REGISTRY_USERNAME= REGISTRY_PASSWORD= ``` ## Step 3 - Sync release artifacts Log in locally with credentials that can push to the destination registry. For ACR: ```bash az acr login --name "${ACR_NAME}" ``` For a generic private registry: ```bash printf '%s' "${REGISTRY_PASSWORD}" | \ docker login "${PRIVATE_REGISTRY_HOST}" \ --username "${REGISTRY_USERNAME}" \ --password-stdin ``` Copy the pinned TARE images and the serve-helm OCI chart: ```bash tare install "${DP_CREDENTIAL}" \ --image-sync "${PRIVATE_IMAGE_REGISTRY}" \ --sync-only \ --parallel 2 \ --stall-threshold 5m ``` `tare` uses the data plane credential to pull from Tetrate's source registry. The local registry login is used only to push to the private registry. ## Step 4 - Apply CRDs Apply CRDs outside Helm: ```bash tare install "${DP_CREDENTIAL}" \ --crds-only \ --image-registry "${PRIVATE_IMAGE_REGISTRY}" ``` CRDs are intentionally managed outside the Helm release so `helm uninstall` does not delete cluster-scoped APIs and cascade-delete custom resources. ## Step 5 - Render Helm values Render values for the Helm release: ```bash tare install "${DP_CREDENTIAL}" \ --image-registry "${PRIVATE_IMAGE_REGISTRY}" \ --print-helm-values > values.yaml ``` Treat `values.yaml` as sensitive because it contains the data plane identity secret. Do not commit it as plaintext. Use SOPS, Sealed Secrets, External Secrets, or the organization's approved secret workflow. If the public gateway URL is not known yet, make sure the rendered values do not set it to the management-plane API URL. Keep these fields empty: ```yaml global: serveUrl: "" controller: config: proxyUrl: "" ``` If needed, override them during Helm install: ```bash --set-string global.serveUrl= \ --set-string controller.config.proxyUrl= ``` ## Step 6 - Create namespaces and pull secrets Create the data plane namespaces: ```bash kubectl create namespace tars-system --dry-run=client -o yaml | kubectl apply -f - kubectl create namespace tars-dataplane --dry-run=client -o yaml | kubectl apply -f - ``` For AKS attached to ACR, no image pull secret is usually needed after: ```bash az aks update \ --resource-group "${RESOURCE_GROUP}" \ --name "${AKS_CLUSTER_NAME}" \ --attach-acr "${ACR_NAME}" ``` For a private registry that requires Kubernetes pull credentials, create the same secret in both namespaces: ```bash kubectl create secret docker-registry "${PULL_SECRET}" \ --docker-server="${PRIVATE_REGISTRY_HOST}" \ --docker-username="${REGISTRY_USERNAME}" \ --docker-password="${REGISTRY_PASSWORD}" \ --namespace tars-system \ --dry-run=client -o yaml | kubectl apply -f - kubectl create secret docker-registry "${PULL_SECRET}" \ --docker-server="${PRIVATE_REGISTRY_HOST}" \ --docker-username="${REGISTRY_USERNAME}" \ --docker-password="${REGISTRY_PASSWORD}" \ --namespace tars-dataplane \ --dry-run=client -o yaml | kubectl apply -f - ``` When using a pull secret, render values with: ```bash tare install "${DP_CREDENTIAL}" \ --image-registry "${PRIVATE_IMAGE_REGISTRY}" \ --image-pull-secret-name "${PULL_SECRET}" \ --print-helm-values > values.yaml ``` ## Step 7 - Install or upgrade with Helm Use the chart version for the TARE release: ```bash CHART_VERSION="0.1.0-alpha.1+17f076b" ``` Install or upgrade the release: ```bash helm upgrade --install tars "oci://${PRIVATE_IMAGE_REGISTRY}/serve-helm" \ --version "${CHART_VERSION}" \ -f values.yaml \ -n tars-system \ --skip-crds \ --set ai-gateway-crds.enabled=false \ --set-string global.serveUrl= \ --set-string controller.config.proxyUrl= ``` Use `--skip-crds` and `--set ai-gateway-crds.enabled=false` together. The first skips chart `crds/` entries; the second disables AI Gateway CRDs rendered as normal Helm templates. Helm expects the SemVer build-metadata form with `+`, such as `0.1.0-alpha.1+17f076b`. OCI stores the tag internally with `_`, but `helm --version` should still use `+`. ## Step 8 - Verify Check the Helm release: ```bash helm list -n tars-system helm status tars -n tars-system ``` Wait for workloads: ```bash kubectl rollout status deployment/controller -n tars-system --timeout=300s kubectl rollout status deployment/egress -n tars-dataplane --timeout=300s ``` Confirm images come from the private registry: ```bash kubectl get deploy egress -n tars-dataplane \ -o jsonpath='{range .spec.template.spec.containers[*]}{.name}={.image}{"\n"}{end}' ``` Run doctor: ```bash tare doctor "${DP_CREDENTIAL}" --verbose ``` A fresh data plane with no routes can report warnings for missing RouteDeployments, AIGatewayRoutes, HTTPRoutes, or EnvoyPatchPolicies. The core install is healthy when pods roll out and images pull from the private registry. ## Step 9 - Expose the data plane This Helm workflow installs the data plane but does not create a public Gateway, ingress, DNS record, or TLS certificate. For gateway exposure, follow the published cloud install guide for the target platform, or use the organization's preferred ingress or Gateway stack. The data plane serves traffic through the in-cluster egress service: - Service: egress - Namespace: tars-dataplane - Port: 10080 - Paths: `/v1/*`, `/mcp/*`, and `/.well-known/*` After the gateway and DNS are ready, set the gateway URL in the dashboard: **Settings -> Data planes -> edit row -> URL**. ## Step 10 - Validation and smoke tests Validate the Helm install first: ```bash helm status tars -n tars-system kubectl rollout status deployment/controller -n tars-system --timeout=300s kubectl rollout status deployment/egress -n tars-dataplane --timeout=300s kubectl get deploy egress -n tars-dataplane \ -o jsonpath='{range .spec.template.spec.containers[*]}{.name}={.image}{"\n"}{end}' tare doctor "${DP_CREDENTIAL}" --verbose ``` Then run traffic smoke tests after onboarding has configured provider keys and routes. This Helm workflow does not create a public Gateway or DNS record. Until the gateway is exposed, use a local port-forward to the in-cluster egress service: ```bash kubectl port-forward -n tars-dataplane svc/egress 18080:10080 ``` In another terminal, set the local endpoint and an API key from the router app: ```bash export DP_SCHEME=http export DP_HOST=127.0.0.1:18080 export TARS_API_KEY= ``` At minimum, verify model listing with and without authentication: ```bash curl -s -o /tmp/tare-models.json -w "%{http_code}\n" \ "${DP_SCHEME}://${DP_HOST}/v1/models" \ -H "Authorization: Bearer ${TARS_API_KEY}" curl -s -o /tmp/tare-models-no-auth.json -w "%{http_code}\n" \ "${DP_SCHEME}://${DP_HOST}/v1/models" ``` Expected result: the authenticated request returns 200; the unauthenticated request returns 401. Also smoke test every API shape enabled during onboarding, such as `/v1/chat/completions`, `/v1/responses`, `/v1/messages`, `/v1/embeddings`, `/v1/images/generations`, and MCP profile paths. ## Upgrade and rollback Use the same split workflow for upgrades: update CRDs first, then upgrade the Helm workload release. If the new TARE release includes CRD changes, re-run CRD apply: ```bash tare install "${DP_CREDENTIAL}" \ --crds-only \ --image-registry "${PRIVATE_IMAGE_REGISTRY}" ``` Render values for the new release: ```bash tare install "${DP_CREDENTIAL}" \ --image-registry "${PRIVATE_IMAGE_REGISTRY}" \ --print-helm-values > values.yaml ``` If a pull secret is used, include it when rendering values: ```bash tare install "${DP_CREDENTIAL}" \ --image-registry "${PRIVATE_IMAGE_REGISTRY}" \ --image-pull-secret-name "${PULL_SECRET}" \ --print-helm-values > values.yaml ``` Upgrade the workload release: ```bash CHART_VERSION="" helm upgrade tars "oci://${PRIVATE_IMAGE_REGISTRY}/serve-helm" \ --version "${CHART_VERSION}" \ -f values.yaml \ -n tars-system \ --skip-crds \ --set ai-gateway-crds.enabled=false \ --set-string global.serveUrl= \ --set-string controller.config.proxyUrl= ``` Rollback uses Helm's normal rollback flow: ```bash helm rollback tars -n tars-system ``` CRDs are not affected by rollback because they are not part of the Helm release manifest. Existing custom resources remain in the cluster. ## If artifacts are already mirrored If Agent Router team has already mirrored the TARE images and chart, skip the `--image-sync` step and start at CRD apply: ```bash tare install "${DP_CREDENTIAL}" \ --crds-only \ --image-registry "${PRIVATE_IMAGE_REGISTRY}" ``` Then render values and install with Helm as above. ## Troubleshooting ### Destination registry reports repository not found Create the destination repository before syncing. For example, with Google Artifact Registry: ```bash gcloud artifacts repositories create tare \ --repository-format docker \ --location ``` ACR and many enterprise registries create repositories on first push, but some registries require pre-creation. ### Helm reports invalid ownership metadata This usually means Helm is trying to adopt CRDs already applied by `tare --crds-only`. Example: ```text CustomResourceDefinition "aigatewayroutes.aigateway.envoyproxy.io" exists and cannot be imported into the current release: invalid ownership metadata ``` Install or upgrade with both: ```bash --skip-crds \ --set ai-gateway-crds.enabled=false ``` ### Helm reports improper constraint `helm --version` expects a SemVer constraint, not a bare commit SHA. Use the chart's full SemVer, such as `0.1.0-alpha.1+17f076b`, or a release-build TARE version such as `v0.1.0-beta.3`. Bare commit SHAs are rejected before Helm contacts the registry. The registry stores SemVer build metadata with `_`, but `helm --version` should use the SemVer form with `+`. ### Pods are stuck in ImagePullBackOff Check that: - the private registry contains every image tag for the TARE release - the cluster identity can pull from the private registry - pull secrets exist in both tars-system and tars-dataplane when required - the pull secret is referenced from `values.yaml`; if not, re-render values with `--image-pull-secret-name` - `values.yaml` points `global.imageRegistry` at the intended registry ### Helm cannot pull the chart from the mirror If `helm install` or `helm upgrade` fails locally with pull access denied, log in to the OCI registry from the operator machine: ```bash helm registry login "${PRIVATE_REGISTRY_HOST}" ``` This is separate from Kubernetes image pull access. Helm pulls the chart artifact from the local machine; the cluster pulls workload images using node identity or image pull secrets. Where to go next Gateway installation Install the data plane gateway components that manage inbound access and configure request routing. Self-Hosted Data Plane See the full install sequence: gateway, SSO, then developer handoff.