# Installation Guide for Azure using Helm

> Install the Agent Router data plane on Azure Kubernetes Service (AKS) using Helm and a private container registry.

This guide installs the Agent Router data plane on Azure Kubernetes Service (AKS) using Helm and a private container registry.

**Important**  
This guide assumes platform teams own the Helm release lifecycle and operate a private container registry (Azure Container Registry, Harbor, JFrog Artifactory, Nexus, or Google Artifact Registry).
If not, refer to the [Installation guide for Azure](/agent-router-enterprise/self-hosted-data-plane/data-plane-installation/installation-guide-az).

:::warning About Helm-based installation
The recommended way to installing a data plane is by using the `TARE` command line utility.  

Using Helm is only for advanced or custom cases, and should be carefully considered.
:::

## Architecture

Agent Router Enterprise uses a split-plane model. A **management plane** hosted by **Tetrate** holds configuration, license data, and the web dashboard. A **data plane** runs in a customer-managed Kubernetes cluster and handles all AI traffic. The two planes communicate over a single outbound HTTPS connection initiated by the data plane. No inbound internet connections are required.

The data plane requires a stable public hostname so applications can reach it; TLS certificates are issued for hostnames, not IP addresses. Gateway exposure, DNS, and TLS are out of scope for this guide.

```mermaid
architecture-beta
  group azure(cloud)[Azure]
  group tetrate(cloud)[Tetrate hosted]

  service mp(server)[Management Plane] in tetrate
  service dp(server)[Data Plane] in azure

  mp:R -- L:dp
```

The workflow is:

- The `tare` CLI syncs Agent Router release artifacts into the private registry.
- `tare` applies CRDs (Custom Resource Definitions) outside Helm.
- `tare` renders Helm values.
- `helm` installs or upgrades the data plane workload from the private registry.

This guide covers data plane installation only. It does not create a public Gateway, ingress, DNS record, or TLS certificate. For gateway exposure, follow the published cloud install guide for the target platform, or use the organization's preferred ingress or Gateway stack.

## Outcomes

By the end of this guide:

- Agent Router release artifacts are mirrored into a private container registry.
- Agent Router CRDs (Custom Resource Definitions) are applied to the target Kubernetes cluster.
- The data plane is deployed via Helm into the `tars-system` and `tars-dataplane` namespaces.
- The in-cluster `egress` service is verified and reachable via port-forward for smoke tests.

Plan for 30 to 60 minutes, plus separate time for Agent Router team to provision the gateway, DNS, and TLS.

## In this guide

- **Prepare for the installation**: obtain the data plane credential and install the CLI
  - [Prerequisites](#prerequisites)
  - [Step 1: Obtain the data plane credential](#step-1-obtain-the-data-plane-credential)
  - [Step 2: Install the `tare` CLI](#step-2-install-the-tare-cli)
  - [Step 3: Set environment variables](#step-3-set-variables)
- **Registry setup** - mirror Agent Router artifacts into the private registry
  - [Step 4: Sync release artifacts](#step-4-sync-release-artifacts)
- **Data Plane installation** - apply CRDs and deploy with Helm
  - [Step 5: Apply CRDs](#step-5-apply-crds)
  - [Step 6: Render Helm values](#step-6-render-helm-values)
  - [Step 7: Create namespaces and pull secrets](#step-7-create-namespaces-and-pull-secrets)
  - [Step 8: Install or upgrade with Helm](#step-8-install-or-upgrade-with-helm)
- **Verify** - confirm the installation works end-to-end
  - [Step 9: Verify](#step-9-verify)
  - [Step 10: Expose the data plane](#step-10-expose-the-data-plane)
  - [Step 11: Run validation and smoke tests](#step-11-run-validation-and-smoke-tests)
- **Operations** - manage and troubleshoot the installation
  - [Upgrade and rollback](#upgrade-and-rollback)
  - [If artifacts are already mirrored](#if-artifacts-are-already-mirrored)
  - [Troubleshooting](#troubleshooting)

## Prerequisites

### Dashboard and Router app access

Agent Router exposes two web surfaces. Both URLs are provided during onboarding:

- **Dashboard** (admin): `https://dashboard.<your-tenant>.tetrate.ai`.
- **Router app** (end-user): `https://router.<your-tenant>.tetrate.ai`.

### Required tools

Install the following on the workstation used for this guide:

| Tool | Install |
| :--- | :--- |
| `az` CLI | `https://learn.microsoft.com/cli/azure/install-azure-cli` |
| `kubectl` | `https://kubernetes.io/docs/tasks/tools` |
| `helm` (3+) | `https://helm.sh/docs/intro/install` |
| `docker` | `https://docs.docker.com/get-docker` |
| `curl` | Preinstalled on macOS and most Linux distributions |
| `tare` CLI | Covered in [Step 2](#step-2-install-the-tare-cli) |

A `data-plane-credentials.json` file is also required. See [Step 1](#step-1-obtain-the-data-plane-credential).

### Infrastructure

A dedicated workload cluster must be provisioned before starting the installation. The cluster requires at least three (3) nodes. See [Cluster sizing](#cluster-sizing) for more detail.

:::warning
Tetrate support does not cover client-side infrastructure provisioning or Kubernetes issues. The instructions for creating clusters and related infrastructure components are provided as a courtesy and should be carefully evaluated before executing them.
:::

Furtermore, make sure you have sufficient systems/infrastructure permissions to:
- Access the target Kubernetes cluster.
- Push images to the private registry mirror.
- Pull images from the target cluster to the same private registry.

### Cluster sizing

The default chart installs multiple always-on components, including the **Agent Router egress proxy** (Envoy-based; referred to as *egress envoy* in manifests and logs) with a minimum of 2 replicas, AI gateway controller and ext_proc, controller and worker, Redis, and rate-limit services. A single-node footprint is not sufficient.

The **egress envoy** is the dominant resource consumer. Its CPU and memory usage scale with the configuration size held in memory: the count of `AIGatewayRoute` and `AIServiceBackend` resources, header-mutation rules, and per-route features. The AI gateway team's [control-plane scaling benchmark](https://aigateway.envoyproxy.io/blog/benchmarking-control-plane-scaling) shows roughly linear CPU and memory growth as routes are added. Plan capacity for route counts that grow as providers, models, and projects are added. General-purpose VM sizes from the `Standard_D*s_v5` family provide a balanced default.

| Size | Use case | Recommended node pool | Approximate allocatable target |
| :--- | :--- | :--- | :--- |
| Small | Dev / test / low traffic | 3 × `Standard_B2s` | ≥ 6 vCPU, ≥ 20 GiB RAM |
| Medium | Staging / light production | 3 × `Standard_D4s_v5` | ≥ 12 vCPU, ≥ 40 GiB RAM |
| High | Production with burst headroom | 3 × `Standard_D8s_v5` (or split into system + data plane pools) | ≥ 24 vCPU, ≥ 80 GiB RAM |

Maintain a minimum of three nodes to tolerate node upgrades and evictions. Demo installs may start at Small; production installations should start at Medium.

:::note
AKS Automatic is not supported.
:::

## Conventions

All commands assume the environment variables defined in [Step 3](#step-3-set-variables) are exported in the current shell. Re-export them when opening a new terminal.

Replace every `<placeholder>` value with a site-specific one.

## Step 1: obtain the data plane credential

In the dashboard, go to **Settings > Data planes**.

If the data plane is not registered yet:

1. Click **Register Data plane**.
1. Set **Data plane ID** to a stable identifier such as `acme-prod`.
1. Optionally set **Name**, **Description**, and **Labels**.
1. Leave **URL** blank for now unless the final gateway hostname is already known.
1. Click **Register Data plane**.

Then generate the credential from the data plane row:

1. Click the row's edit action to open the **Data plane** drawer.
1. In the **Credential** section, click **Generate credential**.
1. Save the downloaded file as `data-plane-credentials.json` on the machine where the install will run.

Each data plane uses its own credential. Credentials can be rotated or revoked from the same **Credential** section in the data plane edit drawer.

## Step 2: install the `tare` CLI

Run the installer script:

```shell
curl -sSL https://tare.tetrate.ai/tools/install.sh | bash
```

Output:

```none
==> tare installer
==> channel: stable

==> Detected platform: darwin-arm64

==> Installing tare for darwin-arm64...
==> Downloading from: https://tare.tetrate.ai/tools/tags/v0.1.0-beta.4/tare-darwin-arm64.tar.gz
ok Installed tare to /Users/johndoe/.tare/bin/tare

==> tare version: tare version v0.1.0-beta.4

ok Installation directory is already in your PATH

==> Get started:
    tare install identity.json --serve-url https://proxy.acme.com
    tare install --help
```

The installer prints the install path (typically `~/.tare/bin/tare`). Add it to `PATH` and verify the version:

```shell
export PATH="$PATH:$HOME/.tare/bin"
echo 'export PATH="$PATH:$HOME/.tare/bin"' >> ~/.zshrc   # or ~/.bashrc
```

```shell
$ tare --version
tare version v0.1.0-beta.4
```

## Step 3: set variables

For Azure Container Registry:

```bash
DP_CREDENTIAL=./data-plane-credentials.json   # Obtained in Step 1 
PRIVATE_REGISTRY_HOST=${ACR_NAME}.azurecr.io
PRIVATE_IMAGE_REGISTRY=${ACR_NAME}.azurecr.io/tare
PULL_SECRET=tare-registry-pull # this value will be used later on, leave it as-is
```

For another private registry:

```bash
DP_CREDENTIAL=./data-plane-credentials.json   # Obtained in Step 1 
PRIVATE_REGISTRY_HOST=registry.acme.example.com   # Replace with your own registry host
PRIVATE_IMAGE_REGISTRY=registry.acme.example.com/tare   # Replace with your own registry full URL
PULL_SECRET=tare-registry-pull   # this value will be used later on, leave it as-is
REGISTRY_USERNAME=<registry-user>
REGISTRY_PASSWORD=<registry-password-or-token>
```

## Step 4: sync release artifacts

Log in locally with credentials that can push to the destination registry.

For ACR:

```bash
az acr login --name "${ACR_NAME}"
```

For a generic private registry:

```bash
printf '%s' "${REGISTRY_PASSWORD}" | \
  docker login "${PRIVATE_REGISTRY_HOST}" \
    --username "${REGISTRY_USERNAME}" \
    --password-stdin
```

Copy the pinned Agent Router images and the `serve-helm` OCI chart:

```bash
tare install "${DP_CREDENTIAL}" \
  --image-sync "${PRIVATE_IMAGE_REGISTRY}" \
  --sync-only \
  --parallel 2 \
  --stall-threshold 5m
```

`tare` uses the data plane credential to pull from Tetrate's source registry. The local registry login is used only to push to the private registry.

## Step 5: apply CRDs

Apply CRDs outside Helm:

```bash
tare install "${DP_CREDENTIAL}" \
  --crds-only \
  --image-registry "${PRIVATE_IMAGE_REGISTRY}"
```

CRDs are managed outside the Helm release so that `helm uninstall` does not delete cluster-scoped APIs and cascade-delete custom resources.

## Step 6: render Helm values

Render values for the Helm release:

```bash
tare install "${DP_CREDENTIAL}" \
  --image-registry "${PRIVATE_IMAGE_REGISTRY}" \
  --print-helm-values > values.yaml
```

Treat `values.yaml` as sensitive: it contains the data plane identity secret. Do not commit it in plaintext. Use SOPS, Sealed Secrets, External Secrets, or the organization's approved secret workflow.

If the public gateway URL is not known yet, confirm the rendered values do not set it to the management-plane API URL. The following fields must remain empty:

```yaml
global:
  serveUrl: ""
controller:
  config:
    proxyUrl: ""
```

Override them during Helm install if needed:

```bash
--set-string global.serveUrl= \
--set-string controller.config.proxyUrl=
```

## Step 7: create namespaces and pull secrets

Create the data plane namespaces:

```bash
kubectl create namespace tars-system --dry-run=client -o yaml | kubectl apply -f -
kubectl create namespace tars-dataplane --dry-run=client -o yaml | kubectl apply -f -
```

For AKS attached to ACR, no image pull secret is usually required after:

```bash
az aks update \
  --resource-group "${RESOURCE_GROUP}" \
  --name "${AKS_CLUSTER_NAME}" \
  --attach-acr "${ACR_NAME}"
```

For a private registry that requires Kubernetes pull credentials, create the same secret in both namespaces:

```bash
kubectl create secret docker-registry "${PULL_SECRET}" \
  --docker-server="${PRIVATE_REGISTRY_HOST}" \
  --docker-username="${REGISTRY_USERNAME}" \
  --docker-password="${REGISTRY_PASSWORD}" \
  --namespace tars-system \
  --dry-run=client -o yaml | kubectl apply -f -

kubectl create secret docker-registry "${PULL_SECRET}" \
  --docker-server="${PRIVATE_REGISTRY_HOST}" \
  --docker-username="${REGISTRY_USERNAME}" \
  --docker-password="${REGISTRY_PASSWORD}" \
  --namespace tars-dataplane \
  --dry-run=client -o yaml | kubectl apply -f -
```

When using a pull secret, render values with:

```bash
tare install "${DP_CREDENTIAL}" \
  --image-registry "${PRIVATE_IMAGE_REGISTRY}" \
  --image-pull-secret-name "${PULL_SECRET}" \
  --print-helm-values > values.yaml
```

## Step 8: install or upgrade with Helm

Set the chart version for the target Agent Router release:

```bash
CHART_VERSION="0.1.0-alpha.1+17f076b"
```

Install or upgrade the release:

```bash
helm upgrade --install tars "oci://${PRIVATE_IMAGE_REGISTRY}/serve-helm" \
  --version "${CHART_VERSION}" \
  -f values.yaml \
  -n tars-system \
  --skip-crds \
  --set ai-gateway-crds.enabled=false \
  --set-string global.serveUrl= \
  --set-string controller.config.proxyUrl=
```

Use `--skip-crds` and `--set ai-gateway-crds.enabled=false` together. The first skips chart `crds/` entries; the second disables AI Gateway CRDs rendered as normal Helm templates.

Helm expects the SemVer build-metadata form with `+`, such as `0.1.0-alpha.1+17f076b`. OCI stores the tag internally with `_`, but `helm --version` must use the `+` form.

## Step 9: verify

Check the Helm release:

```bash
helm list -n tars-system
helm status tars -n tars-system
```

Wait for workloads:

```bash
kubectl rollout status deployment/controller -n tars-system --timeout=300s
kubectl rollout status deployment/egress -n tars-dataplane --timeout=300s
```

Confirm images come from the private registry:

```bash
kubectl get deploy egress -n tars-dataplane \
  -o jsonpath='{range .spec.template.spec.containers[*]}{.name}={.image}{"\n"}{end}'
```

Run doctor:

```bash
tare doctor "${DP_CREDENTIAL}" --verbose
```

A fresh data plane with no routes can report warnings for missing `RouteDeployments`, `AIGatewayRoutes`, `HTTPRoutes`, or `EnvoyPatchPolicies`. The core install is healthy when pods roll out and images pull from the private registry.

## Step 10: expose the data plane

This Helm workflow installs the data plane but does not create a public Gateway, ingress, DNS record, or TLS certificate.

For gateway exposure, follow the published cloud install guide for the target platform, or use the organization's preferred ingress or Gateway stack.

The data plane serves traffic through the in-cluster `egress` service:

- **Service:** `egress`
- **Namespace:** `tars-dataplane`
- **Port:** `10080`
- **Paths:** `/v1/*`, `/mcp/*`, and `/.well-known/*`

Once the gateway and DNS are ready, set the gateway URL in the dashboard: **Settings > Data planes > edit row > URL**.

## Step 11: run validation and smoke tests

Validate the Helm install first:

```bash
helm status tars -n tars-system
kubectl rollout status deployment/controller -n tars-system --timeout=300s
kubectl rollout status deployment/egress -n tars-dataplane --timeout=300s
kubectl get deploy egress -n tars-dataplane \
  -o jsonpath='{range .spec.template.spec.containers[*]}{.name}={.image}{"\n"}{end}'
tare doctor "${DP_CREDENTIAL}" --verbose
```

Run traffic smoke tests after onboarding has configured provider keys and routes. This Helm workflow does not create a public Gateway or DNS record. Until the gateway is exposed, use a local port-forward to the in-cluster `egress` service:

```bash
kubectl port-forward -n tars-dataplane svc/egress 18080:10080
```

In another terminal, set the local endpoint and an API key from the router app:

```bash
export DP_SCHEME=http
export DP_HOST=127.0.0.1:18080
export TARS_API_KEY=<api-key-from-router-app>
```

At minimum, verify model listing with and without authentication:

```bash
curl -s -o /tmp/tare-models.json -w "%{http_code}\n" \
  "${DP_SCHEME}://${DP_HOST}/v1/models" \
  -H "Authorization: Bearer ${TARS_API_KEY}"

curl -s -o /tmp/tare-models-no-auth.json -w "%{http_code}\n" \
  "${DP_SCHEME}://${DP_HOST}/v1/models"
```

Expected result: the authenticated request returns `200`; the unauthenticated request returns `401`.

Also smoke test every API shape enabled during onboarding, such as `/v1/chat/completions`, `/v1/responses`, `/v1/messages`, `/v1/embeddings`, `/v1/images/generations`, and MCP profile paths.

## Upgrade and rollback

Use the same split workflow for upgrades: update CRDs first, then upgrade the Helm workload release.

If the new Agent Router release includes CRD changes, re-run CRD apply:

```bash
tare install "${DP_CREDENTIAL}" \
  --crds-only \
  --image-registry "${PRIVATE_IMAGE_REGISTRY}"
```

Render values for the new release:

```bash
tare install "${DP_CREDENTIAL}" \
  --image-registry "${PRIVATE_IMAGE_REGISTRY}" \
  --print-helm-values > values.yaml
```

When using a pull secret, include it when rendering values:

```bash
tare install "${DP_CREDENTIAL}" \
  --image-registry "${PRIVATE_IMAGE_REGISTRY}" \
  --image-pull-secret-name "${PULL_SECRET}" \
  --print-helm-values > values.yaml
```

Upgrade the workload release:

```bash
CHART_VERSION="<new-chart-version>"

helm upgrade tars "oci://${PRIVATE_IMAGE_REGISTRY}/serve-helm" \
  --version "${CHART_VERSION}" \
  -f values.yaml \
  -n tars-system \
  --skip-crds \
  --set ai-gateway-crds.enabled=false \
  --set-string global.serveUrl= \
  --set-string controller.config.proxyUrl=
```

Rollback uses Helm's normal rollback flow:

```bash
helm rollback tars -n tars-system
```

CRDs are not affected by rollback because they are not part of the Helm release manifest. Existing custom resources remain in the cluster.

## If artifacts are already mirrored

If Agent Router team has already mirrored the Agent Router images and chart, skip the `--image-sync` step and start at CRD apply:

```bash
tare install "${DP_CREDENTIAL}" \
  --crds-only \
  --image-registry "${PRIVATE_IMAGE_REGISTRY}"
```

Then render values and install with Helm as described above.

## Troubleshooting

### Destination registry reports repository not found

Create the destination repository before syncing. For example, with Google Artifact Registry:

```bash
gcloud artifacts repositories create tare \
  --repository-format docker \
  --location <region>
```

ACR and many enterprise registries create repositories on first push, but some registries require pre-creation.

### Helm reports invalid ownership metadata

This usually means Helm is trying to adopt CRDs that were already applied by `tare --crds-only`.

Example:

```text
CustomResourceDefinition "aigatewayroutes.aigateway.envoyproxy.io" exists and cannot be imported into the current release: invalid ownership metadata
```

Install or upgrade with both flags:

```bash
--skip-crds \
--set ai-gateway-crds.enabled=false
```

### Helm reports improper constraint

`helm --version` expects a SemVer constraint, not a bare commit SHA. Use the chart's full SemVer, such as `0.1.0-alpha.1+17f076b`, or a release-build Agent Router version such as `v0.1.0-beta.3`.

Bare commit SHAs are rejected before Helm contacts the registry. The registry stores SemVer build metadata with `_`, but `helm --version` must use the SemVer form with `+`.

### Pods are stuck in imagepullbackoff

Check that:

- the private registry contains every image tag for the Agent Router release
- the cluster identity can pull from the private registry
- pull secrets exist in both `tars-system` and `tars-dataplane` when required
- the pull secret is referenced from `values.yaml`; if not, re-render values with `--image-pull-secret-name`
- `values.yaml` points `global.imageRegistry` at the intended registry

### Helm cannot pull the chart from the mirror

If `helm install` or `helm upgrade` fails locally with `pull access denied`, log in to the OCI registry from the operator machine:

```bash
helm registry login "${PRIVATE_REGISTRY_HOST}"
```

This is separate from Kubernetes image pull access. Helm pulls the chart artifact from the local machine; the cluster pulls workload images using node identity or image pull secrets.

Where to go next

  <Link to="/agent-router-enterprise/self-hosted-data-plane/gateway-installation/gateway-installation-guide" className="tare-nav-card">
    Gateway installation
    Install the data plane gateway components that manage inbound access and configure request routing.
  </Link>
  <Link to="/agent-router-enterprise/self-hosted-data-plane" className="tare-nav-card">
    Self-Hosted Data Plane
    See the full install sequence: gateway, SSO, then developer handoff.
  </Link>