Skip to main content

Moving from the open source project to Agent Router Enterprise

Same gateway, new control plane

Tetrate Agent Router Enterprise runs the same Envoy-based gateway as the open source project. What changes is how it is configured: the Admin Console and Admin API replace the YAML, and a Tetrate-hosted management plane replaces the controller run today. This guide covers what carries over, what is set up again, and how to move traffic with a way back. For teams still deciding, see why teams move to Enterprise.

The open source Agent Router (formerly Envoy AI Gateway) runs in the customer's own cluster, with routes, providers, and credentials held in AIGatewayRoute, AIServiceBackend, and BackendSecurityPolicy resources applied by hand. Agent Router Enterprise installs its own data plane and Controller, and the Controller takes its configuration from the management plane. The open source install is not upgraded in place: the Enterprise data plane is stood up beside it, traffic is moved, and the old gateway is retired when nothing calls it.

What carries over and what is set up again​

AreaOpen source todayWith Agent Router Enterprise
The proxyEnvoy, through Envoy GatewaySame. The Enterprise data plane is Envoy with the Agent Router gateway modules.
Client APIOpenAI-compatibleSame, plus the Anthropic API. Client code changes a base URL and a key.
Providers and credentialsAIServiceBackend and BackendSecurityPolicy YAMLSet up again in the Admin Console or Catalog API. Provider accounts do not change: existing keys are reused, or Bedrock workload identity is used.
Routing, fallback, splitsAIGatewayRoute YAMLSet up again as routing policies in the console. The Controller writes the gateway configuration.
Quotas and rate limitsPolicy YAML, per gatewayBudgets, rate limits, and spend ceilings by project, key, team, and user. Shared across every data plane.
API keys for app teamsKeys already issuedIssued from the console, per project. App teams get new keys.
Gateway endpointThe existing Service or IngressA new hostname behind an ingress created for it. Tetrate has no inbound path to it.
The aigw CLILocal runs and developmentNot used with Enterprise. The tetrate CLI installs, checks, and upgrades the data plane.
Helm and GitOpsExisting charts and manifestsStill Helm. The serve-helm chart renders manifests that can be committed for Argo CD or Flux.
What you know about Envoy GatewayYears of running itMost of that experience still applies. The GatewayClass, namespaces, and deployment names are new.

Two things do not move: hand-written routing YAML and previously issued API keys.

Where the Enterprise data plane can run​

The Enterprise data plane needs a cluster of its own. It installs a GatewayClass named tars-egress under the controller name gateway.envoyproxy.io/gatewayclass-controller. An Envoy Gateway that watches the whole cluster claims that class too, and the two controllers fight over it. The open source install runs Envoy Gateway cluster-wide, so the two cannot share a cluster. There is no override.

A new cluster is planned before starting, and a small one is enough. The prerequisites page lists the sizing and the pre-install check, which is two commands:

kubectl get gatewayclass
kubectl get deployments -A -l app.kubernetes.io/name=envoy-gateway

Both should come back empty on the target cluster.

What leaves the cluster​

Requests and responses travel only through the data plane. The management plane stores configuration, user records, and telemetry such as latency and token counts. At the default settings it also stores request and response bodies, and the text matched by guardrail rules. Setting the request log mode and the guardrail trigger content mode to Metadata only removes that content from those records (see Control what request data leaves the cluster). The Controller polls api.<tenant>.tetrate.ai on TCP 443 for configuration. That is the only connection that stays open. If the link drops, the data plane keeps routing on the last configuration it received. Details are in Network and security and Prerequisites.

Steps​

  1. Create the cluster and check compatibility. A dedicated cluster with no Envoy Gateway on it. The Kubernetes and Gateway API ranges are listed per release on Release compatibility. The installer enforces them.

  2. Get a tenant and onboard. Start from the Fast Track Eval or contact Tetrate. Complete self-hosted onboarding: organization, SSO, and the data plane credential. Download data-plane-credentials.json.

  3. Recreate providers and models. In the Admin Console, add each provider currently defined as an AIServiceBackend, with the same account and credential. Add the models the apps call. See Provision models and providers.

  4. Install the data plane. Follow the Helm install guide or the guide for the target cloud. The Controller lands in tars-system and the gateway in tars-dataplane.

    tetrate agentrouter dataplane install "${DP_CREDENTIAL}"

  5. Expose the gateway. Create an ingress to the egress service on port 10080, with a chosen hostname, and declare it in global.gateway.hostnames. See Gateway installation.

  6. Create a project and issue keys. One project per team or app. See Issue a project-scoped API key.

  7. Recreate routing. Fallback chains and traffic splits become routing policies. Start with one policy that mirrors one AIGatewayRoute. See the Routing policy reference.

  8. Verify.

    tetrate agentrouter doctor "${DP_CREDENTIAL}" --verbose

    Then one request is sent to the new hostname with a project key and located in the request log in the Admin Console.

  9. Move one team. Change that team's base URL and key, watch Analytics for a day, then move the next team.

  10. Retire the open source gateway when nothing calls it.

What app teams change​

Two values: the base URL and the API key. The request body doesn't change. Teams on the Anthropic SDK can use the Anthropic endpoint directly.

How to roll back​

The open source gateway keeps running until step 10, so rollback is a base URL change.

  1. Point the team's base URL and key back at the open source gateway.
  2. To remove the Enterprise data plane, run helm uninstall tars -n tars-system. Leave the CRDs, which move forward only, as the install guide states.
  3. Revoke the data plane credential in the Admin Console. See Data plane credentials.

Two limits apply. The Enterprise install never reads or changes the open source resources, so nothing needs restoring on that side. Budgets, request logs, and analytics live in the management plane, so rolling back loses that view. Traffic keeps flowing.

Try the console first​

The Fast Track Eval is fully managed, so nothing is installed. A first routed request is possible about 15 minutes after onboarding, and a small team with a cost readout by the end of the day. It shows the Admin Console, budgets, and Analytics before the self-hosted cluster is built. The open source gateway keeps running throughout.

Get help​