# Planes and core components
> The split-plane model behind Tetrate Agent Router: a Tetrate-hosted management plane, a data plane that carries all AI traffic (Tetrate-operated or customer-managed), the outbound link between them, and the data-plane components that route AI traffic.
Tetrate Agent Router is built on a split-plane model.
- A management plane hosted by Tetrate holds configuration and exposes the web applications.
- A data plane handles all AI traffic. Where it runs depends on the deployment model: Tetrate operates it in Agent Router Service and Agent Router Enterprise Fully Managed; the customer deploys it in their Kubernetes cluster in Agent Router Enterprise Self-Hosted Data Plane.
The two planes stay in sync over a connection the data plane opens. Live traffic stays on the data plane and never reaches the management plane.
This page explains how Tetrate Agent Router is put together and defines each term as it appears. No prior familiarity with the platform is assumed. The starting point is a single idea that shapes everything else: the parts of the platform that decide what should happen are kept separate from the parts that carry live traffic. Where those traffic-carrying parts run is the choice described in [Deployment models](/product-architecture/deployment-models).
For a first-principles walk-through of what an AI gateway is and how a request travels end to end, start with the Architecture overview. This page goes deeper on the two planes and the components inside the data plane.
## System architecture
A *plane* is a group of software components that share a single job. Tetrate Agent Router is organised into two planes, each with a distinct responsibility.
- The **management plane** is the part hosted and operated by Tetrate in every deployment model. It holds configuration (routing rules, policies, model and provider definitions, user records) and exposes the two web applications used to manage the platform. It is the control room: decisions are recorded here, but no live AI traffic passes through it.
- The **data plane** is the part that handles all AI traffic: every prompt sent to a model, and every response returned, passes through it and nowhere else. In [Agent Router Service](/agent-router-service/quickstarts) and [Agent Router Enterprise Fully Managed](/agent-router-enterprise/fully-managed), Tetrate operates it. In [Agent Router Enterprise Self-Hosted Data Plane](/agent-router-enterprise/self-hosted-data-plane), it is deployed inside the customer's own Kubernetes cluster. *Kubernetes* is the industry-standard system for running containerised software across a group of machines; a *cluster* is one such group.
The two planes communicate over a single outbound Hypertext Transfer Protocol Secure (HTTPS) connection initiated by the data plane. *Outbound* means the connection is opened from the data plane toward Tetrate, never the other way around. In Agent Router Enterprise Self-Hosted Data Plane that means no inbound connections from the internet to the customer environment are required.
The diagram below shows the layout for **Agent Router Enterprise Self-Hosted Data Plane** (data plane in the customer domain). Fully hosted deployments keep the same components and flows, with the data plane in Tetrate's environment instead.
The diagram traces two independent journeys. Configuration travels down the left: the management plane hands updates to the Controller, which applies them to the Agent Router. Live traffic travels along the bottom: client applications send requests to the Agent Router, which forwards them to the upstream AI providers and returns the responses. The two journeys never mix, which is the point of the split.
### Why the architecture is split this way
The separation exists for one main reason: **live prompt and response content stays on the data plane and never reaches the management plane**. The management plane only ever holds configuration and receives telemetry (operational data such as latency and token counts); it never sees the content of a request.
In Agent Router Enterprise Self-Hosted Data Plane, that also means **prompt and response data never leaves the customer's environment**, because the data plane runs in infrastructure the customer controls. The outbound-only link reinforces this: a firewall configured to allow a connection out is far easier for a security team to approve than one that must accept connections coming in, and refusing all inbound connections removes an entire class of attack. The result is data sovereignty (the customer keeps full control of its data) without the customer having to build and operate the control-plane software itself. In Agent Router Service and Agent Router Enterprise Fully Managed the same plane separation holds, but the data plane itself is operated by Tetrate.
## Operating model
The split-plane model divides responsibilities cleanly between a centralized management environment controlled by Tetrate and a data plane that carries all AI traffic. In practice this means each side owns a different set of concerns, and each side can be changed, scaled, or recovered without disturbing the other.
Tetrate managed
Management plane
Hosted and operated by Tetrate in every deployment model. Contains the Developer Console, the Admin Dashboard, and the central backend that stores routing rules, policies, and configuration. Customers interact with these applications to configure the platform but do not manage the underlying infrastructure, so there are no control-plane servers to patch, back up, or keep highly available on the customer side.
Traffic path
Data plane
All prompt and response traffic flows exclusively through this environment. Tetrate operates it in Agent Router Service and Agent Router Enterprise Fully Managed; the customer deploys it in their own Kubernetes cluster in Agent Router Enterprise Self-Hosted Data Plane. The data plane receives configuration from the management plane but operates independently: it holds the most recent configuration locally and continues routing requests normally even if connectivity to the management plane is temporarily interrupted. A brief loss of the link pauses configuration updates, not live traffic.
The two applications in the management plane are two views of the same system, each scoped to a different audience.
| Application | Audience | What it is used for |
| :---- | :---- | :---- |
| **Developer Console** | Developers | Issue API keys, author routing configurations, assemble Model Context Protocol (MCP) profiles, and monitor their own traffic. |
| **Admin Dashboard** | Platform operators (platform / IT) | Provision models and providers, manage users, set budgets and rate limits, review audit logs, and configure single sign-on. |
The three journeys that cross these planes (configuration polling, the live request path, and telemetry export) are traced in detail on the Data flows page.
## Operating multiple data planes
Tetrate Agent Router Enterprise supports running more than one data plane from a single management plane. Each data plane is an independent deployment (typically in its own Kubernetes cluster), and the data planes can sit in any combination of clouds, regions, or on-premises environments while the one management plane keeps their configuration in sync. This fleet model is most relevant to Agent Router Enterprise Self-Hosted Data Plane, where the customer chooses where each data plane runs.
The diagram below illustrates a multi–data-plane layout for Agent Router Enterprise Self-Hosted Data Plane.
More than one data plane is useful whenever a single deployment cannot satisfy every requirement at once. A common pattern is one data plane per region, placing traffic handling close to the applications it serves to reduce latency and to keep requests within a required geography for data residency. Another is one data plane per cloud, so that models offered natively by a provider are reached over that provider's own network: an AWS data plane serving Amazon Bedrock models, for example, alongside an Azure data plane serving Azure OpenAI models. Data planes are also split per business unit or per environment (production, staging) when separate teams need to operate without affecting one another.
Within this model, each data plane provisions and manages one or more *gateways*, where a gateway is a single inference endpoint (one customer-facing URL) mapped to exactly one *project*. A project is the logical boundary that owns a set of models, API keys, and MCP profiles, together with the members allowed to use them, and is defined in full under [Projects](/product-architecture/key-concepts#projects). Because a gateway serves only its project, isolation is enforced at request time: a key issued for one project is refused on another project's gateway, and a model a project has not been granted cannot be called through it. Several projects can share one data plane, each with its own gateway, or be spread across data planes as the region and cloud patterns above require.
Holding configuration centrally in the management plane while each data plane runs independently is what makes a fleet of data planes practical to operate. Data planes can be added as demand grows, and a gateway's runtime can be upgraded in place or moved to a different data plane without changing its customer-facing URL or dropping in-flight requests. The operator-facing procedures are covered in [Run multiple platform instances](/agent-router-enterprise/guides/operate-and-govern/run-multiple-platform-instances) and [Load balance across regional deployments](/agent-router-enterprise/guides/operate-and-govern/load-balance-across-regional-deployments).
## Core components of the data plane
The data plane is not a single program. It is a small set of components that run together wherever the data plane is hosted.
At the center is the AI Gateway, which intercepts and manages all traffic between client applications and upstream AI providers. A *gateway* is a single entry point that requests pass through on their way somewhere else, and this one is built as a *proxy*: a server that receives a request, does something with it (here: choosing a provider, attaching credentials, applying policy), and forwards it on.
Controller
Acts as the bridge between the data plane and the management plane. It polls the backend for configuration updates and translates them into a gateway configuration, keeping routing rules, policies, and provider credentials in sync without requiring the management plane to open inbound connections. Polling means the Controller repeatedly asks for updates on a schedule, so it is always the side that reaches out.
Modules
The AI gateway is extended with high-performance native modules for cost controls, guardrails, and custom extensions. A dynamic module is add-on logic loaded into the proxy at runtime, so it runs inline in the proxy filter chain (the ordered sequence of steps each request passes through) and adds processing without an extra network hop.
Agent Router
The proxy that routes every request to a provider and enforces policy inline.
The Agent Router itself (the proxy every request passes through) is the gateway the other components extend and support. It is built on an Cloud Native Computing Foundation (CNCF) backed open source foundation, co-created and maintained by Tetrate. This open source component is a widely used open-source proxy, which is why the platform can route to many providers reliably and with low overhead. The Controller keeps the gateway configured, the dynamic modules add cost controls and guardrails inline, and Redis lets several copies of the gateway share one view of rate-limit usage.
Why these pieces run inline rather than as separate services: every extra service a request must call adds a network hop, and every hop adds latency and a new thing that can fail. Running cost controls and guardrails as dynamic modules inside the proxy keeps that logic on the request path itself, so policy is enforced without leaving the gateway.
The components divide across the two planes as follows. The management plane components are operated by Tetrate; the data plane components run wherever the data plane is hosted.
| Component | Plane | Role |
| :---- | :---- | :---- |
| Developer Console, Admin Dashboard, central backend | Management | Configuration storage and the web applications operators and developers use |
| Agent Router (AI Gateway) | Data | The proxy that routes every request to a provider and enforces policy inline |
| Controller | Data | Polls the backend and translates configuration into gateway configuration |
| Modules | Data | Inline gateway extensions for cost controls, guardrails, logs, telemetry and more |
## Key terms
The vocabulary introduced above, collected for reference:
| Term | Meaning |
| :---- | :---- |
| **Plane** | A group of software components that share one responsibility. The platform has two: management and data. |
| **Management plane** | The Tetrate-hosted half that stores configuration and serves the web applications. Holds no live traffic. |
| **Data plane** | The half that all AI traffic flows through. Tetrate-operated in Agent Router Service and Agent Router Enterprise Fully Managed; customer-hosted in Kubernetes for Agent Router Enterprise Self-Hosted Data Plane. |
| **Kubernetes** | The industry-standard system for running containerised software across a group of machines (a cluster). |
| **Controller** | The data-plane component that polls the management plane for configuration and applies it to the gateway. |
| **Gateway** | The single entry point every request passes through before reaching a provider. |
| **Proxy** | A server that receives a request, acts on it, and forwards it on. The gateway is built as one, on Envoy. |
| **Dynamic module** | Add-on logic loaded into the proxy at runtime, running inline in the filter chain with no extra network hop. |
| **Redis** | A fast in-memory datastore the gateway uses as a shared counter for rate limiting across instances. |
| **Rate limiting** | Capping how much an API key may consume, enforced consistently across all gateway copies. |
| **Outbound-only connection** | A link opened from the data plane toward Tetrate, never the reverse. In Agent Router Enterprise Self-Hosted Data Plane, that means no inbound firewall openings into the customer environment are needed. |
## Where to go next
- **Data flows**: the three traffic paths that cross these planes. See [Data flows](/product-architecture/data-flows).
- **Guardrails**: how the Guardrails dynamic module inspects traffic inline. See [Guardrails](/product-architecture/guardrails).
- **Deployment models**: where each plane runs in Agent Router Service, Agent Router Enterprise Fully Managed, and Agent Router Enterprise Self-Hosted Data Plane. See [Deployment models](/product-architecture/deployment-models).
Where to go next
Data flows
The configuration, request, and observability paths that cross the two planes.
Guardrails
How the Guardrails dynamic module inspects traffic inline at the gateway.
Deployment models
Agent Router Service, Agent Router Enterprise Fully Managed, and Agent Router Enterprise Self-Hosted Data Plane compared.