Glossary
A consolidated definition list for the Agent Router-specific vocabulary used across the documentation. For a narrative introduction to the same concepts, see Key Concepts. Product and marketing materials on tetrate.io use some alternate names for the same concepts: for example, control plane for management plane and AI Gateway for the data-plane proxy. This glossary uses the names that appear in these docs; cross-references note the marketing equivalents where they differ.
Active fallback chain
The ordered, priority-ranked list of backends that the gateway walks when a request fails with a retryable error. For Console-configured keys the recoverable set is provider responses with HTTP status 401, 413, 422, 429, 500, 503, 504, or 529, plus connection failures, gateway errors, other 5xx responses, and deadline exceeded. The gateway tries the first backend; on failure it moves to the next until one succeeds or the chain is exhausted, the standard provider fallback pattern used by AI gateways. In Agent Router, the chain is the priority-ordered portion of an API key's Fallback policy; it is active only when the Enable toggle on the key's Manual Availability Fallback section is on (a policy that is switched off is saved but not enforced). See Fallback policy, Routing chain, and Improve Resilience with Fallbacks.
Admin Console
The operator-facing web application, used by platform operators to manage data planes, projects, the catalog, users, policy, budgets, SSO, and audit logs. Named in parallel with the Developer Console, and not shortened to "the Console" in any passage that also mentions the developer application.
Advanced routing rules
Routing decisions that depend on attributes of the request itself (a custom header, the requested model name, or request metadata) rather than only on static priority or weight. These rules live on a single API key and govern how a request reaches a backend; the business-level matching of a class of request to the model that should serve it is Smart routing. See Apply Advanced Routing Rules.
Agent Router Enterprise
The full Agent Router deployment: data plane, management plane, Console, and Admin Console, considered as a single system. Often referred to as "Agent Router" after first mention.
Agent Router Enterprise Fully Managed
The deployment model in which Tetrate operates both planes and the customer installs nothing. The gateway is reachable as soon as onboarding completes, and prompts and responses are processed on Tetrate-hosted infrastructure, so data residency is not customer-selected at the infrastructure level. Deployment model is separate from product tier: Fully Managed and Agent Router Enterprise Self-Hosted Data Plane are both Enterprise and differ only in where the data plane runs. Also distinct from Agent Router Service, which is likewise fully hosted but differs by tier rather than by where software runs. See Deployment models.
Agent Router Enterprise Self-Hosted Data Plane
The hybrid deployment model: Tetrate hosts the management plane, and only the data plane is installed in a Kubernetes cluster the customer operates. "Self-hosted" refers to the data plane alone. The Console, the Admin Console, and the configuration backend stay with Tetrate in every Enterprise deployment, and there is no management-plane software to patch or scale.
Because the data plane sits inside the customer environment, prompts and responses never transit Tetrate systems. The data plane opens a single outbound connection to pull configuration, so no inbound firewall openings are required. This is the only model in which private connectivity, guardrail inspection without content leaving the environment, and Bring your own key (BYOK) are available. See Deployment models.
Agent Router Service
The self-serve product tier: sign up with a GitHub or Google account and route through a Tetrate-managed service without a dedicated Enterprise instance. Includes multi-model routing, MCP access, and per-key usage logs. Agent Router Enterprise adds cross-team cost attribution, admin access controls, runtime AI Guardrails, enterprise SSO, and distributed data-plane deployment. This documentation set focuses on Agent Router Enterprise; tier comparison is on the welcome page.
AI Gateway
The model-routing component of the data plane: the proxy that receives LLM requests from applications, applies the policies attached to the key, attaches credentials, and forwards traffic to upstream providers. On tetrate.io this is one of three product pillars (alongside MCP Gateway and AI Guardrails); older materials may use LLM Gateway. Distinct from a Gateway, the project-scoped endpoint resource provisioned on a data plane. Built on Envoy AI Gateway. See Architecture overview.
API key
The credential an application presents to the gateway. Each key is associated with a Console user, its own fallback and traffic-splitting policies, and optional per-key rate limits.
Attribute-based routing
An ordered list of routing rules with a default, in which each rule matches on attributes the calling application declares about a request (sent in the X-Tars-Metadata header) and names the model to serve it. The first matching rule wins, and a request that matches nothing takes the default. Planned for the 2026 Q3 release. See Model alias for the case where every caller moves together, and Smart routing.
Audit log
The immutable record of every state-modifying administrative action: model and provider changes, role and membership edits, SSO configuration changes, API key lifecycle events, and data-plane changes. Each entry carries a timestamp, an actor, an actionType, the affected resource, and structured context, and no user can edit or delete one once it is written. Retention is configured per deployment, separately from the request-log window, and an audit entry survives a request-log purge.
Distinct from Request logs, which record model traffic rather than administrative change: the audit log answers who did what, while request logs and Usage analytics answer what is happening. Read in the Admin Console under Audit Logs. See Audit log events.
Backend
In routing terms, a model-on-a-provider combination plus the credentials needed to reach it. Routing decisions resolve to a choice of backend.
Billed spend
The portion of recorded spend that counts toward a Budget meter. Passthrough usage is excluded: it appears in the Admin Console Spend overview but is not billed spend and does not consume a budget. See Passthrough mode.
Bring your own key (BYOK)
The mechanism by which a consumer's own upstream provider credentials are used in place of centrally managed credentials. BYOK credentials are configured at the Console account level and are slotted into routing chains alongside centrally managed credentials. See Use Your Own Provider Credentials.
Budget
A spend control over a fixed period, created and managed at Usage → Budgets in the Admin Console. A budget carries a name, a scope (Whole team, Each teammate, or One person), a spend limit in US dollars, a period, an Active toggle, and an audit record of who created it and when. The period is daily, weekly, or monthly: a daily budget resets at midnight UTC, a weekly budget on Monday, and a monthly budget on the 1st at 00:00 UTC. The name, the limit, the period, and the Active toggle are all editable in place. A budget measures money, never tokens or request counts, and the figure it measures is calculated cost.
A budget measures; it never enforces. It tracks spend against its limit and reports on it, and that is the whole of what it does. Spend is evaluated on the management plane against aggregated request logs rather than inline at the gateway, so a budget cannot stop a request that would cross its limit, and crossing the limit changes nothing about what the caller can do next.
Stopping calls takes a separate object: a Budget policy. See Set a budget and track spend against it for the measuring half, and Enforce budget caps with policies for the enforcing half. Budgets defines the type itself. A Rate-limit policy is the different control that is enforced inline today.
Budget policy
The object that turns a Budget into a limit that holds. A budget measures spend and holds the threshold; a budget policy takes that threshold as its trigger and acts on the request path when it is crossed. One surface owns the number, the other owns the consequence.
Its action is block, which refuses the call, or degrade, which switches to a cheaper model and keeps serving. Where the action is block, an exhausted budget means the user or client cannot make calls at all until the period resets or the limit is raised. Like every policy, a budget policy is held in monitor or active mode; in monitor it records what it would have done and changes nothing.
Budget policies are planned for the 2026 Q3 release. See Budgets for the type in full, including enforcement lag and fail-open behavior, and Enforce budget caps with policies for the scenarios.
Calculated cost
Spend derived from response tokens priced against a catalog of published provider pricing that is refreshed regularly, rather than read from the provider's invoice. Negotiated rates are entered as per-model overrides so that the catalog reflects the contract in force. Capacity purchases (provisioned throughput, committed spend, and subscriptions) are not comparable to a per-token figure. Calculated cost is the figure a Budget measures.
Canary deployment
A release strategy in which a change is rolled out to a small share of traffic first, evaluated against error rates, latency, and cost, and then widened as confidence grows or withdrawn if the numbers worsen. The name comes from the canary carried into a coal mine as an early warning. Both versions run at once, so the exposure at any moment is bounded by the share of traffic the new one is taking.
In Agent Router the pattern is expressed with Traffic-splitting policy: the new model starts at a small weight and the weight is raised over successive edits. See Reduce Cost with Traffic Splitting.
Catalog
The inventory of models, providers, and MCP servers at a scope. The organization catalog is the organization-wide inventory; a project catalog is the subset granted to one project. Surfaces label it simply "Catalog" and convey scope with a selector inside the view rather than by prefixing the label. Spelled catalog, never catalogue.
Client
A registered application identity, and the parent of API keys. A client is owned by a User and scoped to one Project. Only this sense takes the bare word; every other application-identity concept is qualified, such as OAuth client (MCP).
Console
Short form for the Developer Console.
Control plane
Internal vocabulary; not used on customer-facing surfaces. Inside a data plane, the control plane is the configuration machinery beside the proxy that serves traffic. The customer-facing terms are Management plane for the central system that manages configuration, catalog, and policy, and Data plane for a deployed installation.
Tetrate marketing and the public FAQ have used "control plane" to mean the management plane. The two are not the same thing, which is why the term is avoided here.
Controller
The data-plane component that bridges the Data plane and Management plane. Polls the management plane for configuration updates and translates them into gateway configuration, keeping routing rules, policies, and provider credentials in sync without inbound connections from the internet.
Correlation ID
A UUID attached by the gateway to every request, exposed as the X-Request-ID response header and emitted as a span attribute on the corresponding OpenTelemetry trace. The primary identifier for joining application logs with Request Logs and traces.
Cost attribution
The practice of tying AI spend and usage to a team, application, or agent. In Agent Router Enterprise, attribution relies on per-purpose API keys, Usage analytics, optional export to an observability stack, and (with SSO) authenticated identity on every request. Supports showback and chargeback workflows. An Enterprise-tier capability.
Data plane
The request-path half of Agent Router: a deployment containing the Controller and an AI Gateway proxy built on Envoy AI Gateway. All AI traffic flows through it. Where it runs depends on the deployment model: Tetrate operates it in Agent Router Service and Agent Router Enterprise Fully Managed; the customer runs it in their Kubernetes cluster in Agent Router Enterprise Self-Hosted Data Plane (where prompts and responses stay inside the customer's infrastructure). Only configuration and telemetry cross to the management plane.
Developer Console
The developer-facing application. Where API keys and their policies, MCP profiles, integrations, the Playground, request logs, and personal usage analytics are accessed.
Dynamic module
A high-performance proxy extension (written in Rust or Go and compiled to a shared library) that runs inline in the gateway's filter chain. Dynamic modules implement Agent Router's routing logic, credential handling, and provider-specific translation.
Endpoint picker
A pluggable component (Endpoint Picker Provider, EPP) that selects the best upstream inference endpoint per request from a pool of candidates, using real-time signals rather than static weights or round-robin. Standard signals include KV-cache utilization, queue depth, and prefix-cache affinity. Defined by the Kubernetes Gateway API Inference Extension; integrated in Envoy AI Gateway via InferencePool + Endpoint Picker Provider. In Agent Router, the Endpoint Picker operates within the eligible set defined by the key's policies. It does not override policy boundaries. See Apply Advanced Routing Rules and Load Balance Across Regional Deployments.
Envoy AI Gateway
The open-source, CNCF-backed AI gateway co-created and maintained by Tetrate and Bloomberg. The data-plane proxy technology underneath Agent Router Enterprise. Tetrate Agent Router adds the Management plane, model and MCP catalogs, cost attribution, AI Guardrails, SSO, and multi-gateway governance on top. See the Envoy AI Gateway project for upstream documentation, including the endpoint-picker guide.
Fallback policy
An ordered list of backends per API key. The gateway tries the first backend; if it fails with a retryable error, the gateway walks to the next backend, continuing until a backend succeeds or the chain is exhausted. A fallback policy reacts to the primary backend failing; matching a request to the model that should serve it is Smart routing, a separate concern. See Improve Resilience with Fallbacks.
Gateway
The resource that exposes and manages a Project's AI traffic on a Data plane: a single inference endpoint with one customer-facing URL, provisioned on a data plane and mapped to exactly one project. Applications call the gateway's hostname over the Gateway APIs with a key issued for its project. Routing is by hostname, so every gateway carries its own unique hostname, and isolation is enforced at request time: a key from another project returns 403, and a model the project has not been granted returns 404.
A gateway is created only by provisioning. Selecting a data plane on a project provisions its first gateway automatically; there is no standalone create flow. The data plane is shared capacity that can host many gateways, while each gateway serves exactly one project, which is how a project is placed in a particular region or cloud. Distinct from the AI Gateway, the proxy component inside the data plane that gateways are provisioned on. The API and audit-log surfaces name the same resource project_gateway. See Provision a gateway for a project.
On Agent Router Enterprise, additional gateways can be attached to a project for high availability, and a project with zero gateways remains fully configured, just not serving. The high-availability sub-vocabulary:
- Gateway set: the gateways attached to one project, interchangeable by construction because every member receives the identical project configuration.
- Member: one gateway in a project's set. Detaching a member empties it but never deletes the record.
- Membership probe: the presence-coupled health endpoint on a member's own hostname whose meaning is "ready member of this project", the target for DNS health checks. Published at
/healthz(the original/healthz/membershippath still answers the same body and is not deprecated). - Project endpoint: the operator-owned, client-facing name shown to developers in place of any member URL. Display only; Agent Router never routes on it.
See Gateway sets and DNS-level failover.
Group
A directory group whose membership is held in the corporate identity provider and read by Agent Router from the OIDC groups claim at request time. A group is never created in Agent Router: it takes effect only once mapped, and the Agent Router-side label it maps to is the business function that rate limits and budgets are scoped to. A membership change takes effect at that member's next sign-in rather than immediately.
Distinct from a Team, which is created in the Admin Console and allows one team per user. One collision to watch: the Access page in the Admin Console calls its team principals "groups", so "users and groups" there means users and teams. See Map Entra ID groups to business functions.
Guardrail
A named bundle of content and safety rules, evaluated on requests and responses. Guardrails and their sub-objects are scoped to a Project: they are configured per project and evaluate that project's traffic. Implemented inline in the data plane by a Dynamic module, so a blocked request never reaches the provider.
Its sub-vocabulary:
- Guardrail rule: the enforcement unit inside a guardrail. A rule names a check (PII, toxicity, prompt injection, and others), the engine that evaluates it, whether it runs on input, output, or both, and an action, block or redact, applied in enforce mode or only recorded in monitor mode.
- Engine: the configured evaluator a rule references. The Tetrate engine combines the built-in regex and Tetrate evaluators and is configured in-product; external engines delegate configuration to the external system. An engine is not a Provider, which is reserved for upstream model vendors.
- Pattern: a reusable, categorized regex definition that rules reference by name.
- Trigger: the record of a guardrail firing, carrying which rules evaluated, their scores, and what each action did or would have done.
On tetrate.io the capability is marketed as AI Guardrails. See Configure guardrails and the guardrails reference.
Identity provider (IdP)
The external system that holds user identities and issues the token Agent Router accepts at sign-in. Agent Router keeps no password database of its own, so the identity provider is the only authority on who may sign in. OIDC is the only supported protocol; SAML is not supported. Microsoft Entra ID, Google Workspace, Auth0, Okta, and any OIDC-compliant provider are documented.
The identity provider is the authority for three separate things: authentication, role assignment through Role mapping, and the Group membership that rate limits and budgets are attributed to. See Single sign-on (SSO) and Configure SSO.
Logical model name
A name exposed by a key's policies that the gateway resolves to a specific provider model identifier. Decouples application code from provider model version strings. Configured through model-name overrides; see Apply Advanced Routing Rules.
Managed mode
The default credential path, in which Agent Router presents centrally managed provider credentials and the resulting traffic is billed to Agent Router. Contrast Passthrough mode, where a flat provider subscription pays for the traffic, and Bring your own key (BYOK), where the consumer's own provider credentials are presented instead.
Management key
The operator credential for the management APIs. Distinct from an API key, which authenticates inference requests, and from a Provider key, which bills upstream traffic. Management keys are authorized through role bindings at organization or project scope; inference keys are outside that model.
Management plane
The Tetrate-hosted control surface. Stores routing rules, policies, user records, API key metadata, audit history, and analytics rollups. Communicates with the data plane through a small, well-defined interface; no application payloads cross the boundary. Tetrate marketing and the public FAQ refer to the same component as the control plane; the terms are interchangeable.
MCP catalog
The set of MCP servers registered in Agent Router by operators. The catalog is the universe from which developers assemble MCP profiles in the Console. Managed in the Admin Console under MCP Servers.
MCP Gateway
The MCP-routing and governance component of Agent Router: operators curate an MCP catalog centrally, developers assemble MCP profiles in the Console, and agents reach approved tools through governed endpoints. On tetrate.io this is a distinct product pillar alongside the AI Gateway. Tool calls receive the same attribution and logging as LLM requests.
MCP profile
A named collection of MCP servers exposed through a single Agent Router URL. AI clients connect to the profile URL once and receive access to every server included in the profile. See Aggregate MCP Servers into a Profile.
MCP server
A Model Context Protocol server offered in the Catalog and grantable to projects. Assembled into an MCP profile for use by AI clients.
Model
A callable model in the Catalog, offered by a Provider. Logical model name and Resolved model are technical vocabulary describing how a request resolves to one, not separate product objects.
Model alias
A stable name the organization owns that resolves to a model chosen in the catalog, so that a version change, an end of life, or a reprice is one catalog edit rather than a release in every service that names the model. An alias has no rules and no conditions: one name, one model, repointed by an operator. Available today. See Smart routing.
Model context protocol (MCP)
An emerging standard for exposing tools, data sources, and context to AI clients. MCP servers act as adapters between AI clients (Claude Code, Cursor, VS Code, and others) and the systems those clients need to reach.
OAuth client (MCP)
A configuration record holding the credentials and endpoints needed for OAuth-authenticated MCP servers. Managed in the Admin Console under MCP OAuth Clients.
OpenTelemetry (OTel / OTLP)
The open observability standard Agent Router uses for trace and metric export. OTLP is the wire protocol. The gateway exports traces over OTLP; metrics are exposed on a Prometheus scrape endpoint that can be forwarded as OTLP by an external collector.
Organization
The top-level container. It holds users, teams, API keys, billing, the shared model, provider, and MCP catalogs, and every Project. It is the widest Scope a policy can take, and an organization-scoped policy binds: a project cannot override it, which is why creating one reports its blast radius before saving. Selected from the switcher at the top left of the Admin Console, above the list of projects inside it.
Passthrough mode
A credential mode in which a flat Anthropic subscription fee (Claude Pro or Max, for example) continues to pay for the traffic while Agent Router Enterprise monitors it through Request Logs, usage, and the Admin Console Spend overview. Anthropic still authenticates the caller with Claude Code's stored OAuth token, so the gateway routes and observes without putting those tokens on Agent Router billing. In the Spend overview, passthrough usage is the theoretical API-rate cost that traffic would have carried without the subscription, and it does not count toward a Budget. Passthrough mode is separate from BYOK: a Claude subscription is not pasted into Bring Your Own Key. Contrast Managed mode.
Permission
A single capability on one resource, in one of two kinds. A read permission decides whether a menu and its page are visible; a write permission decides whether the New, Edit, and Delete actions appear and are accepted. Enforcement happens at the menu and again at the page, so a hidden menu is a real restriction rather than a cosmetic one.
Permissions are bundled into a Role and normally granted that way. Granting one directly to a person is possible but leaves the access invisible to anyone reviewing roles. Effective access is the union of every grant a person holds, directly and through a team, and no grant can confer access the operator making it does not already hold. See Manage roles and permissions.
Platform operator
The persona responsible for running Agent Router: provisioning models and providers, governing access, configuring SSO, auditing usage, and operating the system across environments.
Playground
An interactive testing surface inside the Console. Sends messages to any enabled model and renders responses, token counts, and latency in real time. Playground traffic flows through the same gateway as application traffic. See Test Prompts in the Playground.
Policy
An admin-managed rule that decides what happens to a model request. Each policy carries a type, which control it is; a Scope, which API keys it governs; a configuration belonging to that type; and a mode, monitor or active. Five controls are built on that one shape, each pairing a trigger with an action: budget, availability fallback, traffic split, rate limit, and guardrail.
Setting a rule once and having it reach a whole class of keys, rather than configuring the same control on hundreds of keys by hand, is the reason the model exists. A key's own settings are simply a policy scoped to that one key. At most one policy of a given type is active per target, editing replaces rather than adds, and a narrower policy can never loosen a broader one. See Policies.
Project
The logical boundary that owns configuration and access: the models and providers granted to it, its API keys, its MCP servers and profiles, its routing and policy, its budgets, and its members. It is also the unit of attribution, so usage and audit records are tied to the project the traffic ran under. A project plays the role a workspace or a tenant plays in other tools.
A project is logical and runs no software of its own. A Gateway is the physical endpoint that serves it, one gateway to one project, and a project has no callable URL until a gateway is provisioned for it. Isolation is enforced at request time rather than only in the applications: a key issued for one project is rejected on another project's gateway with 403, and a model the project has not been granted returns 404. Distinct from a Team, which groups people and is not an isolation boundary. See Create a project.
Prompt and response
The text sent to a model and the text it returns. A prompt is not necessarily one block of prose: it can carry plain text, images, documents, tool output, retrieved context, and the model's own thinking, and a guardrail rule declares which of those parts it inspects. Responses are delivered whole or as Streaming server-sent events. Neither ever reaches the management plane; both stay on the data-plane request path. See Data flows.
Provider
An upstream AI service (OpenAI, Anthropic, Google, Azure OpenAI, Mistral, and others) that the gateway routes requests to. Provider configurations carry credentials and connectivity details.
Provider key
A credential for an upstream Provider, so that traffic to that provider's models bills to the owning account. Two kinds: an organization provider key, configured when the organization enables a provider and used by default for all traffic to it; and a BYOK key, registered by a project or by an individual user under the Bring your own key (BYOK) feature, which overrides the organization key for that project's or that user's traffic.
BYOK names the feature; provider key names the object.
Provider translation
The transparent conversion the gateway performs between its OpenAI-compatible request surface and the provider-specific APIs of upstream services. Includes header mutations, body mutations, path rewriting, model-name override, and response normalization. See Gateway APIs.
Rate-limit policy
A cap on the rate of traffic, enforced inline by the gateway. Three limit types are configured in the Admin Console: tokens per minute (TPM), requests per minute (RPM), and maximum parallel requests, each applicable at the model, API key, user, or group scope. Each API key additionally carries Console-side token limits over a rolling hour, available on input tokens, output tokens, total tokens, or any combination. Every applicable limit is evaluated independently and the most restrictive one governs; a request that would breach an active limit receives HTTP 429 instead of being forwarded upstream. A rate limit is flow control over a short window, not a spend cap: Budget is the different control, measuring money over a budget period and never rejecting anything, and Budget policy is what refuses a caller whose budget is exhausted. See Stop runaway workloads before they burn the budget.
Redirect URI
The tenant-specific callback URL that the identity provider returns the user to after authentication, in the form https://auth.<tenant>.tetrate.ai/api/auth/sso/callback/corporate. Surfaced in the IdP setup guide panel of the Configure SSO step, and registered on the OIDC client in the identity provider. Most providers require a character-for-character match. See Configure SSO.
Request logs
The per-request record in the Console. Captures the resolved model, provider, token counts, latency, cost, and full request and response payloads for every gateway call under the user's API keys.
Resolved model
The model that actually served a request, after the key's policies have been evaluated. May differ from the model the application asked for (the requested_model) due to fallback walks, traffic splits, or logical-name overrides.
Role
An Agent Router-level permission set. The role model is small: super_admin, model_admin, provider_admin, mcp_admin, user_admin, billing_admin, and user. Roles can be assigned manually or driven by SSO claim mapping. The first super_admin is established during onboarding, and can then be granted to further users, so a deployment is not limited to one.
Role mapping
The mechanism by which OIDC claims (app roles, group memberships) are translated into Agent Router roles on every login. Configured on the SSO provider in the Admin Console. See the SSO role mapping guide.
Routing chain
The ordered or weighted list of backends attached to an API key. May be a fallback chain (priority-based), a traffic split (weight-based), or a combination.
Routing configuration
Deprecated. Not used. Name the specific policy instead: Fallback policy, Traffic-splitting policy, Rate-limit policy, or Budget policy. The settings on one API key are that key's policies, not a separate object with its own name.
Routing policy
An ordered list of routing rules with a required default, attached to a scope and held in one of two modes. A routing policy targets one of four scopes: an organization, a project, a tag selector, or a single key. It is managed in the Policies area of the Admin Console.
A routing policy is not a per-key setting. The fallback, traffic-splitting, rate-limit, and budget policies on one API key are edited on that key and apply only to it. A routing policy is a separate object that reaches a whole class of keys at once, and exactly one of them applies to any given key. See Routing policy reference.
Scope
Two different things carry this word.
Policy scope is how a Policy chooses the API keys it governs, drawn from a set of six: organization, project, tag selector, key, team, and user. The first four select keys by looking at the key itself; the last two select by looking at who owns the key, which is a different question rather than a narrower one. Only key, project, and organization sit inside one another, so those are the only three where calling a scope "more specific" means anything. Not every policy type offers all six. See Policies.
Scope levels are the access-control sense. On a Role, they state where that role is allowed to be granted, organization or project, while the Apply to setting on an individual grant states where it actually applies. See Manage roles and permissions.
Semantic routing
Choosing a model from the content of a request, for example a classifier deciding that a question needs the expensive model. Agent Router does not do this. Every Agent Router routing mechanism matches on what the caller declares (the API key presented, the model name requested, or the attributes in the X-Tars-Metadata header), never on what the prompt says. The term is listed here as an explicit non-feature, because it is widely assumed to be present.
Session affinity
The gateway property that keeps a multi-turn conversation, agent workflow, or MCP session pinned to a consistent processing path for its entire lifetime. Applied automatically; no configuration required.
Shared workspace URL
The single URL a data plane answers on before any project gateway has a URL of its own, registered as the Data plane URL during onboarding or under System → Settings → Data planes. While it serves, it carries a default route to every model. Registering a URL on the first project Gateway suppresses that default route, and the data plane then serves only project gateway hostnames. See Provision a gateway for a project.
Single sign-on (SSO)
Delegated authentication through a corporate identity provider over OpenID Connect (OIDC). Configured during onboarding, and thereafter in the Admin Console under Settings → SSO. See Configure SSO.
SSO is the only way to sign in. Agent Router holds no credentials of its own and offers no email and password path against its own user store. What a person actually types at the sign-in prompt is decided by the identity provider: a username and password is the common case, and whether any fallback exists when the identity provider is unreachable is the identity provider's behavior, not Agent Router's. The practical consequence is that administrative access must be bootstrapped through the Admin emails field before the first sign-in, because there is no local account to fall back on. In Agent Router Enterprise, SSO also means every gateway request can carry authenticated user and team identity, enabling per-team cost attribution and audit; this is an Enterprise-tier capability.
Smart routing
Documentation framing for the business-level mechanisms that match a class of request to the model that should serve it: Model alias and Attribute-based routing. Distinguished from low-level traffic routing (Fallback policy, Traffic-splitting policy, and Rate-limit policy), which governs how a request reaches a backend rather than which model answers it. See Smart routing.
Streaming
Server-sent events (SSE) delivery of incremental response tokens, carried over HTTPS on both HTTP/1.1 and HTTP/2. Supported across all three API formats; see Gateway APIs for the per-format SSE shape and the per-endpoint list of what streams. The gateway relays chunks without buffering them, which is also what prevents a failure after the first chunk from being retried or normalized: see Streaming behavior for the failure, fallback, guardrail, and latency semantics.
tare (CLI)
The command-line interface, one tool across both product lines. It installs and runs as tare, which is the name used in every example on this site and throughout the CLI reference.
Team
A named group of users in the Agent Router directory, created on Directory → Teams. Teams exist so that access can be granted once to a group instead of person by person, and so that cost and usage can be attributed to a group rather than only to individuals. Membership is exclusive: a user belongs to one team at a time, and adding a user to a second team moves them out of the first.
A team carries no permissions of its own and is not an isolation boundary; the boundary for models, keys, MCP servers, and policy is the Project. A team is one of the six policy scopes and one of the three subjects a Budget can take. Adding a team to a project copies its members at that moment rather than linking them, so people who join the team later are not added retrospectively, whereas a role granted to a team stays live. Distinct from a Group, which lives in the identity provider. See Create and manage teams.
Tetrate Agent Router
The umbrella product name for Tetrate's AI gateway offering. Includes Agent Router Service (self-serve) and Agent Router Enterprise (dedicated instance with full governance). Sits in front of existing agent frameworks and provider SDKs via an OpenAI-compatible API; built on Envoy AI Gateway. This documentation set documents Agent Router Enterprise unless a page explicitly compares tiers.
Traffic-splitting policy
A routing strategy in which all backends sit at the same priority and share traffic by weight. Used for cost reduction, A/B evaluation, and gradual migration. A traffic split is a weighted percentage configured on an API key and applies whatever the requests are; matching a request to the model that should serve it is Smart routing, a separate concern. See Reduce Cost with Traffic Splitting.
Usage analytics
The aggregated traffic surface. Two variants exist: the Console version, scoped to the signed-in user's own data, and the Admin Console version, aggregated across all users and API keys in Agent Router.
User
A human principal that signs in. Users belong to the Organization. A project's Users list is its membership, and each user carries a role in that project. The people in a project are labeled Users on every surface; "project member" is not used as a noun.
Where to go next