Routing policy reference
Enterprise Tier
Smart routing in Tetrate Agent Router is built from a small set of objects, presented here in the order they relate to one another: the request attribute an application declares, the routing policy that holds an ordered list of rules, the scope that decides which policy applies to a key, the mode that decides whether it acts, and the record every request leaves behind. Each entry states what the object holds and how it is resolved. There are no procedures here: the scenarios that show these objects being used are on the smart routing page, and each of them links back here.
Attribute-based routing is scheduled for the 2026 Q3 release. It is not available yet, and is documented ahead of the release so that the design can be reviewed and planned against.
The shapes shown in text blocks on this page are illustrative, to show what an object holds. None is a schema.
Persona: Platform operator working in the Admin Console.
Estimated time: 15 minutes to read.
The objects
Three objects, and each is defined below in this order.
| Object | What it is | Where it lives |
|---|---|---|
| Request attribute | A named thing an application may declare about a request | The catalog |
| Routing policy | An ordered list of routing rules with a default, attached to a scope | The Policies area |
| Routing rule | One condition and one destination, inside a policy | Inside a policy |
The division is between the catalog entry, which is a reusable definition that many policies reference, and the policy itself, which is an attachment: its rule list is its configuration, so it holds no catalog entry of its own.
Request attribute
A request attribute is a catalog entry, in the same way a guardrail rule is: a reusable definition that many policies reference. Defining one changes no traffic.
Request attribute
name workload, task, plan
source where the value comes from
value type one of a named set, or free text
allowed values when the value type is a named set
| Field | Holds |
|---|---|
| Name | The attribute's identifier, used unprefixed by the calling application and prefixed by its source in a rule |
| Source | Where the value comes from, and therefore whether the caller can set it |
| Value type | Either a named set of permitted values, or free text |
| Allowed values | The permitted values, when the value type is a named set |
An attribute whose value type is a named set can be matched only against a value from that set. A request may still carry a value outside it: the value is recorded, matches no condition, and the request takes the policy's default.
Source, and the name a rule uses
Source records where an attribute's value comes from. One source exists today, metadata, which means the value is set by the calling application on each request. A rule names an attribute by its source, so a rule reads metadata.workload rather than workload. The catalog and the rule builder state the source on every attribute, so a rule author can see what a condition depends on.
The prefix distinguishes values the caller sets from values the caller does not, so a source that is not caller-supplied can be added later without an existing rule becoming ambiguous.
The name is unprefixed in the header the application sends, because the header is the source. See how a routing rule knows what a request is for the wire form.
Routing policy
A routing policy is an ordered list of routing rules with a required default, attached to a scope and held in one of two modes.
Routing policy
scope organization | project | tag selector | key
mode monitor | active
rules an ordered list, evaluated top to bottom
default required, applied when no rule matches
A routing policy is a policy type, so it is created, targeted, monitored, activated, and listed the same way every other policy type is.
Routing rule
Routing rule
condition attribute, operator, value
destination a model available to the project
A condition compares one attribute against one value, or against a set of values. Conditions may be combined within a rule, so one rule can require both that metadata.workload is batch and that metadata.task is summarization.
A rule's destination is a model available to the project. Routing chooses between the models a project already has, and cannot reach one the project does not.
The default
Every policy has a default, and it is required. A request that matches no rule is served by the default. A routing policy cannot reject a request, so a caller that declares nothing, declares a value the catalog does not allow, or declares an attribute that does not exist is served the default rather than an error.
Evaluation order
The first matching rule wins. Rules are evaluated top to bottom, and evaluation stops at the first match.
Three things cause a rule not to match, and all three have the same consequence: evaluation continues at the next rule, and reaches the default only when every rule has been tried.
| Cause | Example |
|---|---|
| The attribute the condition names was not declared on the request | A rule on metadata.workload against a request that sent no attributes |
| The value declared is not the value the condition tests for | A rule on metadata.workload is batch against workload=interactive |
| The rule's destination is unavailable | The model named by the rule is down |
Scope resolution decides which rule list applies to a key. The rule list is evaluated per request. A policy resolves to one rule list for a key, and which rule inside that list wins varies request by request.
Scope and resolution
The scope set
A routing policy targets one of four scopes.
| Scope | Selects |
|---|---|
| Organization | Every key in the organization |
| Project | Every key issued in that project |
| Tag selector | Every key carrying the tags named by the selector |
| Key | One API key |
There is no team scope and no user scope. A policy selects keys by where they are issued and how they are tagged, never by key ownership. Routing traffic per team is covered under things a routing rule cannot do.
Precedence
A key resolves to exactly one routing policy, because two rule lists cannot be merged. Where more than one policy could apply, precedence decides.
| Order | Scope | Note |
|---|---|---|
| 1 | Organization | Always wins |
| 2 | Key | The narrowest of the remaining three |
| 3 | Tag selector | |
| 4 | Project | The widest of the remaining three |
An organization-scoped routing policy binds. A project cannot override it. A project whose routing is overridden this way sees that on its own routing page, read-only, with a link to where the policy is managed.
Below the organization, the policy targeting the narrower workload wins, so a key-scoped policy replaces a project-scoped one for that key rather than combining with it.
Impact before saving
Creating or editing an organization-scoped routing policy shows what it will change before it is saved: how many keys, in how many projects, and which projects' own routing will stop being used.
The coverage counts, the inheritance view, and the effective policy view that apply to every policy type apply to routing policies too, and answer what a given key resolves to.
Modes
A routing policy is held in one of two modes.
| Mode | What happens to traffic | What is recorded |
|---|---|---|
| Monitor | Nothing changes. Every request goes where it would have gone with no policy at all | The rules are evaluated against real traffic, and how many requests each rule would have matched over a recent window, with the model each would have moved to |
| Active | The rules decide the destination | The full decision record |
Monitor is the mode a policy is held in before it is activated. Reviewing a policy in monitor before it takes effect is covered in A policy has to be reviewed before it takes effect.
Monitor proves matching, not quality. The destination a rule names is never actually called in monitor, so nothing recorded there says whether its answers would have been suitable.
The decision record
Every request records why it went where it went.
| Recorded | Detail |
|---|---|
| The routing that applied | Which policy, and where in the rule order the match happened |
| The rule that matched | A request that matched no rule records that the default applied |
| The attributes evaluated | Every attribute the evaluation read, and the source of each |
| The destination | The model that served the request |
| How it turned out | Whether it succeeded, how long it took, and what it cost |
The response itself identifies the model that actually served the request, so a caller can tell what served a call without opening anything.
Attributes travel with the request into the usage records, which is what makes a routing decision reviewable along the same axes spend is read on. See Know what every app and project costs.
Where each object is managed
| Object | Surface | Operations |
|---|---|---|
| Request attribute | The catalog | Create, edit, remove, and see which policies reference one before changing it. The catalog also shows whether a name has been seen on real traffic recently |
| Routing policy | The Policies area | Create, target, monitor, activate, list |
Everything available in the Admin Console is available in the admin API, so a routing policy can be managed as code.
Policy changes reach the data planes by propagation rather than instantly.
Roles
| Role | On routing |
|---|---|
| Administrator | Defines attributes in the catalog, and creates, targets, and activates routing policies |
| Developer | Declares attributes on the requests their application sends. Developers hold no dashboard role in routing: they do not create or see routing policies |
A rule matching on an attribute is inert until the application team that owns the calling code sends it.
What a routing policy is not
Routing decides which of a project's models serves a request. Four nearby things are decided elsewhere, and each is a separate mechanism with its own guide.
| Not a routing policy | What decides it | Where it is covered |
|---|---|---|
| Which models a key may use at all | The project's model set | Create a project |
| Where a request goes when the destination fails | A fallback policy | Improve resilience with fallbacks |
| A fixed percentage across two backends | A traffic split | Reduce cost with traffic splitting |
| What happens when a spend limit is reached | A budget enforcement action | Enforce a budget cap |
The cases a routing rule cannot cover, with what to do instead, are listed under things a routing rule cannot do.
Where to go next
Smart routing
These objects worked through ten scenarios, from a hardcoded model name to a degraded provider at 2am.
Create a project
The model set that bounds every destination a routing rule can name.
Glossary
The routing terms used across these pages, including the ones that name mechanisms Agent Router deliberately does not have.