Smart routing: match each request to the right model

Enterprise Tier

Smart routing is matching a request to a model from attributes the caller sends. Ten operator scenarios cover the rules people write most often, worked against one company so the examples build.

---

Availability note

Smart routing is scheduled for a 2026 Q3 release.

Everything else named on this page is available today: model aliases, covered in the section below, fallback, traffic splitting, advanced routing rules, and rate limiting.

Persona: Platform operator working in the Admin Console, in partnership with the application teams that own the code sending the requests.

Estimated time: 15 minutes to read; 10 to 20 minutes per policy, plus whatever the owning application team needs to ship one request header.

Related concepts

Smart routing decides which model serves a request from what the calling application declares that request to be. Several nearby mechanisms answer different questions, and each has its own guide.

Not covered here	What it does instead	Where it is covered
Availability fallback	Moves a request to another backend when the primary model fails	Improve resilience with fallbacks
Traffic splitting	Sends a set percentage of requests to a second backend, selected by chance rather than by what the request is	Reduce cost with traffic splitting
Budget-based fallback	Switches a request to a cheaper model because a spend limit has been reached	Enforce a budget cap
Rate limiting	Refuses requests once a token, request, or concurrency ceiling is hit, without changing which model is chosen	Stop runaway workloads

Routing in Tetrate Agent Router Enterprise

Available now

Mechanism	What decides the model	Covered in
Model alias	A single central decision, applied to every caller	Model aliases
Fallback	The primary model failing	Improve resilience with fallbacks
Traffic split	A percentage set on the API key's routing configuration	Reduce cost with traffic splitting
Advanced routing rules	Operator-defined routing configuration	Apply advanced routing rules
Rate limiting	Nothing: a rate limit is flow control, not a choice of model	Stop runaway workloads

Coming in 2026 Q3

Mechanism	What decides the model	Covered in
Attribute-based routing (smart routing)	What the calling application declares the request to be	Attribute-based routing

The objects all of these are built from, and the rules that resolve them, are described in the Routing policy reference.

Model aliases (available today)

A model alias is a name that outlives the models behind it. Applications send claude-opus, and the catalog decides what that resolves to: today claude-opus-5, next year claude-opus-5-1. The applications never learn the difference.

That indirection is worth having before it is needed. Model names change every few months (a new version, an end of life, a reprice), and without an alias each of those changes becomes a release in every service that names the model. With an alias, it is one edit in the catalog. The teams that hardcode claude-opus-5 today are the ones that cannot upgrade quickly next year, because the name is spread across repositories nobody has a list of.

An alias has no rules and no conditions: one name, one model, repointed whenever the operator chooses. Aliases are managed in the catalog, alongside the models themselves. The 1: hardcoded model name scenario works through the case they exist for.

A provider's own undated name (claude-opus-5 rather than claude-opus-5-20260112) already moves with that provider's releases. An alias goes further, because it is a name the organization owns: it can outlive the provider as well as the version.

Aliases are the cheapest part of smart routing to adopt, because they are available today and need no change in any calling application.

Attribute-based routing (coming in 2026 Q3)

Attribute-based routing is an ordered list of routing rules with a default. Each routing rule matches on what the calling application says the request is, and names the model to serve it. The first matching rule wins, and a request that matches nothing takes the default. It is configured in the Policies area. Every scenario from 2: overnight batch work onwards is built out of these rules.

The choice between the two mechanisms comes down to one question: whether every caller of a name should move together.

• Every caller should move together (a version bump, an end of life, a reprice): use an alias.
• The same name should resolve differently depending on what the request is: use a routing rule.

The two also compose. A rule names the model it routes to, and that name can itself be an alias, so a rule written once keeps working through a version bump.

How a routing rule knows what a request is

The calling application sends attributes with the request, in one header. A rule matches on them. Nothing reads the request body, and nothing is inferred.

POST /v1/chat/completions
Authorization: Bearer <api-key>
X-Tars-Metadata: workload=batch

{ "model": "opus", "messages": [ ... ] }

Everything else about the call is unchanged: same endpoint, same body, same SDK. The application still names a model; the rules decide what actually serves it.

One header carries every attribute. A request with more than one thing to say separates them with commas, and a rule may match on either attribute or on both:

X-Tars-Metadata: workload=batch,task=summarization

A rule names an attribute by its source. The header key stays bare, because the header is the source, and the rule that reads it is written metadata.workload. One source exists today, metadata, meaning a value the calling application sets on each request. The prefix is what allows a source the caller cannot set to be added later without any existing rule becoming ambiguous. See Source, and the name a rule uses.

Three steps sit behind every working rule, and they belong to different people:

1. An operator declares the attribute in the catalog: its name, and the values it is allowed to take.
2. The application team sends the header on the calls its code makes.
3. An operator writes the rules that match on it.

The middle step is the one an operator does not control, and most of the cautions on this page trace back to it.

Attributes are not only for routing. They travel with the request, so the same values appear in the usage records, which is what makes a routing decision reviewable after the fact. See Know what every app and project costs for reading spend along those same axes.

How a request resolves

Whichever branch is taken, the destination has to be a model the project is allowed to use. A project is the unit that holds a set of models, the API keys issued against them, and the policies that apply to those keys. Routing chooses between the models a project already has. It cannot reach one the project does not.

Three properties that hold for every rule

A request that sends no attributes keeps working. It takes the policy's default. No rule can make a request fail, so a team that has not added the header yet is unaffected by a policy turned on today.

Routing reaches applications that can be changed, and no others. Work that runs through a tool outside the organization's control cannot send the header, so no rule applies to it. Restricting the 10: group's project is the option for that case.

A rule is a preference, not a control. The caller sets the value, so a caller that omits it, misspells it, or sets it to something else gets the default rather than an error. That is what stops a rule breaking an application, and it is also why a rule cannot keep a request away from a model. A request that must never reach a model is handled by removing that model from the project: see things a routing rule cannot do.

Prerequisites

• Administrator access to the Admin Console, with permission to edit the catalog and the Policies area.
• A project whose model set already contains every model the rules name. Routing selects among a project's models and cannot add one. See Create a project.
• Agreement with the application team that owns the calling code, for any rule that matches on an attribute. The rule is inert until that team sends the header.

The company used in every scenario

A bank's retail division. Three pieces of work use models:

The work	What it does	Models it can use
customer chat	answers cardholders and handles disputes	opus, sonnet, haiku
statements	summarizes statements overnight, nobody waiting	sonnet, haiku, flash-lite
fraud screening	scores transactions for fraud	opus only

Each is a project, and each project decides which models it may use at all.

Eleven application teams share these projects, on four release trains, so anything that needs every team to ship at the same time does not happen. That constraint is why the scenarios below are written around edits an operator makes alone, and why the ones that do need an application change need only a single header.

How each scenario is written

Every scenario below has the same four parts.

Part	What it gives
The scenario	The problem in full, in the terms an operator would use to describe it
The setup	The catalog and Policies configuration, plus anything an application team has to send
What happens	The resulting behavior, including what is left untouched
What to watch for	The failure modes, and the limits of the approach

Match the scenario to the outcome

Scenario	Outcome
1: A model version changes and its name is hardcoded in application code	A model alias absorbs the change, so a version bump is one catalog edit rather than a release in every service
2: Overnight batch work is billed at interactive rates	The nightly job is served by a cheaper model, and nothing else in the project moves
3: Customer tiers should get different models	Each request is served according to the tier the application declares
4: One agent run mixes planning and execution steps	A single run plans on a large model and executes on a cheap one, with nothing for the user to configure
5: A new model needs real traffic before it is trusted	The beta cohort that opted in gets the new model, and nobody else does
6: A model is being retired and its traffic has to move in stages	Batch traffic proves the replacement first, then the default follows it
7: A policy has to be reviewed before it takes effect	Monitor records what each rule would have matched, while every request still goes where it goes today
8: One request cost far more than its neighbors	The model that served it, the rule that chose it, and the attributes behind the decision are all on the record
9: A provider is degraded and one application has to move now	One API key is pinned to a healthy model, and nothing else moves
10: A group should not have the expensive model at all	The group's project carries only the models they should have, whatever tool they use

Things a routing rule cannot do are listed at the end, with what to do instead.

---

1: A model version changes and its name is hardcoded in application code

The scenario. A provider is retiring the model that applications name directly in their source. Eleven teams shipping a release because of a provider's calendar is not an acceptable answer.

The setup. Publish the name applications should use (claude-opus, or fast, or whatever suits the organization) and point it at the model that should serve it. No routing rules are involved.

When the provider ships a new version or retires the old one, repoint the name. That is the whole operation.

What happens. Applications keep sending the same name and are served whatever the alias currently points at. There is no redeploy, and the change takes effect without a separate publish step. The response still names the model that actually served the request, so a caller can always see what the name resolved to.

What to watch for. Three things.

A repoint moves every caller at once. That is what makes an alias right for a version bump and wrong for giving one group cheaper models: there is no way to make one name mean different things for different people. For that, see 10: a group without the expensive model.

Pointing a name at a model does not grant access to it. The project still has to be allowed to use the destination model. An alias is a nickname, not a key.

A name that already means something must not be reused. Publishing an alias over a name a real model already uses changes what every existing caller of that name receives, without anything about their requests having changed.

2: Overnight batch work is billed at interactive rates

The scenario. Statement summarization runs overnight with nobody waiting on it, and it is billed at the same rate as a request from a customer sitting in front of a screen.

The setup

1. In the catalog, create a request attribute workload with the allowed values batch and interactive.

2. In Policies, create attribute-based routing on the statements project, with these routing rules:

   	Condition	Serve with
   Rule 1	metadata.workload is batch	haiku
   Default	no condition	sonnet, the model in use today

3. Ask the statements team to add one header to the calls their nightly job makes. One line, in one place:

   X-Tars-Metadata: workload=batch

What happens. The nightly job is served by haiku. Everything else in the project is untouched, including anything not sending the header yet.

The call sends	Served by	Why
workload=batch	haiku	Rule 1 matches
workload=interactive	sonnet	No rule matches, so the default applies
no header at all	sonnet	There is nothing to match on, so the default applies
workload=btch	sonnet	The value is not one the catalog allows, so no rule can match it
priority=high	sonnet	priority is not an attribute in the catalog, so it can never match

What to watch for. The saving depends entirely on the attribute being sent, and until the statements team ships the header every request takes the default. A rule nobody is sending the attribute for and a rule that is working look the same from the policy screen: both show traffic on the default. 7: Reviewing the policy before it takes effect is how to tell the two apart.

3: Customer tiers should get different models

The scenario. Wealth clients should be served by the best model available. Everyone else does not need it.

The setup

1. In the catalog, create a request attribute plan with the values trial, retail and wealth.

2. In Policies, create attribute-based routing on customer chat, most valuable rule first:

   	Condition	Serve with
   Rule 1	metadata.plan is wealth	opus
   Rule 2	metadata.plan is retail	sonnet
   Default	no condition	haiku

3. The chat backend already knows the tier from the session it is holding, so it sends it:

   X-Tars-Metadata: plan=wealth

What happens. Each request is served according to the tier the application declared. A request carrying no tier gets the default, haiku.

What to watch for. Order matters, and the first matching rule wins. The specific rule goes above the general one. And this steers cost between the organization's own applications: it is not a way to stop anyone reaching the expensive model. See things a routing rule cannot do.

4: One agent run mixes planning and execution steps

The scenario. One agent run makes a plan and then takes forty small steps. The plan is worth a large model; the steps are not. The person using the agent should not have to configure anything.

The setup

1. In the catalog, create a request attribute task with the values in use, planning and execution here.

2. In Policies, create attribute-based routing on the project, with these routing rules:

   	Condition	Serve with
   Rule 1	metadata.task is planning	opus
   Rule 2	metadata.task is execution	haiku
   Default	no condition	sonnet

3. Have the agent framework send the header on each call it makes, carrying the value for that step:

   X-Tars-Metadata: task=planning

What happens. One run uses two models and the person running it configures nothing, because the agent declares what kind of step each call is.

What to watch for. The agent framework declares what it is doing, rather than something guessing on its behalf. That is the point: the rule can be read, watched before it takes effect, and relied on to give the same answer every time. It also means the approach reaches only the code that can be changed. A tool outside the organization's control cannot send the header, and 10: restricting the group's project is the option for that case.

5: A new model needs real traffic before it is trusted

The scenario. The people who should see the new model first are the beta users who opted in and know it may be rough. A percentage split would reach random production customers instead.

The setup

1. In the catalog, create a request attribute cohort with the values beta and ga.

2. Have the mobile backend send X-Tars-Metadata: cohort=beta for enrolled users only, and nothing for everyone else.

3. In Policies, create attribute-based routing with these routing rules:

   	Condition	Serve with
   Rule 1	metadata.cohort is beta	the model under evaluation
   Default	no condition	the current model

What happens. The rule follows the cohort exactly: all of beta, none of anyone else, however much the cohort grows. A percentage split cannot express that, because it selects requests by chance rather than by who sent them.

What to watch for. Nothing judges whether the new model is any good. What the rule produces is a clean set of records to compare, separated by cohort in the usage data; the comparison itself is a human job. Control request log data covers what is captured per request.

6: A model is being retired and its traffic has to move in stages

The scenario. A provider is retiring a model. The safest order is to move the traffic nobody is waiting on first, prove the replacement on it, then move the rest.

The setup. Two edits, weeks apart, and no application changes.

1. With workload already in use from 2: the overnight batch scenario, add one rule above the default and leave the default alone:

   	Condition	Serve with
   Rule 1	metadata.workload is batch	the replacement
   Default	no condition	the retiring model

2. Watch. Batch is the proof run: high volume, nobody waiting.

3. Once the replacement has proved itself, change the default to the replacement and delete the rule. The retiring model then receives nothing and can be removed from the project.

What happens. Overnight traffic moves first, customer-facing traffic stays where it is, and the records tell the two apart, so the comparison is available.

What to watch for. The stages available are the attributes that exist. "Move 10% of the interactive traffic too" is not something a rule can say: that is a traffic split, a different policy.

7: A policy has to be reviewed before it takes effect

The scenario. A change advisory board wants to see what a rule will do before it does it.

The setup. Save the policy in monitor rather than switching it on.

What happens. The rules are evaluated against real traffic and the results recorded, and every request still goes exactly where it goes today. The policy screen shows how many requests each rule would have matched over a recent window, and which model they would have moved to, next to the button that turns the policy on.

Monitor is also the cheapest way to find out whether an application team has actually shipped the header, which is the failure mode behind 2: overnight batch work and most of the scenarios above it.

What to watch for. Two things.

Monitor proves matching, not quality. The other model is never actually called, so nothing here says whether its answers would have been good enough.

Match counts climbing means a team shipped the header, not that they got the values right. A team sending a value the catalog does not allow shows up as no match at all, which is indistinguishable from a team that has not started.

8: One request cost far more than its neighbors

The scenario. Finance is asking why one request cost ten times its neighbors.

The setup. None. What follows is available on every request by default.

What happens. Every response identifies the model that actually served the request, so the caller can tell what served a call without anything having to be opened.

For anything beyond that, open the request in the Admin Console. The record shows the routing that applied, the rule that matched, the attributes the request was carrying, and what the request cost. From there the policy can be opened to see where in the rule order the match happened.

What to watch for. Every value shows where it came from, so a decision that rested on something the calling application sent is visible as such. That distinction decides who fixes it: a wrong attribute value is a change for the application team, not a policy edit.

9: A provider is degraded and one application has to move now

The scenario. A provider is degraded at 2am, and customer chat needs to be somewhere healthy without eleven application teams being paged.

The setup. Create attribute-based routing on that one API key, with the healthy model as its default. No rules are needed; the default serves every request from that key.

What happens. That key uses the healthy model. Nothing else moves, because the policy is attached to the key rather than the project. Deleting the policy once the incident is over returns the key to whatever applied to it before.

What to watch for. Two things.

This is manual. A rule whose model has gone down does not match, so evaluation continues at the next rule and reaches the policy's default only when every rule has been tried. Both the later rules and the default may name the degraded model, so neither is assumed to cover an outage. Automatic movement on provider health is a fallback policy, not a routing rule. See Improve resilience with fallbacks.

A change takes a short time to reach the data planes. It is not instant. A request served the old way in the first moments after the edit is not evidence that the policy failed.

10: A group should not have the expensive model at all

The scenario. A group (designers writing code, say, or anyone outside engineering) does not need the most expensive model. The choice should be made on their behalf rather than asked of them, and the tool they use may not be one that can be changed.

The setup. This one is not a routing rule at all. It is the project.

1. Create a project for that group. See Create a project.
2. Assign it only the models the group should have.
3. Issue their API keys in that project. See Issue a project API key.

What happens. A project can only route the models it was assigned. A request for a model outside that set has nowhere to go, whatever the caller asks for and whatever tool it is using. Nobody in the group configures anything, or is asked to choose.

What to watch for. Four things, and the first decides whether the approach fits at all.

This refuses; it does not quietly downgrade. A request for a model the project does not have fails, because there is no route for it. Where the group uses a tool that names an expensive model and the intent was to have a cheap model serve that request instead, that is not what this does, and there is no other way to do it either. The choice today is between the group's tool working on the models it was given, and the group's tool failing when it asks for something else.

The unit is the project, not the group. If those people have to share a project with engineers who do need the expensive model, this does not work: the project's model set is shared by everyone in it.

Routing policies cannot do this. A rule sends a request to a model; it never takes a model away. Steering with a rule instead leaves a caller that names the expensive model directly still getting it.

Adding a model to the project undoes it. The project's model list is the control, so it is the thing to review.

---

Things a routing rule cannot do

Stop a group of people using an expensive model. Routing decides where a request goes; it never changes what a key is allowed to use. A team that asks for opus by name gets opus, whatever the rules say. What restricts a model is the project's model set, which is 10: a group without the expensive model. The limit there is that the unit is the project, so a group that needs its own model set needs its own project.

Pick a model by how hard the request looks. There is no automatic routing on reasoning complexity, or on what a request says. Every rule matches on attributes the calling application sent. Where the people to be steered use a tool outside the organization's control, there is nowhere to attach an attribute at all, and 10: a project without the expensive model is the answer rather than a rule.

Enforce anything. Everything a rule reads is supplied by the calling application. A rule such as "regulated data goes to the EU-hosted model" works perfectly while applications send the attribute, and the moment one does not (a new service, an old caller, a bug) that request goes to the default instead, and nothing reports a problem, because a missing attribute legitimately means the rule does not apply.

A rule of that kind is therefore a cost or quality preference, never a control. If a request must not reach a model, the project must not offer that model. Then it does not matter what any caller sends.

Route on anything the application does not declare. There is no routing on how big a request is, how many tokens it will use, which path it came in on, or what it says. Only on the attributes the application sends.

Require an application to send attributes. A caller that sends none is served the default.

Send a percentage of traffic. Rules match on attributes, not on chance. Percentages are a traffic split.

Switch a model because a budget ran out. A rule reads what the caller declared, and a spend limit is not something the caller declares. Acting at a limit is a budget enforcement action: see Enforce a budget cap.

Route by time of day. There is no way to express "batch work only between midnight and six". A rule is either on or off.

Route by team or by person. A policy is targeted at the organization, a project, a tag selector on API keys, or a single key, never at who owns the key. Traffic that has to be routed per team is routed by tagging that team's keys and writing the policy against the tag, or by setting the policy on the projects that team works in. See Scope and resolution.

Report that a rule has stopped working. A rule that never matched and a rule that stopped matching both look like traffic on the default, and nothing raises it. The catalog shows whether a name has been seen recently, and checking it is something someone has to remember to do.

Show what routing saved. Every request records what it cost, and nothing totals those costs per policy. Working out what a change saved means comparing spend before and after by hand, in Usage Analytics.

Keep one conversation on one model. Each request is routed on its own. There is no memory of what served the previous one.

Reach a model in another cluster, or another provider's endpoint. Routing chooses among the models a project already has. Making a model reachable from somewhere else is a provisioning and networking job: see Manage models and providers.

Apply two sets of routing rules at once. Exactly one attribute-based routing policy applies to a key, because two lists of rules cannot be merged. An organization-scoped policy always wins and binds; below it the narrower target takes over rather than combining, in the order key, tag selector, project. Check what a key resolves to before adding a second policy: see Precedence.

---

Where to go next

Routing policy reference

The objects behind every scenario on this page: attributes and their source, evaluation order, scope precedence, modes, and the decision record.

Improve resilience with fallbacks

The mechanism that moves a request when the primary model fails, rather than when the caller declares something. Available today.

Reduce cost with traffic splitting

Percentages across backends, for the cases a routing rule cannot express. Available today.

Create a project

The model set that bounds every routing decision, and the only place a model is genuinely restricted.

Know what every app and project costs

Read spend along the same attributes the routing rules match on.