Policies
A policy is an admin-managed rule that decides what happens to a model request. One policy carries a type, a scope that chooses which API keys it governs, a configuration belonging to that type, and a mode. Five controls are built on this one shape: budgets, availability fallbacks, traffic splits, rate limits, and guardrails. Setting a rule once and having it apply to a whole class of keys, rather than configuring the same control on hundreds of keys by hand, is the reason the model exists.
The guides for each individual control cover how it is configured; the machinery they share is the subject here.
A policy attaches a rule to a set of API keys
Every policy has four parts.
| Part | What it holds |
|---|---|
| Type | Which control it is: budget, availability fallback, traffic split, rate limit, or guardrail |
| Scope | How it chooses the API keys it governs, from the six scopes below |
| Configuration | The settings belonging to that type: an amount, a chain of models, a set of weights, a token cap, a set of checks |
| Mode | Monitor or active |
The API key becomes the subject a policy attaches to rather than the place configuration lives. Per-key configuration still exists and behaves as it always has: a key's own settings are simply a policy scoped to that one key, edited on the key's own page. What the model adds is the ability to write one rule that reaches a class of keys.
Five policy types, each with its own trigger
Every policy pairs a trigger, the event that fires it, with an action, the consequence it applies.
| Type | Fires when | Action |
|---|---|---|
| Budget | a budget is exhausted | block, or degrade to a cheaper model |
| Availability fallback | the primary model fails, on error or timeout | move to the next model in the chain |
| Traffic split | every request | route by weight |
| Rate limit | a token cap would be exceeded | block, or degrade |
| Guardrail | a content or safety check matches | block, or redact |
Moving to a cheaper model is one action fired by two different triggers. A model failure drives it for resilience; a spend threshold drives it for graceful degradation. The mechanism is identical, which is why a budget's degrade action is configured on the budget, where the threshold that fires it lives, rather than on the availability fallback surface. Putting it there would separate the trigger from its action and imply that a model failure and an exhausted budget are the same event.
New triggers slot in under the same shape, so a new control type adds a policy type rather than a new kind of object.
Measuring and enforcing are separate jobs
Not every trigger originates on the request path. A guardrail match and a model failure both happen while a request is in flight, but a threshold can also be crossed by an accumulating meter that no single request is responsible for. Such a threshold is a cross-cutting trigger: it originates outside the request path and reaches into it.
Budgets are the case where that separation matters most. A budget measures: it tracks spend, attributes it to a subject, and holds the threshold, and it never stops a request whatever the meter reads. A policy enforces: it takes that threshold as its trigger and acts on the request path once the threshold is crossed. One surface owns the number and the report, the other owns the consequence, so the two never compete for the same job.
Budgets covers the type in full, and Enforce budget caps with policies works it through in scenarios.
Definitions live in the catalog, attachments live in policies
A control type contributes an entry to the catalog only when it has a reusable definition, something that can be named once and applied in several places.
| Control | Catalog entry | Why |
|---|---|---|
| Guardrail | Yes, the guardrail rule | A rule is a reusable definition, like a model. It is inert until a policy attaches it |
| Budget, rate limit, traffic split, availability fallback | No | An amount, a token cap, a set of weights, and an ordered chain are the policy's configuration, not something reusable that sits apart from it |
A definition on its own changes nothing. A guardrail rule that no policy attaches inspects no traffic. Attaching it to a project, a tag, or a key is the policy.
Which models a project may use at all is the catalog, not a policy. An organization that wants a model gone everywhere removes it from the catalog. An organization that wants that model's traffic sent elsewhere writes a routing policy.
Scope runs on two axes
Every policy type draws from one shared set of six scopes.
| Scope | Selects |
|---|---|
| Organization | every key in the organization |
| Project | every key in one project |
| Tag selector | every key carrying all of the named tag pairs |
| Key | one key |
| Team | every key owned by any member of the team |
| User | every key owned by one person |
The first four select keys by looking at the key itself. The last two select keys by looking at who owns the key, which is a different question rather than a narrower one. A team is neither bigger nor smaller than a project: it picks keys a different way, and the same key can be picked by both.
Three consequences follow:
- A team policy and a project policy can both cover a key, and neither one wins.
- A tag reaches keys sitting in several projects and owned by several people.
- An agent's spend counts against whoever created its key, because the key's owner is the only person there is to count it against.
Only key, project, and organization sit inside one another. That is the one place where calling a scope "more specific" means anything at all.
Not every type offers every scope
| Type | Scopes offered | Why the rest are missing |
|---|---|---|
| Budget | all six | |
| Guardrail | organization, project, tag, key | Selecting keys by owner has so far only been wanted for spend |
| Rate limit | organization, project, tag, key | As above |
| Availability fallback, traffic split | organization, project, tag, key | A routing policy names models, and models are enabled per project, so a scope whose keys span projects cannot name one |
The routing constraint is permanent rather than a phase, and it is the same constraint that decides where a budget can degrade rather than block: an action that names a model is only available where a single project can be identified. Routing's four scopes are set out in the Routing policy reference.
A narrower policy can never loosen a broader one
Where two policies of the same type cover one key, how they combine depends on what the policy carries.
| Type | How two combine |
|---|---|
| Guardrail | Both run, and nothing is removed. The same check arriving twice with different actions takes the stricter one |
| Spend cap | Each is checked against its own spend, and whichever is exhausted first stops the request |
| Rate limit | The lower ceiling applies, per dimension |
| Routing | Exactly one applies, since two orderings cannot be merged |
Two corrections this rules out:
Spend caps are not a minimum. A $1,000 team pool and a $500 agent budget do not resolve to "$500 for this key". If the team's other members have already spent $800, the agent stops at $200 of its own spend. Only rate limits resolve to a minimum, because a rate has no accumulator to run down. Overlapping budgets all apply, and the first exhausted wins sets out the accrual rules in full, including the one exception.
A narrower policy cannot exempt a key from a broader one. Raising a key's own budget above a team pool that covers it changes nothing, because the pool still stops the request. A key escapes a policy only by falling outside its scope.
At most one policy of a given type is active per target. Editing replaces it rather than adding a second.
Resolution happens centrally, and moves on its own
The effective configuration for each key is computed in the management plane and distributed to the data plane, which authenticates the key and enforces what it was given.
Most scopes select their keys indirectly, so the set a policy covers changes without anyone editing the policy. A team gains a member. A key is minted, retagged, or moved. A project is created and falls under the organization's policies.
- The effective policy on a key is recomputed when its membership changes, not only when a policy is edited.
- Coverage counts shown on a policy reflect what it matches now, not what it matched when it was created.
- Where such a change leaves a policy unable to do its job, such as a degrade action reaching a project with no cheaper model configured, that project is shown as a gap on the policy rather than discovered at request time.
Monitor is a mode, not an action
Every policy is held in one of two modes, and the mode means the same thing for every type.
| Mode | What happens |
|---|---|
| Monitor | The policy records what it would have done. Nothing about the request changes |
| Active | The policy applies its action |
Monitor is not a third choice alongside block, degrade, and redact. It is the state a policy is held in before any of them takes effect, and it reports how many requests it would have affected over a period. A newly created organization-scoped policy starts in monitor.
Coverage, no-effect, and blast radius appear before saving
Three checks run while a policy is being created, all aimed at surprises that would otherwise surface as a blocked request nobody can explain.
- Coverage. The scope step reports how many keys the chosen scope currently matches, and which projects they sit in, before the policy is created.
- No effect. A policy that would never apply, because a tighter one already covers the same keys, is named as such before it is saved, along with the policy responsible and its value. It is a warning rather than an error: an admin may be setting a limit ahead of a change above it.
- Blast radius. Creating or editing an organization-scoped policy reports how many keys it will change, in how many projects, and how many currently get a different outcome. Organization-scoped changes are confirmed explicitly rather than saved on a single click.
Every key shows the policies that resolve onto it
The effective policy view is where "why did that request behave that way" is answered, because the key is where overlapping scopes are actually reconciled.
For any key it shows every policy that applies, grouped by type, with what each resolves to and which policy produced it. Budgets show what is left in each rather than only the limit, because which one will stop the key depends on spend by other members and a single resolved number would be wrong. A policy running in monitor is shown differently from one that is enforcing, and a policy that was replaced by an override is shown with the reason.
The same question asked of the governed object rather than the key is answered in the same place: a tag lists every policy targeting it, and a project lists every policy targeting it plus what it inherits, read-only, with a link to where each is managed.
Terms introduced on this page
Terms below appear only on this page.
Also used here and defined in the Glossary, with the definition on hover: policy, scope.
| Term | Definition |
|---|---|
| Type | Which of the five controls a policy is |
| Trigger | The event that fires a policy's action |
| Action | The consequence applied when the trigger fires: block, degrade, or redact |
| Mode | Monitor or active |
| Effective policy | The resolved outcome per type on one key, and which policy produced it |
| Catalog definition | A reusable object a policy attaches, such as a guardrail rule. Inert until attached |
Where to go next
Budgets
The budget type: amount, subject, and action, and where a budget stops being a guarantee.
Guardrails
The guardrail type, and how inline inspection works at the gateway.
Routing policy reference
The routing types: attributes, rules, evaluation order, and the decision record.
Stop runaway workloads
The rate limit type, enforced inline at the gateway today.