Choose the right cost control for each workload
Enterprise Tier
AI traffic has a habit of growing without anybody noticing until the invoice arrives. A research team starts a benchmark, leaves it running over a weekend, and bills three weeks of normal spend in two days. A new agent framework gets wired into a production path and starts making ten calls where a human would have made one. A leaked credential lands in a public repository and an attacker spends a few hours mining it before the security team notices. Each of these scenarios is preventable, but only with the right control in the right place, and the controls are not interchangeable.
Agent Router's cost story is layered. There is no single "budgets" screen that owns the entire mechanism end-to-end; instead, it is the combination of three things: rate limits, which enforce ceilings on traffic inline at the gateway and are the only control that rejects requests; budgets, which track cumulative spend per user or team over a period and raise alerts as a ceiling is approached; and Usage Analytics, which is where spending against those ceilings becomes visible. The API-key-per-purpose convention established in Onboard developers and issue keys is what makes all three precise rather than blurry. This page maps each control to the situation it fits, so the guides that follow are applied to the right problem.
Persona: Platform operator working in the Admin Dashboard, often in partnership with the developer teams that own specific API keys and the finance owners who set spend ceilings.
Estimated time: 10 minutes to read; the linked guides carry their own estimates.
Match the situation to the control
| Situation | Control that fits |
|---|---|
| A production application's spend should not exceed a known monthly ceiling | A budget, with monitoring on a cadence |
| A research or evaluation team should not run away with the bill | A budget to track spend and alert plus aggressive rate limits on dedicated keys for the inline hard stop |
| Each team's spend should be attributable for chargeback | Per-team keys with chargeback statements per cycle |
| Spend must be booked against an app or project that does not map to a user or team | Caller-set attribution tags |
| Overall spend across all consumers needs to stay under a contractual ceiling | Sum of per-key limits, plus alerting on the aggregate |
| A leaked credential needs to be bounded in damage even before it is revoked | A rate limit at the consumer's known traffic level, then containment |
For Bring Your Own Key (BYOK) consumers (developers who present their own provider credentials, as described in Use your own provider credentials), spend is attributed to the BYOK provider account directly, not to Agent Router's own usage records. The budgeting concerns there are a matter for the team that owns the BYOK account; the controls on this page cover centrally managed traffic.
Hard ceilings or soft monitoring
There are two ways to use these mechanisms, and the right choice depends on the consumer. The distinction is not a setting on a single control: an inline hard stop comes from a rate limit, while a budget always operates as soft monitoring because it tracks spend and alerts rather than rejecting requests.
| Strategy | What it does | Control that provides it | When to use it |
|---|---|---|---|
| Hard ceilings | Limits enforced inline; requests beyond the ceiling are rejected | Rate limits | Experimental or CI keys where a runaway should be cut off automatically; any path where unbounded throughput is unacceptable |
| Soft monitoring | Spend tracked against a ceiling, with Usage Analytics watched actively and alerts wired to someone who acts | Budgets and Usage Analytics | Production workloads where a false-positive block would be worse than occasional overspend; accountability and forecasting against an agreed number |
Most deployments end up using a mix. Production keys for critical applications rely on budgets for soft monitoring, because a sudden block cascade is the wrong failure mode for them. Research, evaluation, and CI keys add rate limits for a hard ceiling, because a runaway there is more annoying than a rejection.
Rate limit or budget
The two ceilings are easy to conflate, but they answer different questions and operate on different axes.
| Budget | Rate limit | |
|---|---|---|
| Controls | Total spend over a period, tracked with alerts | Request or token throughput over a short window, flow control |
| Unit | Money or cumulative tokens, per user or team | Tokens, requests, or parallel calls per window, per key, user, group, or model |
| Question answered | "How much has this person or team spent this month, and am I near the ceiling?" | "How fast may this credential consume right now?" |
| Enforcement | Alert only; requests continue to be served | Inline at the gateway; requests over the ceiling are rejected |
| Failure it surfaces | A cumulative overrun, flagged as spend approaches the ceiling | A burst or runaway happening in real time |
A rate limit set generously can still allow a month-long drift that quietly exhausts a budget; a budget can be intact while a single misconfigured key bursts hard enough to need a rate limit. The two are configured to complement each other: the rate limit blocks the fast runaway at the key, and the budget flags the slow accumulation at the user or team so someone can act on it. Most workloads warrant both.
The cost-control guides
Catch spend drift before the invoice does
The regular usage review that surfaces slow cost creep while it is still cheap.
Keep each team's spend inside an agreed ceiling
Turn a number agreed with finance into a budget that tracks spend and alerts as the ceiling nears.
See budget exhaustion coming in time to act
Read budget burn, headroom, and the members driving it, in time to intervene.
Raise a spend ceiling without interrupting delivery
Raise the ceiling for a team that legitimately outgrew it, and keep the change auditable.
Bill AI spend back to the teams that incur it
Chargeback and showback statements finance can load into its own systems.
Know what every app and project actually costs
Caller-set tags add the app and project axis the user and team dimensions cannot express.
Stop runaway workloads before they burn the budget
Token, request, and concurrency limits that wall off retry storms and fan-out bursts.
Get alerted to cost spikes as they happen
Telemetry export into the observability stack, with alert rules instead of dashboard-watching.
Contain a leaked key before it drains the budget
The incident response: bound the damage, revoke the credential, pause the workload.
Configuring request logs
The logging modes behind the usage data every control on this page depends on.