About cost and quota management
Enterprise Tier
AI spend is brought under control in three stages, in order: understand what is being spent, monitor that spend against a limit agreed with finance, then enforce that limit with a policy that stops calls once it is reached. A budget only ever measures, so a cap set without a baseline is a guess, a baseline nobody watches is a report, and a limit without a policy behind it stops nothing.
Persona: Platform operator working in the Admin Console, usually with the finance owner who sets the number and the engineering manager who owns the workload behind it.
Estimated time: 5 minutes to read. Each stage links to guides that carry their own estimates.
The three stages
Understand
Read what is being spent, by team, by model, by key, and over time. Nothing is limited at this stage. The output is a baseline and a set of owners.
Monitor
Set a budget: a daily, weekly, or monthly limit on a team, its members, or an API key, with a live meter and an alert as the limit is approached. In its default action a budget measures and reports without stopping anything.
Enforce
Choose an action that acts at the limit rather than reporting on it. A rate limit rejects traffic over a throughput ceiling. Since 0.3.0 a budget can hard-stop its subject once the limit is spent, or degrade one API key's traffic to a cheaper model.
Measuring is every budget's job; stopping is opt-in. A budget tracks spend against its limit and reports on it. What happens at the limit is the budget's action: Watch spend (the default) raises an alert and changes nothing about what the caller can do next, Hard stop refuses further requests until the period resets or the limit is raised, and Degrade gracefully, available on a single API key, reroutes them to a cheaper model instead. The arrangements are worked through in Enforce budget caps with policies.
Budget enforcement lags the spend by minutes, because spend is totaled from usage rollups before it reaches the gateway. Where an inline, immediate hard stop is needed, that job belongs to a rate limit, which is enforced at the gateway on the request itself.
Stage 1: understand what is being spent
Spend cannot be governed before it is attributable. The unit of attribution is the API key, so the per-purpose key convention in Onboard developers and issue keys is what keeps every later stage precise.
| Question | Where it is answered |
|---|---|
| What is the organization spending, and is it trending up | See where AI spend is going |
| Which team, person, or key is driving it | See where AI spend is going |
| How do two teams or two people compare side by side | Compare user and team spend in Analytics |
| What does each app or project cost, when neither maps to a team | Know what every app and project costs |
| What does each cost center owe for a closed period | Split the AI bill across teams |
Stage 2: monitor spend against a limit
A budget turns a number agreed with finance into a live meter. It has a scope (a whole team, each teammate, or one API key), a spend limit in US dollars, a period that resets daily at midnight UTC, weekly on Monday, or monthly at the start of the calendar month, and an action that decides what happens at the limit.
What a budget does:
- Tracks cumulative spend against the limit and shows burn, headroom, and days remaining.
- Raises an alert as the limit is approached and as it is crossed.
- Records who created it and when, so the number has an owner.
What a budget in its default action never does:
- Reject a request. Watch spend measures and alerts, and an exhausted budget leaves the caller free to keep calling. Stopping calls is Stage 3, chosen per budget as its action.
- Cover tokens or request counts. A budget is money. Throughput is a rate limit.
| Question | Where it is answered |
|---|---|
| How is a limit created, and what scopes can it take | Set a budget and track spend against it |
| How much headroom is left this period, and which members are consuming it | See budget exhaustion coming in time to act |
| How is a limit raised, paused, or retired once it no longer fits | Raise a spend ceiling without interrupting delivery |
| What catches drift that stays under every limit | Catch spend drift before the invoice does |
| How is spend alerted on without anyone watching a dashboard | Get alerted to cost spikes as they happen |
Stage 3: enforce a cap
Enforcement divides by what is capped. Throughput is capped by a rate limit, which rejects requests over its ceiling inline at the gateway, and is covered in Stop runaway workloads before they burn the budget. Spend is capped by a budget's enforcing action, available since the 0.3.0 release.
Spend is capped by the budget's action. A Hard stop budget refuses its subject once the limit is spent, until the period resets or the limit is raised; nothing needs to be manually re-enabled. A budget on a single API key can instead Degrade gracefully, in which case calls continue on a cheaper fallback model rather than stopping. Enforcement happens in the gateway, on live traffic, minutes behind the spend it acts on.
The scenarios, the interactions between overlapping budgets, and what a blocked caller sees are worked through in Enforce budget caps with policies.
Budgets and rate limits are not the same control
A budget measures money over its period and reports on it, minutes after the spend lands. A rate limit measures throughput over a short window and rejects what exceeds it, inline. Setting one where the other belongs is the usual reason a cost control never fires.
Most workloads warrant both: the rate limit walls off the fast runaway, and the budget flags the slow accumulation. The full comparison, and a per-workload decision between them, is in Choose the right cost control for each workload.
Start here
See where AI spend is going
Stage 1. The organization trend, the cut by team, and the cut by model.
Set a budget and track spend against it
Stage 2. Create a budget, read the meter, change the number later.
Enforce a budget cap
Stage 3. Block or fall back when the limit is reached.