Level 3: Profile agent efficiency
An agent's bill is a behavior, not a number: how much context it re-reads, how often its requests hit the cache, how many model calls one task really takes. The final level of the usage analytics onboarding path turns one agent's day of traffic into an efficiency scorecard in twenty timed minutes. The scorecard is a baseline, and baselines exist to be beaten; that is what makes this level the one that keeps getting replayed.
Persona: Developer or engineering leader working in the Developer Console.
Estimated time: 20 minutes, played against a timer.
Progression: Level 3 of 3 in the usage analytics onboarding path. Requires Level 2.
Outcomes
By the end of this level:
- One agent's traffic has been isolated and profiled: input/output token split, cached-request share, and cost per task.
- A baseline efficiency scorecard has been filled in.
- One number on the scorecard has been chosen as the target to beat on the next run.
Prerequisites
- Levels 1 and 2 of this path completed.
- A coding agent or application running on its own API key, as set up in Coding Agents and Tools, with at least one working session of traffic behind it.
Step 1 (minutes 0--4): isolate the agent
- Start the timer.
- Open Monitoring → Usage and filter by the agent's key over the window of its last working session.
- Note the totals: requests and spend for the session.
Time check, minute 4: the agent's session is isolated, with its request count and total spend visible.
Step 2 (minutes 4--9): read the token split
- From the filtered view, compare total input tokens against total output tokens.
- Open two or three individual requests in Request Logs and confirm the pattern in their token breakdowns.
Coding agents typically send far more input than they receive as output, because context is resent on every call. The input-to-output ratio is the first efficiency number: a rising ratio means the agent is re-reading more than it is producing.
Time check, minute 9: the session's input-to-output token ratio is written down.
Step 3 (minutes 9--13): find the cached share
- In the request detail panels, look at the token breakdown for input tokens served from cache, where the provider reports them.
- Estimate the share of the session's requests that benefited from caching.
Cached input tokens are billed at a fraction of the list price, so the cached share is leverage: the same agent behavior at a higher cache-hit rate is the same work for less money.
Time check, minute 13: the cached-request share is estimated, even roughly.
Step 4 (minutes 13--17): compute cost per task
- Count the tasks the agent actually completed in the session (pull requests, tickets, answered questions; whatever the unit of work is).
- Divide the session's total spend by that count.
Cost per task is the number that travels: unlike tokens or requests, it can be compared across agents, models, and weeks, and it is the number a budget discussion eventually lands on.
Time check, minute 17: cost per task is computed for the session.
Step 5 (minutes 17--20): fill the scorecard
Record the baseline:
| Measure | Baseline (this session) | Next run | Best |
|---|---|---|---|
| Requests | |||
| Input / output token ratio | |||
| Cached-request share | |||
| Cost per task | |||
| Total spend |
Pick one number to beat on the next run and write down the intended lever: a tighter system prompt, better cache reuse, a cheaper model on a routing rule for the easy calls.
Time check, minute 20: the scorecard's baseline column is full and a target is chosen.
Level complete
- The agent's session isolated and totalled.
- Input-to-output token ratio recorded.
- Cached-request share estimated.
- Cost per task computed.
- Scorecard baseline filled and a number chosen to beat.
Head-to-head: run the same profile for a second agent or teammate and put the two scorecards side by side; the differences are where the interesting conversations start. Cache detective: find the most-repeated prompt prefix behind the cached share and name which part of the agent's context makes it cacheable. Beat the baseline: improve one scorecard number on the very next session and record what changed.
Where to go next