Fast-track evaluation: from first login to organization-wide cost analysis
A fast-track product evaluation built around the jobs an engineering leader does in the first week: get in, prove the value on personal traffic the same day, then bring the team. It starts from a literal first login with nothing configured and ends with a populated cost breakdown by team, model, and app, with no prior setup assumed. Every job takes minutes, ends with something concrete on screen, and builds on the state the previous jobs created. The graded sections inherit that state, so this page runs first.
Introduction
This page is the recommended entry point of the Agent Router product evaluation. It compresses the first days of real product usage into three time-boxed milestones, so an evaluator experiences Tetrate Agent Router the way an organization would adopt it: one person gets access, proves the value on their own traffic, and then brings colleagues on board. It replaces the older approach of trialing each capability independently, where every section started from scratch and none of the results connected.
The intended reader is the person running the evaluation, typically an engineering leader or platform owner deciding whether Agent Router earns a wider rollout. No prior familiarity with the product is assumed; every job links to the guide that explains the concepts involved.
A completed fast track produces three things:
- A working environment: an onboarded organization with SSO, provider credentials, live traffic, budgets, and a first group of real users. The graded sections build on this state instead of recreating it; the carry-over table lists the scenarios it pre-completes.
- Evidence: a ticked verification for every job, plus the measured time each milestone took.
- The numbers for the rollout conversation: a shadow bill, an agent efficiency baseline, and spend broken down by team, model, and app.
Why the evaluation runs in this order
- Each job's output is the next job's input. Independent feature trials repeat setup work and never show how capabilities compound. Here the key created on Day 1 is the key the first budget covers, and the traffic it carries is the traffic the Day 2 comparisons analyze.
- Personal value comes before anyone else's time. The order mirrors real adoption: the evaluator convinces themselves on Day 1, on traffic that already exists, before asking teammates to invest time on Day 2.
- Passthrough mode keeps the stakes low. The early jobs route existing subscription traffic through the gateway without changing how it is billed, so the evaluation observes real work rather than a synthetic demo, and can be stopped at any point without unwinding anything.
- The clock is part of the test. Milestones are time-boxed and elapsed time is recorded because time-to-value is one of the properties being evaluated. A gateway that takes weeks to show value has failed part of the test regardless of how well individual features grade.
- It is ungraded on purpose. The fast track exists to build momentum and gather evidence. Rigor comes afterwards, in the graded sections, which score each capability against the grading rubric.
How to run it
- Work through the three milestones in order:
- Each is time-boxed, and each starts from the state the previous one created.
- Every job ends with a verification: a specific thing that should be visible on screen. Tick it off when it is.
- Record the elapsed active time per milestone; time-to-value is itself a result being evaluated.
How to find information
- The knowledge and information how to perform a job is provided as links to the relevant documentation pages. Note that the docs are not step-by-step guides, but will provide adequate information and details to complete each step.
- It is recommended to open each documentation page in a new browser tab, to prevent you from having to navigate back and forward between documentation pages.
For your convenience, this page can be downloaded as PDF or printed directly. Site navigation is omitted, the scoring table keeps its rows intact with the header repeated on every page, and the Grade and Notes columns leave room for handwriting.
Day 0: Initial access and onboarding
Access gates everything that follows, and it is also the first impression the product makes, which is why it is measured rather than waved through. SSO is configured up front, not because the evaluator needs it on day one, but because it turns the Day 2 invitations into a single link with no account administration.
Goal: gain access for the evaluator and the organization.
Time: 30 minutes of active work, plus IdP turnaround.
Start state: nothing but the evaluation environment and the IdP details.
End state: an onboarded organization and SSO sign-in into both applications.
It is NOT recommended to use corporate iDPs (identity providers) for testing or evaluation purposes. Unless sanctioned and supported by your organization, it is highly recommended to use an external Identity Provider, such as Auth0, for this evaluation fasttrack. The SSO configuration guide contains a detailed description of using Auth0 for SSO.
| ID | Job to be done | Guide | Done | Time / notes |
|---|---|---|---|---|
| D0.1 | Complete the onboarding wizard: organization details, providers, and models (self-hosted paths also download the data plane credential here). Verify: the Admin Dashboard is populated with the chosen providers and models. | Onboarding guide | ||
| D0.2 | Configure single sign-on against the corporate IdP (if applicable) or Auth0 (alternative), so the rest of the organization can log in later with zero extra setup. Verify: sign-in works with corporate credentials. | Configure single sign-on | ||
| D0.3 | Open both applications under the SSO identity: the Console (developer view) and the Admin Dashboard (operator view). Verify: both applications load and show the organization. | Where to begin | ||
| D0.4 | (optional) Send one prompt in the Playground, so even the access day ends with a model answering. Verify: a response arrives through the gateway. | Test prompts in the Playground |
Day 1: Set up for personal traffic
Nothing convinces an engineer like their own traffic. This milestone routes work that already happens (a personal subscription, a working agent) through the gateway, then reads the results: what that traffic would cost at API list prices, how efficiently the agent spends tokens, and what a budget looks like against real usage. Billing does not change while any of this happens; the gateway only observes.
Goal: make Agent Router useful for one person before asking anyone else to try it.
Time: about one hour of active work. Target: first routed request within 30 minutes.
Start state: the SSO access from Day 0.
End state: a personal key carrying live traffic, a shadow bill with a real number on it, an agent efficiency baseline, and a first budget.
| ID | Job to be done | Guide | Done | Time / notes |
|---|---|---|---|---|
| D1.1 | Create an API key, then connect a personal Claude subscription in passthrough mode (BYOK), so existing traffic flows through the gateway on the new key without changing how it is billed. Verify: a subscription-billed request routes through the gateway. | Make an API call, Use your own provider credentials | ||
| D1.2 | Point one working agent at the gateway on the new key (Claude Code, Cursor, or any OpenAI-compatible client). Verify: the agent completes a real task through the gateway. | Coding agents and tools | ||
| D1.3 | Work through the first usage analytics onboarding guide: send a burst, find it in the logs, and explain one request's cost, tokens, and latency. Verify: the guide's closing checklist is complete and the elapsed time is recorded. | First insights in ten minutes | ||
| D1.4 | Total the shadow bill: what the day's subscription traffic would have cost at API list prices, projected to a month. Verify: "this subscription is worth $X per month" is a sentence with a real number in it. | Read the shadow bill | ||
| D1.5 | Profile the agent: input/output token split, cached-request share, and cost per task. Verify: the efficiency scorecard is filled in and one number is chosen to improve. | Profile agent efficiency | ||
| D1.6 | Create a hello-world budget: a small personal cap in warn mode covering the new API key. Verify: the budget meter shows used, remaining, and days left after the day's traffic. | Keep team spend inside a ceiling, See budget exhaustion coming |
Day 2: Set up for teams
Usage insights become decision material only when there is more than one user to compare. This milestone repeats the Day 1 setup for a handful of the organization's heaviest AI users and turns personal proof into team cost analysis: who spends what, on which models, for which applications. That comparison is the evidence a rollout decision rests on.
Goal: turn one convinced user into 2 to 5, and personal insights into team cost analysis.
Time: spread over the week, about 30 minutes of active work per teammate.
Start state: one convinced user with traffic and a budget.
End state: 2 to 5 active users, teams with budgets drawing down, and spend comparable by team, model, and app.
| ID | Job to be done | Guide | Done | Time / notes |
|---|---|---|---|---|
| D2.1 | Invite 2 to 5 of the organization's heaviest AI users and have them sign in through SSO. Verify: each new user reaches the Console without help. | Onboard developers and issue keys | ||
| D2.2 | Each teammate connects their own subscription in passthrough mode. Verify: subscription traffic from at least three users flows through the gateway. | Use your own provider credentials | ||
| D2.3 | Move the teammates' agents onto gateway API keys. Verify: agent traffic is attributable per key in the request logs. | Coding agents and tools | ||
| D2.4 | Create teams and give each a budget sized from the Day 1 baseline. Verify: each team's meter draws down as its members work. | Keep team spend inside a ceiling | ||
| D2.5 | Compare patterns across the group: spend by team, model, and app; cached-request share; subscription versus API cost. Verify: at least one notable difference between users or agents is found and explained. | Catch spend drift before the invoice does, Know what every app and project costs |
Wrap-up
Record the result of each milestone. Active time is hands-on work only, excluding the IdP turnaround on Day 0 and the calendar time Day 2 is spread across.
| Measure | Target | Actual |
|---|---|---|
| Day 0 active time (access and onboarding) | 30 minutes, plus IdP turnaround | |
| Day 1 active time (personal traffic) | About one hour | |
| Time to first routed request (Day 1) | 30 minutes | |
| Day 2 active time (teams) | About 30 minutes per teammate | |
| Jobs completed | 14 (+1 optional) | |
| Teammates routing traffic by end of Day 2 | 2 to 5 |
The fast track succeeds when, by the end of Day 2, spend is visible by team, model, and app for real traffic, and at least one budget meter is moving.
Where to go next
The fast track leaves behind a working environment; the graded sections put it under load.
- The evaluation overview holds the grading rubric, the summary scorecard, and the carry-over table that maps fast-track jobs to the graded scenarios they pre-complete.
- Cost control and budgets is the natural next section: it builds directly on the budgets and teams created here.