# Evaluation Scenarios

> Hands-on evaluation scenarios for Tetrate Agent Router: graded, verifiable exercises that follow the installation and guide documentation end to end.

# Hands-On evaluation scenarios

  An Agent Router evaluation expressed as a sequence of hands-on scenarios. Each scenario follows one or more documentation guides, ends with a verifiable outcome, and receives a grade, so the evaluation produces evidence rather than impressions.

How to use these pages:

1. Run the [fast track](/agent-router-enterprise/fully-managed/fast-track/) first: it builds the organization, SSO, keys, traffic, teams, and budgets the graded sections assume, and pre-completes several scenarios (see the [carry-over table](#fast-track-carry-over)).
1. Then work through the graded sections in order; each section is a separate page and notes which evaluation categories it feeds.
1. For each scenario, follow the linked guide, run the scenario, and check the verification.
1. Grade the scenario Pass, Pass with gaps, or Fail using the rubric below, and note anything surprising (gaps, workarounds, documentation problems).
1. Total each section and carry the results into the scorecard.
The Installation and onboarding section applies to Self-Hosted Data Plane evaluations only; fully hosted evaluations (Service or Enterprise Fully Managed) start at First requests and developer onboarding. Scenarios marked *(optional)* depend on the evaluation scope.

  Scoresheet document
  
    {props.body ?? 'For your convenience, this page can be downloaded as PDF or printed directly. Site navigation is omitted, the scoring table keeps its rows intact with the header repeated on every page, and the Grade and Notes columns leave room for handwriting.'}
  
  
    <button
      type="button"
      className="tare-print-button"
      onClick={() => window.print()}
    >
      <svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" strokeWidth="2" strokeLinecap="round" strokeLinejoin="round" aria-hidden="true">
        <polyline points="6 9 6 2 18 2 18 9" />
        <path d="M6 18H4a2 2 0 0 1-2-2v-5a2 2 0 0 1 2-2h16a2 2 0 0 1 2 2v5a2 2 0 0 1-2 2h-2" />
        <rect x="6" y="14" width="12" height="8" />
      </svg>
      Export to PDF
    </button>
  

## Grading rubric

| Grade | Points | Definition |
| ----- | ----- | :---- |
| **Pass** | 2 | Scenario completed as documented; at most minor polish items. |
| **Pass with gaps** | 1 | Scenario completed, but the documentation was unclear or workarounds not in the documentation were needed. |
| **Fail** | 0 | Scenario could not be completed, or only partially; capability missing or non-functional. |

On a printed scoresheet, mark the Grade column P, G, or F; the points are tallied in each section's subtotal.

## Prerequisites checklist

Before starting, confirm the following:

- [ ] **Console access**: sign-in works for the Agent Router Console of the evaluation environment
- [ ] **Admin Dashboard access**: sign-in works with the initial admin credentials
- [ ] **At least one AI provider credential** on hand (for example, an OpenAI API key)
- [ ] **Terminal or HTTP client** available (curl, Python, Postman)
- [ ] **OpenID Connect (OIDC) identity provider (IdP) details**: issuer URL, client ID, client secret, and an IdP administrator reachable during setup

For self-hosted data plane configurations:
- [ ] **A Kubernetes cluster** meeting the [prerequisites](/agent-router-enterprise/self-hosted-data-plane/prerequisites/), plus cloud CLI access

Optionally, depending on the evaluation scenario:
- [ ] **Model Context Protocol (MCP) client** installed (Claude Code, Cursor, or VS Code)
- [ ] **OpenTelemetry-capable observability backend** for telemetry export testing

## Evaluation sections

The ungraded [Fast-Track Evaluation](/agent-router-enterprise/fully-managed/fast-track/) runs first and covers the end-to-end path in an afternoon, from first access to an onboarded team. The graded sections follow:

- [Installation and onboarding (Self-Hosted Data Plane only)](/product-architecture/product-evaluation/installation-and-onboarding/)
- [First requests and developer onboarding](/product-architecture/product-evaluation/first-requests-and-developer-onboarding/)
- [Routing, resilience, and provider abstraction](/product-architecture/product-evaluation/routing-resilience-and-provider-abstraction/)
- [MCP gateway](/product-architecture/product-evaluation/mcp-gateway/)
- [Observability and audit](/product-architecture/product-evaluation/observability-and-audit/)
- [Cost control and budgets](/product-architecture/product-evaluation/cost-control-and-budgets/)
- [Safety and data protection](/product-architecture/product-evaluation/safety-and-data-protection/)
- [Identity and developer governance](/product-architecture/product-evaluation/identity-and-developer-governance/)
- [Models, providers, and projects](/product-architecture/product-evaluation/models-providers-and-projects/)
- [Enterprise operations and compliance](/product-architecture/product-evaluation/enterprise-operations-and-compliance/)

## Fast-track carry-over

Work done in the fast track counts here. Grades for fully covered scenarios can be claimed from fast-track evidence; partial rows list what remains.

| Fast-track step | Pre-completes | Coverage |
| :---- | :---- | :---- |
| [Step 1: Sign in and onboard](/agent-router-enterprise/fully-managed/fast-track/#sign-in-and-onboard) | 8.1 Configure single sign-on | Full |
| [Step 1: Sign in and onboard](/agent-router-enterprise/fully-managed/fast-track/#sign-in-and-onboard) | 9.1 Add a provider and enable models | Partial: verify the disabled-model error separately |
| [Step 2: Route the first request](/agent-router-enterprise/fully-managed/fast-track/#first-request) | 2.1 Make a first API call | Full |
| [Step 3: Put real traffic through it](/agent-router-enterprise/fully-managed/fast-track/#your-real-traffic) | 2.5 Connect a coding agent | Full |
| [Step 4: See what it costs](/agent-router-enterprise/fully-managed/fast-track/#see-the-cost) | 5.1 Monitor traffic and usage | Partial: locate a specific fast-track request with its provider, tokens, and latency |
| [Step 5: Set a budget](/agent-router-enterprise/fully-managed/fast-track/#set-a-budget) | 6.4 Exhaust a test budget | Partial: exhaust the test budget and watch the meter and alert |
| [Step 6: Bring teammates](/agent-router-enterprise/fully-managed/fast-track/#bring-teammates) | 8.3 Onboard a developer and issue keys | Partial: key revocation still to test |
| [Step 6: Bring teammates](/agent-router-enterprise/fully-managed/fast-track/#bring-teammates) | 6.3 Create a team budget | Full |

## Summary scoring

Count only the scenarios in scope for the evaluation (skip Installation and onboarding for fully hosted paths; skip scenarios marked optional that were not attempted) and compute the percentage of the applicable maximum.

| Section | Feeds categories | Scenarios | Max | Score |
| :---- | :---- | ----- | ----- | ----- |
| Installation and onboarding | F, G | 6 | 12 | |
| First requests and developer onboarding | G2 | 5 | 10 | |
| Routing, resilience, and provider abstraction | D | 7 | 14 | |
| MCP gateway | E | 4 | 8 | |
| Observability and audit | C, A3 | 5 | 10 | |
| Cost control and budgets | D2, D3 | 9 | 18 | |
| Safety and data protection | A, B | 6 | 12 | |
| Identity and developer governance | F1 | 3 | 6 | |
| Models, providers, and projects | D1, G1 | 11 | 22 | |
| Enterprise operations and compliance | F2, F3, G1 | 7 | 14 | |
| **Total** | | **63** | **126** | |

| Percentage | Fit | Typical profile |
| ---------- | :---- | :---- |
| 90 to 100% | Excellent fit | Nearly all scenarios pass clean; none fail |
| 70 to 89% | Strong fit | Most scenarios pass; some documentation gaps |
| 50 to 69% | Acceptable fit | Scenarios complete, but with gaps throughout or a few failures |
| below 50% | Not recommended | Failures on core scenarios |

A Fail on any non-optional scenario caps the outcome at Acceptable fit, regardless of the percentage.

## Reporting issues

When a scenario fails or a guide does not match product behavior, record:

1. **Scenario ID** and the guide followed
1. **Expected behavior** per the documentation
1. **Actual behavior**, with the exact error message or screenshot
1. **Environment**: deployment path (Service, Fully Managed, Self-Hosted), browser, and OS

Share the record with the Tetrate contact for resolution, and grade the scenario on what was achieved, not on the promise.

Where to go next

  <Link to="/where-to-begin/" className="tare-nav-card">
    Where to begin
    Set up the evaluation environment: the role and path router for all three deployment paths.
  </Link>
  <Link to="/reference/compliance/" className="tare-nav-card">
    Compliance
    Residency, retention, and the due-diligence answers behind the Enterprise operations and compliance section.
  </Link>
  <Link to="/reference/sizing-and-scale/" className="tare-nav-card">
    Sizing and scale
    Throughput, resource sizing, and the deployment footprint for load planning.
  </Link>