Skip to main content

Provision models and providers

Providers are connected and models enabled in the Tetrate Agent Router Admin Console catalog, so there is something to grant to a project. A default project exists from onboarding (Key concepts → Projects); after enablement here, grant models in Create a project and grant models.


Provider keys come first

Enabling a model does not make it callable. The provider behind it needs a working provider key first, and on Enterprise that key is customer-supplied: Tetrate does not provide provider keys on this tier.

Without one, the model still appears in the catalog but returns 404 when a developer calls it, and nothing in the Console marks it as broken. Set up the provider key first, in Step 1, and check that the provider shows Key stored in the Auth Method column and Enabled in the Availability column before enabling anything.

Persona: Platform operator working in the Admin Console.

Estimated time: 20 to 30 minutes for an initial provisioning pass, depending on how many providers and models are involved.

When this guide applies​

SituationWhy this guide helps
First-time setup of a fresh deploymentThe catalog must have enabled models before any project can be granted access to them
Adding a new AI provider relationship (a contract with a vendor, or a self-hosted endpoint)The provider must be configured and verified before any of its models can be enabled
Adding new model versions from an existing providerAgent Router exposes only the models that have been explicitly enabled, not everything the provider supports
Rotating provider keys on a schedule or after a security eventA clean key rotation is the recurring operational task that this layer requires
Retiring a model or provider as part of a sunset or contract changeDisabling at the provider or model level cleanly removes capability from the developer-facing surface

Outcomes​

By the end of this guide:

  • At least one provider is configured with a stored provider key (Key stored in the Auth Method column) and Enabled in the Availability column.
  • At least one model from that provider is enabled and visible to developers under Discover → Models in the Developer Console.
  • Key rotation and provider removal are understood.
  • Private-link connectivity, where appropriate, keeps model traffic on the cloud backbone.

Prerequisites​

  • Administrator access to the Admin Console, typically the super_admin or provider_admin role. The exact role model is covered in Manage roles and permissions.
  • Valid credentials for at least one upstream AI provider: an API key, an Azure deployment endpoint, or whatever the provider expects.
  • For private-link setups, the network-level configuration (the private endpoint, route, or service connection) is already in place in the cloud environment. The Admin Console configures only the endpoint URL that the gateway dials; it does not provision the underlying network.

Orient in the Admin Console​

The Admin Console is reached at the deployment's dashboard URL, for example https://dashboard.poc.tetrate.ai/, with administrator credentials or corporate SSO, and opens on the Home page. The Admin Console configures what is available, to whom, and under what conditions; models and providers enabled here are what developers access in the Console.

Who uses the Admin Console​

Three operator personas typically work in the Admin Console:

RoleTypical tasks
Agent Router administratorsEnable and disable models, manage provider keys, configure SSO
IT operationsMonitor Agent Router health, review audit logs
Security leadsReview user access, audit administrative actions, manage API keys

The Admin Console sidebar groups its items under the following sections. A scope switcher at the top of the sidebar selects Organization or a project.

SectionPurpose
HomeThe landing page; displays organization-wide metrics and the data plane status
Catalog → ModelsBrowse the model registry, enable or disable models, and filter by provider, mode, or status
Catalog → ProvidersManage connections to AI providers, view each provider's availability, configure credentials, and add new connections
Catalog → MCP ServersManage the catalog of Model Context Protocol (MCP) servers available to Console users, configure authentication types, and view the tools each server exposes
Catalog → Request AttributesDefine organization-wide routing attributes shared by every project. Defining an attribute does not change how any request is routed
Catalog → MCP OAuth ClientsCreate and manage OAuth client configurations for MCP servers that require OAuth-based authentication
Usage → AnalyticsOrganization-wide usage metrics across all users and API keys, with breakdowns by user, model, provider, and API key, time-range filtering, and cost analysis
Usage → Request LogsLook up a request by its request ID, with the requesting user, the requested and answering models, and the policies that acted on it
Policies → BudgetsBudgets that track spend per user or team, with a billed vs passthrough spend overview; tracking comes first and enforcement is a separate opt-in
Policies → FallbackCreate and manage fallback policies
Policies → Attribute-Based RoutingLists attribute-based routing policies across all projects. Opening a policy manages it in its project
Directory → ProjectsSet up and manage projects. Data planes carry a project label, and apps reach a project through any data plane with that label
Directory → Data PlanesRegister and manage data planes, with their state, project label, config, verified status, and version
Directory → API KeysAdmin-level view of all API keys across Agent Router (key owners, creation dates, and last-used timestamps) with the ability to revoke keys
Directory → PrincipalsLists principals, filterable by caller kind (Person, Application, or Unknown)
Directory → TeamsCreate and manage teams and their users
Directory → RolesDefine roles and their permissions. Built-in roles are editable, and Copy starts a new role from an existing one
Directory → AccessWho has access: role and direct-grant assignments to users and groups, grouped by principal
Directory → UsersView and manage all registered users, including roles, last sign-in timestamps, and activity
System → ReleasesWhat shipped in each release, and the chart and image tags it carries
System → Registry CredentialsManage registry credentials
System → Audit LogsTamper-resistant chronological history of all administrative actions, filterable by user, action type, resource, and time range
System → Content → AnnouncementPublish organization-wide announcements, with a schedule and a preview
System → Content → Sign-in PageManage the sign-in page content
System → SettingsThe Settings page, with the Profile, SSO, Guardrails, Request logs, and Passthrough tabs

Read the Home page of the Admin Console​

The Home page shows organization-wide metrics for a selected time range, with a Refresh button beside the range selector. The page displays these cards:

  • Active Users
  • Total Tokens
  • Total Spend
  • API Requests
  • Success Rate
  • Errors

The page also shows a data plane status card, which links to Directory → Data Planes, and the Most used models and Active users tables.

Step 1: configure the provider​

Providers are the upstream services the gateway is allowed to reach. Each provider entry carries the credentials and endpoint information for one external AI service.

  1. Sign in to the Admin Console.
  2. Open Catalog → Providers from the sidebar.
  3. Review the existing provider list. Each entry shows the provider, its API base URL, its auth method, its availability, and the last updated timestamp.
  4. Click Add provider at the top of the list.
  5. Enter the Provider identifier. The identifier uses lowercase letters and digits only and must start with a letter.
  6. Enter the API base URL. The dialog describes it as the base URL for the provider API endpoint.
  7. Optionally, enter a Display name. The dialog shows it in tables and search filters.
  8. Select the Auth method. The dialog lists only known auth methods, so select the one that matches how models authenticate against the provider.
  9. Select the Secret type and enter the Provider secret. Secret values are not shown.
  10. Leave the Enable provider switch on to make the provider available. Switch it off to keep the provider hidden until it is configured.
  11. Click Add provider. The button stays disabled until the required fields are filled.

Rows for providers with a stored key show Key stored in the Auth Method column. Rows without one show No key.

A common cause of a provider whose requests fail right after setup is a credential pasted with leading or trailing whitespace, which the provider rejects without any visible cause. Re-pasting from a known-good source is the fastest first fix.

Field reference: provider list​

Each entry in the provider list carries the following columns:

ColumnDescription
ProviderThe provider name, a button that copies the provider ID, and a First-party label for first-party providers
API Base URLThe base URL of the provider API endpoint, with a button that copies it
Auth MethodThe auth type, with Key stored or No key
AvailabilityEnabled or Disabled, set with a switch
Last UpdatedThe timestamp of the last update
ActionsA menu for each provider, labeled Actions for followed by the provider name

Private provider connectivity​

For deployments where compliance or performance requirements demand that provider traffic stay on the cloud backbone rather than traversing the public internet, the API base URL field can be set to a private-link address (Azure Private Link, AWS PrivateLink, or GCP Private Service Connect), and the data plane reaches the provider over that private connection.

Agent Router configures only the URL it dials. The underlying private endpoint (DNS, routing, and network policy) is configured at the cloud level beforehand, and the value entered here references the result of that work. For the architectural detail on this setup, see Architecture overview.

Step 2: enable models for the provider​

Models are registered in Catalog → Models with Add model, Sync from published catalog, or Import. The Add model dialog accepts only an enabled provider, and its Availability switch is on by default.

  1. Open Catalog → Models from the sidebar.
  2. Review the model list. Each row shows the model, its provider, its mode, its availability, and the last updated timestamp.
  3. Use the search box to find a model by name, or the provider filter to narrow the list. Both work in real time.
  4. Click Edit in the Actions column of a model row to open the model's edit drawer, which shows its provider, identifier, mode, availability, Autofallback setting, pricing, context window, and capabilities. Click Cancel to close the drawer and return to the list.
  5. Locate a model that should be exposed and toggle its Availability switch to Enabled.

A newly enabled model appears under Discover → Models in the Developer Console immediately. Developers whose key policies target it can start sending requests right away.

Field reference: model list​

Each row in the model list carries the following columns:

ColumnDescription
ModelThe model name, with a button that copies the model ID
ProviderThe AI provider that serves the model
ModeThe provider mode classification, such as Chat
AvailabilityEnabled or Disabled, set with a switch
Last updatedThe timestamp of the last update
ActionsEdit and Delete buttons for each model

The Add model dialog sets the context window and the input and output token prices (per 1M tokens) for a model.

To change several models at once, click Batch Edit in the toolbar. The list switches to a batch mode with a checkbox on each row, a Select all loaded models on this page for bulk edits checkbox in the header, and Input Modalities and Output Modalities checkboxes that apply to the selected models. Exit Batch Edit returns to the normal list.

tip

Enabling every model from every provider gives developers a confusing menu, complicates Request Logs and Usage analytics, and makes model-specific policy harder to enforce. Default to the smaller list: only the models the organization has decided to support.

Disabling a model does not delete it from the registry, and re-enablement is immediate. Requests that target a disabled model receive an error indicating the model is unavailable.

Configure model-level auto-fallback​

A model's edit drawer exposes an Autofallback setting. When it is enabled and one or more fallback models are chosen, the data plane attaches that fallback chain to the model itself: if the model's primary provider returns a recoverable error, requests fall back to the chosen fallback model automatically. This is the operator-side, catalog-level counterpart to the per-key fallback policy developers configure in the Console.

The two differ in whose traffic the fallback protects:

  • Model-level auto-fallback (this setting, in the Admin Console) applies to the model across the organization catalog. It is the right tool for organization-wide resilience that should not depend on each developer configuring their own key.
  • Per-key fallback (configured developer-side in the Console, under Improve resilience with fallbacks) applies only to the single API key it is attached to. A request sent with any other key does not inherit it, and falls through to the model's global route instead.

To enable it:

  1. Click Edit in the model's row in Catalog → Models.
  2. Turn on Autofallback.
  3. Choose one or more fallback models, in priority order.
  4. Click Save changes.

The fallback model must itself be enabled in the catalog (Step 2); a fallback entry that points at a disabled or unlisted model is skipped, leaving the chain shorter than intended. When the fallback provider differs from the primary (for example the same model served from a second region), confirm the fallback model resolves to the correct upstream identifier before relying on it, by sending a test request that forces the fallback path (see Step 3).

Step 3: verify end-to-end through a developer-side request​

A short end-to-end check from the developer surface confirms that the model is reachable from the gateway and that the credentials flow correctly.

  1. Open the Console (a separate sign-in to the developer surface).
  2. Create or reuse an API key with policies that target the newly enabled model. The mechanics are covered in Route requests across providers.
  3. Send a test request through the gateway.
  4. Confirm a successful response.
  5. Return to the Admin Console.
  6. Open Usage → Analytics. Within a short delay, the test request appears in the organization-wide usage view, attributed to the model and provider just provisioned.

If the request fails with an authentication error on the upstream side, the provider's credentials are the most likely cause; if the request fails because the model is unavailable, the enablement step has been missed or has not yet propagated. Both failure modes are distinguishable in the Console's Request Logs.

A model that is enabled but not callable​

Six distinct conditions present the same way: the model is enabled in the Admin Console, and a request for it fails. The error code the gateway returns is what separates them, so capture it before changing any configuration.

SymptomConditionResolution
400, code model_not_availableThe model is listed by GET /v1/models but its catalog entry is still propagating (data planes on 0.4.0 or later)Retry. This clears on its own
503, code model_not_readyThe route exists but is not active yet. The response carries Retry-After: 30Retry. This clears on its own
404, code model_not_routed, clearing within a minuteThe change has not reached this data plane yetWait, then retry. See Configuration propagation
400, code model_not_foundThe model is enabled in the organization catalog but not assigned to the project the key belongs to. The gateway answers exactly as for a model that does not exist, so a caller cannot learn what another project holdsAssign it, see Add providers and models to projects
404, code model_not_routed, persistingThe model is enabled and assigned, but no route for it is deployed on this data planeConfirm the data plane serves the project and that the model's provider has deployed; see Gateway behavior
404 or an upstream authentication error, no model behind the provider workingThe provider has no working credential, so no backend exists for its routes to point atSet or rotate the credential on the provider entry

Work through it in this order:

  1. Read the error code, not just the status. The bodies are listed in Gateway behavior.
  2. Call GET /v1/models with the same key. It returns what that key can actually reach on that data plane, which is the authoritative answer and frequently disagrees with the catalog count. A model absent from that list is not callable, whatever the Console shows.
  3. Check the provider before the model. One missing credential takes out every model behind it at once, which reads as a catalog-wide failure rather than a provider one. The failure stops at that provider: since 0.5.0 an unreadable credential no longer stalls route deployment for other providers, so the affected provider stays on its last deployed configuration and is picked up again once its credential is readable, while every other provider keeps updating normally. On earlier data planes, one unreadable credential could hold up route updates for unrelated providers too.
  4. On a self-hosted data plane, run tetrate agentrouter doctor. Routes that reference a backend that does not exist are the signature of the credential case. See Reading tetrate agentrouter doctor output.

Step 4: operate the provisioned layer​

Recurring operations use the same screens as the initial setup. Releasing and retiring models can be automated: Keep the model catalog current reconciles the deployment against Tetrate's published catalog, disabling retired models and picking up new ones and price changes.

Rotate provider keys​

Provider keys are rotated on a schedule, after a suspected compromise, or when a contract migrates to a new account.

  1. Open Catalog → Providers.
  2. Open the Actions menu for the provider and select Rotate secret.
  3. Enter a new secret.
  4. Confirm that the provider still shows Key stored in the Auth Method column and Enabled in the Availability column.

The rotation takes effect on the next request. In-flight requests that were authorized with the old credential complete with it; subsequent requests use the new one. There is no service restart and no downtime window.

Since 0.4.0, changing a provider's authentication type revokes the credential the provider previously held: the stored key and the policies that inject it are removed from the data plane on the next configuration push, with no action needed beyond making the change. On earlier releases the stored key kept authenticating managed traffic after such a change, with nothing in the Console to show it was still live, so retiring a key on an older release means also revoking it on the provider side.

Remove a provider configuration​

Retiring a provider (contract ended, account consolidation, or provider deprecated) ends with a single delete, after the provider's models are gone.

  1. Delete every model registered under the provider, and revoke any Bring Your Own Key (BYOK) keys registered for it.
  2. Open the provider's actions menu and select Delete.
  3. Type DELETE and confirm.

The delete is refused while the provider still has models or BYOK keys, or while it is the only provider of a project that still has model assignments. The dialog then stays open and shows a message naming the blocker, listed in Console messages and states. Any key policy that targeted the removed models falls through to its configured fallback, if one exists, or fails with an unavailable-model error otherwise. Before removing a provider that is actively serving traffic, scan the Console-side key policies and reroute or retire any that depend on it.

Add new model versions​

A new provider model version is registered in Catalog → Models with Add model, Sync from published catalog, or Import. Agent Router never enables new models on the operator's behalf; enabling is a separate step:

  1. Locate the new model in the list.
  2. Review its context window and token prices.
  3. Toggle its Availability switch to Enabled.
  4. Coordinate with the developer side, through internal communication or through the Communicate with Agent Router users flow, so that developers know the model is available.
  5. Optionally pair the introduction with a canary rollout on the developer side, using the patterns from Apply advanced routing rules.

With the catalog in place, grant models to a project, then onboard keys and apply spend controls as needed.