Guides for Developers

Written for the developer persona: the engineer who consumes AI models through the gateway and works primarily in the Developer Console. Each guide is organized around a single goal rather than a single screen.

The recommended starting point is Route requests across providers, which establishes the request path that every other developer guide builds on. The remaining guides can be read in any order as the need arises.

Quickstarts

Make an API call through Agent Router

Use Python and the OpenAI SDK to make API calls through Agent Router.

Build a chat app

Use Python and the OpenAI SDK to build a chat app that uses some of Agent Router's core functionality.

Common tasks

Route requests across providers

Create a key, route it to a backend, and send the first request through the gateway.

Improve resilience with fallbacks

Formalise the multi-provider chain into an ordered fallback policy and tune failover.

Reduce cost with traffic splitting

Distribute requests across backends by weight for cost management and gradual rollout.

Apply advanced routing rules

Use logical model names and attribute-based dispatch for A/B and canary patterns.

Use your own provider credentials

Aggregate MCP servers into a profile

Combine multiple Model Context Protocol (MCP) servers into a single endpoint for AI clients.

Integrate the gateway with an app

Point SDKs, frameworks, and code assistants at the gateway.

Test prompts in the Playground

Experiment with prompts and compare models interactively.

Monitor traffic and usage

Investigate individual requests and aggregate usage across keys, models, and providers.

Export telemetry to an observability stack

Send traces and metrics to an external observability platform.