Guides for Developers
Written for the developer persona: the engineer who consumes AI models through the gateway and works primarily in the Developer Console. Each guide is organized around a single goal rather than a single screen.
The recommended starting point is Route requests across providers, which establishes the request path that every other developer guide builds on. The remaining guides can be read in any order as the need arises.
Quickstarts
Make an API call through Agent Router
Use Python and the OpenAI SDK to make API calls through Agent Router.
Build a chat app
Use Python and the OpenAI SDK to build a chat app that uses some of Agent Router's core functionality.
Common tasks
Route requests across providers
Create a key, route it to a backend, and send the first request through the gateway.
Improve resilience with fallbacks
Formalise the multi-provider chain into an ordered fallback policy and tune failover.
Reduce cost with traffic splitting
Distribute requests across backends by weight for cost management and gradual rollout.
Apply advanced routing rules
Use logical model names and attribute-based dispatch for A/B and canary patterns.
Use your own provider credentials
Register Bring Your Own Key (BYOK) credentials and compose them with routing patterns.
Aggregate MCP servers into a profile
Combine multiple Model Context Protocol (MCP) servers into a single endpoint for AI clients.
Integrate the gateway with an app
Point SDKs, frameworks, and code assistants at the gateway.
Test prompts in the Playground
Experiment with prompts and compare models interactively.
Monitor traffic and usage
Investigate individual requests and aggregate usage across keys, models, and providers.
Export telemetry to an observability stack
Send traces and metrics to an external observability platform.