Skip to main content

Guardrails best practices & FAQ

Guardrails in Agent Router Enterprise run inline on every request that crosses the gateway, on both inputs and outputs and on traffic to large language models (LLMs) and Model Context Protocol (MCP) servers. Rolling them out safely means understanding how the three guardrail layers differ in latency and capability, promoting rules through monitor mode before enforcement, and choosing fail-open versus fail-closed behavior deliberately. This page collects the operational practices and frequently asked questions that recur when teams deploy guardrails in production: how to avoid self-inflicted outages, how to investigate a blocked request, and what the runtime layer protects — and what falls outside its scope.

Persona: Platform operator or security stakeholder rolling out guardrails in production.

Estimated time: 15--20 minutes to read; ongoing for rollout and tuning.

What you will learn​

  • How the three guardrail layers work and how they differ in latency and capability.
  • How to roll out guardrails without causing outages, using monitor mode and deliberate fail-open and fail-closed settings.
  • How to investigate a blocked request and which role to grant for read-only review.
  • What guardrails protect at runtime, and what falls outside their scope.

Best practices​

1. Roll out new guardrails in monitor mode first​

Every guardrail can run in one of three modes:

  • Monitor — audit-only. The guardrail logs what it would have done without changing the request or response. Use this as a shadow mode to validate a rule against real traffic.
  • Redact — modifies the request or response inline, substituting anonymized placeholders or masking the matched content.
  • Block — stops the request and returns a canned response with a correlation identifier.

The most common cause of self-inflicted outages is promoting a guardrail straight to block mode before validating it. Start in monitor mode, confirm the trigger rate and false-positive rate are acceptable, then promote to redact or block. The step-by-step configuration workflow is covered in Configure custom guardrails for PII and content.

2. Order guardrails so cheap checks run first​

Latency varies by guardrail type:

  • Pattern-based (regular expression) — microseconds.
  • Small language model — roughly 10--50 ms, depending on the compute allocated to the models.
  • Third-party — 100 ms or more, driven mostly by the network hop.

Stack guardrails so the cheap checks evaluate first. A pattern match can short-circuit the evaluation before a small language model or external call ever runs, which keeps tail latency down on the common path. For third-party integrations, see Configure vendor guardrails.

3. Set fail-open vs. fail-closed deliberately​

Guardrail rules default to fail-closed. Both the failure behavior and the timeout are configurable per rule. Choose based on the path: fail-closed when safety dominates, fail-open when availability dominates. Set an explicit timeout rather than relying on the default.

4. Scope code-execution guardrails narrowly​

Guardrails that reject responses containing executable code will break any coding assistant if applied across the fleet. Apply them to contexts where generated code is never expected — a support agent, for example — rather than globally. See Protect coding assistant traffic for scoping guidance.

5. Treat tuning as a controlled change​

Guardrails expose real configuration knobs. Tightening sensitivity too far produces false-positive blocks that affect end users. Validate threshold changes in monitor mode before applying them to live traffic, and restrict who can modify guardrail configuration (see the guardrail-reader role below).

FAQ​

Where do guardrails run? Does my data leave my network?​

Built-in guardrails run co-located with the router, inside your network. Data evaluated by built-in guardrails does not leave your network. Data leaves only if you explicitly configure a third-party guardrail — for example, Azure Content Safety or Amazon Web Services Bedrock guardrails — in which case you incur the network hop to that provider. See Configure vendor guardrails.

Where does Agent Router Enterprise sit relative to an agent?​

An agent is the combination of the client (the programmatic harness), the tools, and the LLM. Agent Router Enterprise sits between the client and the LLM and the MCP servers — in front of the model and the tools. It inspects what the client sends to the model and what comes back. Guardrails therefore apply to both inputs and outputs, on traffic to and from both LLMs and MCP servers. For the developer-side view of which traffic is protected, see Protect requests with guardrails.

What guardrail types are supported?​

Three built-in layers, plus extensions:

  1. Pattern-based (regular expression) — fast pattern recognition for known formats such as email addresses and credit card numbers. Ships with a built-in pattern library; custom patterns are supported.
  2. Small language model — small models for tasks such as personally identifiable information (PII) and protected health information (PHI) redaction, toxicity detection, bias detection, and jailbreak detection. Several task-specific models ship out of the box. They are small enough to run without significant compute overhead.
  3. Third-party — external services such as Azure Content Safety or Amazon Web Services Bedrock guardrails, for cases that need capabilities the built-in layers do not cover.

Fully custom webhook-style guardrails are also supported, for example an additional entitlements check that calls out to a third-party service. Configuration details are in Configure custom guardrails for PII and content and Configure vendor guardrails.

Can I plug in my own models?​

Yes. The small language model layer is plug-and-play with your own or off-the-shelf open-weight models, and supports fine-tuning with low-rank adaptation (LoRA) adapters.

Do guardrails run on outputs as well as inputs?​

Yes. Guardrails evaluate both inputs and outputs, on traffic to and from both LLMs and MCP servers. You can, for example, redact PII on the way in and block disallowed content on the way out.

A user was blocked and believes they shouldn't have been. How do I investigate?​

Blocked responses carry a correlation identifier. The user provides that identifier, and an operator with the appropriate privileges can look up which rules triggered, the confidence score, and the actual input and output that caused the block. Use the correlation identifier to join against your own logging system for the full request context. Request Logs and audit workflows are covered in Audit Agent Router activity.

What is the guardrail-reader role?​

A read-only role for reviewing guardrail logs and events. It lets security teams read which rules triggered and review guardrail activity without granting permission to modify guardrail settings. Reading unredacted inputs and outputs requires sufficient privilege.

Can guardrails enforce a budget or cost cap?​

Budgets are an observability feature, not a blocking control. This is intentional: blocking on a budget would let a single typo cause an outage. Set budgets per team or per user, export them as metrics, and drive alerts from your own alerting stack. To constrain abuse or runaway usage, use rate limiting instead — by source address or by account, on either request count or token count. See Working with budgets.

How are logs and metrics handled?​

Agent Router Enterprise is designed so it never has to handle your sensitive request and response data. Request and response logs go directly to your own logging tenant. The product sees only the metadata required to drive Agent Router behavior, such as token usage and timing. Metrics are exposed over OpenTelemetry for ingestion into your own monitoring system, and audit logs (configuration changes, administrative interface access, user management) can be written out for ingestion into your security information and event management (SIEM) system.

Can guardrails handle non-English or obfuscated attacks?​

Built-in guardrails cover common cases. Some language-specific models exist but are not deployed by default. A language-allowlist guardrail can block any request that is not in an approved language, which is a useful control for high-risk locales. Exotic evasion — rare languages, novel encodings, or techniques like poetry injection — may require a specialist third-party guardrail. For jailbreak and injection detection specifically, see Detect and block prompt injection. This is an ongoing area: new evasion techniques appear regularly, and coverage is a moving target.

What do guardrails protect, and what is out of scope?​

Agent Router Enterprise is a runtime layer. It protects anything crossing the network to or from an LLM or MCP server. Think of it as data loss prevention (DLP) that does not itself require inference.

Out of scope:

  • Training-time and pre-deployment concerns — data poisoning, manipulation of model weights, and anything in the artificial intelligence software development lifecycle before the model is served.
  • Client-side activity — lateral movement that manipulates the client itself, or data that never crosses the network (for example, a local file read by a coding agent).
  • Direct client-to-retrieval access — the product protects data on its way to the model, but not the gap between a retrieval-augmented generation (RAG) or vector store and the client that queries it directly. If the same data is exposed over MCP instead, the product can apply protection at both the access layer and the model layer.
  • Supply chain — Agent Router Enterprise ships as versioned Helm charts with pinned component versions. The rest of the supply chain — the agents, application programming interfaces (APIs), and MCP servers you choose to deploy — remains your responsibility. Version Agent Router configuration through automation as described in Manage configuration as code.

What to do next​

Where to go next

Configure custom guardrails

Define organisation-specific rules and pilot them in monitor mode before enforcement.

Configure vendor guardrails

Layer provider-native or third-party safety filters where built-in layers are not enough.

Protect requests with guardrails

The developer-side view of attaching a guardrail to a specific request path.

Detect and block prompt injection

Handle jailbreak and indirect injection separately from content guardrails.