Skip to main content

Guardrails best practices & FAQ

Guardrails in Agent Router Enterprise run inline on model traffic that crosses the gateway — on both inputs and outputs to large language models (LLMs). Model Context Protocol (MCP) traffic is governed separately through identity, access control, and audit; content guardrails do not evaluate standalone MCP profile requests today. Rolling guardrails out safely means understanding how the provider kinds differ in latency and capability, promoting rules through monitor mode before enforcement, and choosing fail-open versus fail-closed behavior deliberately. This page collects the operational practices and frequently asked questions that recur when teams deploy guardrails in production: how to avoid self-inflicted outages, how to investigate a blocked request, and what the runtime layer protects — and what falls outside its scope.

Persona: Platform operator or security stakeholder rolling out guardrails in production.

Estimated time: 15 to 20 minutes to read; ongoing for rollout and tuning.

What you will learn​

  • How the provider kinds (regexp, builtin, tetrate) work and how they differ in latency and capability.
  • How to roll out guardrails without causing outages, using monitor mode and deliberate fail-open and fail-closed settings.
  • How to investigate a blocked request and which role to grant for read-only review.
  • What guardrails protect at runtime, and what falls outside their scope.

Best practices​

1. Roll out new guardrails in monitor mode first​

Each rule has an action and a mode:

  • Action: redact — modifies the request or response inline, substituting anonymized placeholders or masking the matched content.
  • Action: block — stops the request and returns an HTTP 40x with an optional message (often including a correlation identifier).
  • Mode: monitor — audit-only. The gateway records what it would have done without changing the request or response. Use this as a shadow mode to validate a rule against real traffic.
  • Mode: enforce — applies the configured action.

The most common cause of self-inflicted outages is promoting a rule straight to enforce with block before validating it. Start in monitor mode, confirm the trigger rate and false-positive rate are acceptable, then switch mode to enforce. The step-by-step configuration workflow is covered in Configure guardrails.

2. Order guardrails so cheap checks run first​

Latency varies by provider kind:

  • regexp — near zero; in-process pattern matching against the Patterns catalog.
  • builtin — near zero; in-process multi-layer keyword and regex pipeline.
  • tetrate (and any remote / external guardrail service where one is configured) — typically 100 ms or more, driven mostly by the network hop and inference.

Stack rules so cheap local providers evaluate first. A pattern match can short-circuit evaluation before a remote ML call ever runs, which keeps tail latency down on the common path. See Configure guardrail providers.

3. Set fail-open vs. fail-closed deliberately​

Guardrail rules default to fail-closed. Both the failure behavior and the timeout are configurable per rule. Choose based on the path: fail-closed when safety dominates, fail-open when availability dominates. Set an explicit timeout rather than relying on the default.

4. Scope code-execution guardrails narrowly​

Guardrails that reject responses containing executable code will break any coding assistant if applied across the fleet. Apply them to contexts where generated code is never expected — a support agent, for example — rather than globally. See Protect coding assistant traffic for scoping guidance.

5. Treat tuning as a controlled change​

Guardrails expose real configuration knobs. Tightening sensitivity too far produces false-positive blocks that affect end users. Validate threshold changes in monitor mode before applying them to live traffic, and restrict who can modify guardrail configuration (see the guardrail-reader role below).

FAQ​

Where do guardrails run? Does my data leave my network?​

Local providers (regexp and builtin) run co-located with the router, inside your network. Data they evaluate does not leave your network. Data leaves only if you explicitly use a remote provider such as tetrate, or an external guardrail service where one is configured, in which case you incur the network hop to that endpoint. See Configure guardrail providers.

Where does Agent Router Enterprise sit relative to an agent?​

An agent is the combination of the client (the programmatic harness), the tools, and the LLM. Agent Router Enterprise sits between the client and the LLM and the MCP servers — in front of the model and the tools. Content guardrails inspect what the client sends to the model and what comes back on that model path. MCP profile traffic (/mcp/*) is subject to identity, access control, and audit, but not to content guardrails today. For the developer-side view of which traffic is protected, see Protect requests with guardrails.

What guardrail providers are supported?​

Three provider kinds ship by default, arranged as a cost-and-capability ladder:

  1. regexp — fast pattern recognition for known formats such as email addresses and national identifiers. Uses the Patterns library; custom patterns are supported.
  2. builtin — a five-layer local keyword and regex pipeline (blocked keywords, allowlist gate, subject-plus-term co-occurrence, detection patterns, category keywords).
  3. tetrate — remote ML detection (embeddings, classifiers, NLI) for PII, toxicity, jailbreak, prompt injection, banned topics, and related check types.

Where an external guardrail service is configured, it appears as another provider instance rules can reference. Configuration details are in Configure guardrails and Configure guardrail providers.

Can I plug in my own models?​

The tetrate provider is backed by Tetrate's semantic router and ships with task-specific models for the supported check types. Where an external guardrail service is configured, its capabilities depend on that service's contract. See Configure guardrail providers.

Do guardrails run on outputs as well as inputs?​

Yes. Guardrails evaluate both inputs and outputs on model traffic. You can, for example, redact PII on the way in and block disallowed content on the way out. Standalone MCP profile traffic is not evaluated by content guardrails.

Note that not every check runs on both stages for every provider kind: some (provider, check) pairs are input-only or output-only, and a rule that leaves its Run on setting unset is asking for both stages. See the capability matrix.

Why was my rule or provider rejected with an invalid-argument error?​

The API validates every guardrail rule and provider write against the platform's capability matrix: the check type must be supported by the provider's kind, and the requested stage (unset means both) must be within the stages that (kind, check) pair supports. Unsupported combinations are rejected at write time with an error naming the supported set, so a rule that could never fire cannot be created. Provider creation likewise rejects unknown kinds, and external-service providers must declare a valid, non-empty check list. See Write-time validation for the full contract.

An existing guardrail rule never fires. Why?​

Rules created before write-time validation existed are grandfathered: they still run, but an unsupported (provider kind, check type, stage) combination scores zero and never fires. The control plane logs a warning (with the rule ID, provider kind, check type, and stage) whenever it ships such a rule, and the dashboard shows an unsupported stored check as disabled in the rule form. The troubleshooting steps are in Rules that never fire.

A user was blocked and believes they shouldn't have been. How do I investigate?​

Blocked responses carry a correlation identifier. The user provides that identifier, and an operator with the appropriate privileges can look up which rules triggered, the confidence score, and the actual input and output that caused the block. Use the correlation identifier to join against your own logging system for the full request context. Request Logs and audit workflows are covered in Audit Agent Router activity.

What is the guardrail-reader role?​

A read-only role for reviewing guardrail logs and events. It lets security teams read which rules triggered and review guardrail activity without granting permission to modify guardrail settings. Reading unredacted inputs and outputs requires sufficient privilege.

Can guardrails enforce a budget or cost cap?​

Budgets are an observability feature, not a blocking control. This is intentional: blocking on a budget would let a single typo cause an outage. Set budgets per team or per user, export them as metrics, and drive alerts from your own alerting stack. To constrain abuse or runaway usage, use rate limiting instead — by source address or by account, on either request count or token count. See Working with budgets.

How are logs and metrics handled?​

Agent Router Enterprise is designed so it never has to handle your sensitive request and response data. Request and response logs go directly to your own logging tenant. The product sees only the metadata required to drive Agent Router behavior, such as token usage and timing. Metrics are exposed over OpenTelemetry for ingestion into your own monitoring system, and audit logs (configuration changes, administrative interface access, user management) can be written out for ingestion into your security information and event management (SIEM) system.

Can guardrails handle non-English or obfuscated attacks?​

Shipped providers cover common cases. A language-allowlist rule (language check type) can block any request that is not in an approved language, which is a useful control for high-risk locales. Exotic evasion — rare languages, novel encodings, or techniques like poetry injection — may require a remote or external provider beyond local pattern matching. For jailbreak and injection detection specifically, see Detect and block prompt injection. This is an ongoing area: new evasion techniques appear regularly, and coverage is a moving target.

What do guardrails protect, and what is out of scope?​

Agent Router Enterprise is a runtime layer. Content guardrails protect the model request and response path — traffic to and from an LLM. Think of that path as data loss prevention (DLP) that does not itself require inference. Standalone MCP profile traffic (/mcp/*) is out of content-guardrail scope today; it is governed by identity, access control, and audit instead.

Out of scope:

  • Training-time and pre-deployment concerns — data poisoning, manipulation of model weights, and anything in the artificial intelligence software development lifecycle before the model is served.
  • Client-side activity — lateral movement that manipulates the client itself, or data that never crosses the network (for example, a local file read by a coding agent).
  • Direct client-to-retrieval access — the product protects data on its way to the model, but not the gap between a retrieval-augmented generation (RAG) or vector store and the client that queries it directly. If the same data is exposed over MCP instead, Agent Router can apply access-layer controls (identity, ACL, audit) on the MCP path; content guardrails apply only when that data later rides the model path.
  • Standalone MCP profile traffic — client calls to /mcp/* are not inspected by content guardrails. Govern which servers a profile may reach through Govern MCP server access.
  • Supply chain — Agent Router Enterprise ships as versioned Helm charts with pinned component versions. The rest of the supply chain — the agents, application programming interfaces (APIs), and MCP servers you choose to deploy — remains your responsibility. Version Agent Router configuration through automation as described in Manage configuration as code.

What to do next​

Where to go next

Configure guardrails

Define project policies and rules and pilot them in monitor mode before enforcement.

Configure guardrail providers

Choose regexp, builtin, or tetrate for each check type.

Protect requests with guardrails

The developer-side view of attaching a guardrail to a specific request path.

Detect and block prompt injection

Handle jailbreak and indirect injection separately from content guardrails.