# Guardrails > How the Guardrails dynamic module inspects model traffic inline at the Tetrate Agent Router gateway: interception, evaluation, enforcement, egress inspection, and recording, defined from first principles for newcomers. A guardrail is a named project policy that inspects model traffic as it passes through the gateway and can change or stop that traffic before it reaches a model, or before a model's reply reaches the application. In Tetrate Agent Router, guardrails run as a dynamic module inside the gateway rather than as safety code written into each application. Every model request that crosses the gateway passes through the module, which evaluates configured rules, and acts on the verdict before the request reaches a model provider. The same inspection applies to the response on the way back. Standalone Model Context Protocol (MCP) profile traffic (`/mcp/*`) does not pass through content guardrails; it is governed by identity, access control, and audit instead. This page explains how that inspection works from first principles, defining each term as it appears. No prior familiarity with Tetrate Agent Router, with proxies, or with the internals of an AI gateway is assumed. The operator and developer guides linked at the end cover how guardrails are configured and used once the underlying mechanism is clear. For the wider context this page sits in, the Architecture overview explains what the gateway is and where it runs, and Planes and core components describes the data-plane components (including the dynamic modules) in one place. The Glossary is a quick lookup for any single term. Three pages cover guardrails, and they answer different questions. This page explains *where the module sits and what that placement costs and buys*. [About guardrails](/agent-router-enterprise/guides/operate-and-govern/safety-and-data-protection/about-guardrails) explains *how an operator reasons about guardrails, rules, patterns, and providers* as objects to configure. The [guardrails reference](/reference/guardrails) is the lookup surface for the fixed vocabularies: the 23 check types, every rule-level setting and its permitted values, and the provider-by-check-type matrix. ## What a guardrail is, and why it runs at the gateway A large language model (LLM) takes a piece of text (a *prompt*) and returns generated text (a *response*). Prompts and responses are not always safe to pass along untouched. A prompt might contain Personally Identifiable Information (PII), meaning data that identifies a person, such as a name, an email address, a phone number, or a payment-card number. A prompt might also be an attack. A response might reproduce sensitive data, or return content the organisation does not permit. A *guardrail* is a named policy that addresses these risks. On its own it does nothing: its power lives in the **rules** it contains. Each rule names a **check type** (what to look for) and a **provider** (the engine that scores the content). Providers return a normalised score; the Guardrails module compares that score to a threshold and applies the rule's **action** (`block` or `redact`) when the rule's **mode** is enforce. The full vocabulary of check types, and every setting a rule carries, is tabulated in the [guardrails reference](/reference/guardrails#rule-level-settings). Placing that policy at the gateway (on the model path every LLM request already passes through) rather than inside each application means one policy protects every application in the project at once, and applications need no safety code of their own. Guardrails are **bound to a project**, not to the organisation as a whole. The Admin Dashboard's project selector switches which set of guardrails, patterns, and guardrail providers is under management. A guardrail authored under one project does not apply to another. The rest of this page explains the machinery that makes that inspection possible, and then how a single request moves through it. ## Proxies, filters, and dynamic modules Understanding guardrails means understanding a little about how the gateway is built. A *proxy* is a server that sits between a client and a destination: it receives a request, does something with it, and forwards it on. The Agent Router gateway is a proxy. It receives each AI request, selects a model provider, applies policy, and forwards the request to that provider. It is built on Envoy, a widely used open-source proxy. A proxy like this handles a request through a *filter chain*: an ordered pipeline of small processing stages called *filters*. Each filter inspects or modifies the request in turn, and the request travels through the chain in sequence before being routed to its destination. A filter chain is how a proxy composes many independent behaviours (authentication, routing, rate limiting, safety checks) into one coherent path without any single filter needing to know about the others. Adding a new behaviour to that pipeline used to mean recompiling the proxy from C++, or accepting the slower execution of a WebAssembly (Wasm) or Lua extension. A *dynamic module* is the modern alternative: add-on logic written as a high-performance extension (often in the Rust or Go programming languages) and loaded into the proxy at runtime as a *shared library*, a compiled file (for example a `.so` file) that a program loads and runs as part of itself rather than as a separate process. Because the module runs inside the proxy, it adds no extra network hop and executes at close to native speed. Guardrails are one such dynamic module. Cost controls are another. The key property is *inline processing*: the module runs as part of handling the request, inside the same filter chain, rather than as a separate service the gateway has to call over the network and wait for. The Guardrails module is one of the data-plane components. Its place alongside the Controller and Redis is described in Planes and core components. Remote evaluation, when used, calls out to a configured guardrail provider (for example the shipped tetrate ML provider, or an external guardrail service where one is configured). ## How the Guardrails dynamic module works The module operates in five stages within the filter chain: interception, evaluation, enforcement, egress inspection, and recording. The first three apply to the prompt on its way to the model. The fourth applies the same logic to the response on its way back. The fifth is not sequential: a record is emitted for each rule evaluation, wherever in the request it occurred. ### Stage 1: interception (the prompt) When an application sends a prompt to an AI model, the request reaches the gateway first, not the model provider. The Guardrails module intercepts the HTTP *payload* (the payload is the actual content the request carries) before it is forwarded to the downstream model provider, whether that provider is an external service such as OpenAI or a model the organisation runs itself. A payload is not necessarily one block of prose. A rule declares which part of it to inspect, choosing between all content, plain text, an image, a document, tool output, the model's thinking, or retrieved context; the permitted values are listed in the [guardrails reference](/reference/guardrails#rule-level-settings). The distinction matters because the surfaces differ in who controls them. Text is what the user typed, while tool output and retrieved context arrive from a backend and are read by the model with the same authority as the rest of the prompt, which is what makes them the vector for the indirect attacks described under evaluation below. Interception is what makes everything that follows possible: the module holds the prompt while it decides what to do, and nothing is forwarded until that decision is made. ### Stage 2: evaluation Once the module holds the payload, it runs the project's active guardrail rules. Each rule is scored by its **provider**: - Locally: engines such as `regexp` (named patterns from the Patterns catalog) and `builtin` (a multi-layer keyword and regex pipeline) run in process inside the gateway. Local checks add minimal latency because no network call is involved and the data never leaves the gateway. - Remotely, by callout: for analysis that local pattern matching cannot perform, a provider such as `tetrate` (or an external guardrail service, where one is configured) pauses the request, calls a remote endpoint, and resumes once the service replies. A callout is more capable but slower than a local check, because it involves a network round trip. **Providers score; the module decides.** A provider returns a normalised score in the range 0.0 to 1.0 (and, for redaction, a sanitised copy of the text). It never chooses to block or redact and never compares against a threshold. The module compares `score ≥ threshold` to decide whether the rule was violated. How each engine arrives at its score is covered in [About guardrails](/agent-router-enterprise/guides/operate-and-govern/safety-and-data-protection/about-guardrails#guardrail-providers). Evaluation is bounded by the guardrail's **evaluation timeout**, and a provider can also simply fail. Either way the rule produces no score, and the guardrail's **failure mode** decides the outcome: fail-close, the default, treats the rule as violated and the content is blocked; fail-open skips the rule and the content passes. This is the most consequential piece of the design to hold in mind, because it means an unreachable remote provider stops traffic unless the guardrail was deliberately configured not to. Providers are not retried, so that deterministic behaviour is the gateway's alone. Two categories of threat are worth naming here, because they drive what evaluation looks for. - Prompt injection is text that tries to subvert the model rather than being unsafe in itself: instructions smuggled into the prompt (or into content the model is given to work with) that make the model ignore its own instructions, leak data, or act incorrectly. - A jailbreak is a direct form of prompt injection in the user's own prompt, the familiar "ignore previous instructions" family of attacks and role-play framings intended to escape the model's configured behaviour. ### Stage 3: enforcement (action and mode) When a rule is violated, two orthogonal fields decide what happens: - **Action:** `redact` alters the payload (for example replacing a detected credit-card number with a placeholder) and lets the request continue; `block` stops the request and returns HTTP `403` with an optional message and a correlation identifier, so the model provider never receives the request. - **Mode:** `enforce` applies the action; `monitor` records the would-be action and lets the content pass unchanged. Monitor mode is how a rule is validated against live traffic before enforcement, and a monitor rule is always effectively fail-open, because it never stops a request. A block ends evaluation immediately. A redaction does not: because the payload has changed, the module swaps in the sanitised text and re-runs any providers whose rules are still pending against the new content. One rule's redaction therefore determines what later rules see, which is why the order in which cheap deterministic engines and expensive remote ones are stacked has an effect beyond latency. If no rule is violated, the filter chain continues and the request is forwarded. ### Stage 4: egress inspection (the response) Guardrails are not limited to incoming prompts. *Egress* means traffic leaving toward its destination, and here it refers to the model's response on its way back to the application. When the model generates a response, that response flows back through the proxy, and the module inspects it using the same rule evaluation before it reaches the caller. Egress inspection is what catches a response that reproduces sensitive data or returns disallowed content, closing the gap that inspecting only the prompt would leave open. Egress inspection is not free, and its cost is paid in streaming. A model normally returns its answer token by token, and the gateway forwards each fragment as it arrives. A rule cannot judge content it has not seen in full, so if any rule on the output stage is in enforce mode, the response is buffered until it is complete, evaluated, and only then delivered. The caller receives one whole answer rather than a stream, and waits for the full generation before receiving anything. Where every output-stage rule is in monitor mode the trade does not arise: evaluation runs asynchronously and streaming is unaffected. Enforcing on the output stage is therefore a deliberate exchange of perceived latency for the ability to stop a response, and it is the one architectural decision on this page that end users notice directly. ### Stage 5: recording Every rule evaluation emits a **guardrail-trigger** record, whether the rule fired or not and whether its mode was enforce or monitor. The record carries the rule and guardrail identifiers, the score, the action and mode, the stage, the content type, the execution duration, and the project and consumer context. This is a distinct record type from the Audit Logs, which capture administrative mutations such as a change *to* a guardrail rather than the rule firing. Where those records go is a data-protection question in its own right, because a trigger record can contain the matched content, and the copy the data plane's own collector produces always does. The management-plane copy is the one that can be scoped, through the Admin Dashboard's **Settings → Guardrails** card, to full content, metadata only, or off. A backend subscribed to the `guardrailtrigger` signal receives full content regardless of that setting, by design, so that a strict management-plane setting cannot silently break an organisation's own security pipeline. [Custom observability backends](/agent-router-enterprise/guides/observability-and-analytics/configure-custom-observability-backends#guardrail-trigger-destinations) covers both controls. ## The request path, end to end Putting the stages in order, a single guarded request moves like this: 1. An application sends a prompt to the gateway. The Guardrails module intercepts the payload before it is forwarded (interception). 1. The module evaluates active rules against the content types each one declares: providers score the content; the module compares scores to thresholds. A provider that times out or errors is resolved by the guardrail's failure mode (evaluation). 1. On a violation, the module applies the rule's action when mode is enforce, or records the would-be action when mode is monitor. A block ends the request; a redaction re-runs the providers with rules still pending against the sanitised content (enforcement). 1. If the request was allowed or redacted, the gateway forwards it to the model provider, and the response returns through the proxy. 1. The module inspects the response with the same logic before it reaches the application, buffering the response first where any output-stage rule enforces (egress inspection). 1. Each evaluation emits a guardrail-trigger record to the data plane's collector, and to the management plane subject to the configured content mode (recording). Every step happens inside the data plane, the environment all AI traffic flows through. Local checks keep the content inside it entirely. A remote provider call is the one point at which content is sent outside, and only when such a provider is configured and selected by a rule. Who operates that data plane depends on the deployment model, and this is the question to settle before treating the paragraph above as a residency guarantee. On Agent Router Service and Agent Router Enterprise Fully Managed, Tetrate runs the data plane, so prompts and responses are processed on Tetrate-hosted infrastructure. On Agent Router Enterprise Self-Hosted Data Plane, the data plane runs in the organisation's own Kubernetes cluster and the content never leaves it. See [Deployment models](/product-architecture/deployment-models). ## Why guardrails run as a module, not per application Running guardrails inside a dynamic module rather than writing safety checks into each application has two consequences that motivate the design. - Centralised governance: every application that uses the project's gateway path is protected by the same rules automatically, with no per-application integration work. A rule changed once applies everywhere in that project the moment it takes effect. - Cost and safety at the edge: blocking a non-compliant or malicious prompt before it reaches the model means the model provider is never invoked for a request that would fail policy. The organisation does not pay for a call it never wanted to make, and unsafe content never reaches the model in the first place. ## Key terms The vocabulary introduced above, collected for reference. **Guardrail**, **Dynamic module**, and **Provider** are defined in the [Glossary](/reference/glossary) alongside the rest of the Agent Router vocabulary, and the permitted values of every rule setting are in the [guardrails reference](/reference/guardrails#rule-level-settings). | Term | Meaning | | :---- | :---- | | **Rule** | One check type plus action and mode, run by a chosen provider against input, output, or both. | | **Check type** | A name from a fixed vocabulary (`pii`, `toxicity`, `jailbreak`, and others): *what* a rule looks for. | | **Content type** | Which part of a payload a rule inspects: all, text, image, document, tool output, thinking, or context. | | **Pattern** | A named, reusable regex in a shared catalog that rules can reference by name. | | **Threshold** | Sensitivity in the range 0.0 to 1.0; the module flags a violation when the provider score is at or above it. | | **Action** | `block` or `redact`: what enforcement would do on a violation. | | **Mode** | `enforce` applies the action; `monitor` records the would-be action only. | | **Failure mode** | If a provider cannot evaluate a rule: fail-close (default) treats it as blocked; fail-open lets traffic pass. | | **Evaluation timeout** | The bound on how long evaluation may take before the failure mode decides the outcome. | | **Prompt / response** | The text sent to a model, and the text it returns. | | **Proxy** | A server that receives a request, processes it, and forwards it on. The gateway is a proxy, built on Envoy. | | **Filter / filter chain** | A single processing stage, and the ordered pipeline of such stages a request passes through in the proxy. | | **WebAssembly (Wasm)** | A portable sandboxed code format; an older, slower way to extend a proxy than a dynamic module. | | **Shared library** | A compiled file (for example a `.so`) that a program loads and runs as part of itself. | | **Inline processing** | Work performed as part of handling the request, rather than in a separate service called over the network. | | **Payload** | The content a request carries; here, the prompt or the response. | | **Personally Identifiable Information (PII)** | Data that identifies a person, such as a name, email address, phone number, or payment-card number. | | **Redaction** | Removing or masking matched content so the request can still proceed without the sensitive text. | | **Callout** | A pause in processing to call a remote guardrail provider and resume on its reply. | | **Prompt injection / jailbreak** | Text that tries to subvert the model; a jailbreak is a direct injection in the user's own prompt. | | **Egress** | Traffic leaving toward its destination; here, the model's response inspected on its way back to the application. | | **Guardrail-trigger record** | The record one rule evaluation emits, distinct from an Audit Log entry, which records a change to a guardrail. | ## What to do next The operational configuration and developer-side use of guardrails are covered in the guides below: - [About guardrails](/agent-router-enterprise/guides/operate-and-govern/safety-and-data-protection/about-guardrails) is the operator's conceptual view of the same machinery: the objects, the rule form, and how each engine detects. - [Guardrails reference](/reference/guardrails) is the lookup surface: the 23 check types, every rule-level setting, and the provider-by-check-type matrix. - [Configure guardrails](/agent-router-enterprise/guides/operate-and-govern/safety-and-data-protection/configure-custom-guardrails) and [Configure guardrail providers](/agent-router-enterprise/guides/operate-and-govern/safety-and-data-protection/configure-vendor-guardrails) set up the policies and engines the module enforces. - [Detect and redact sensitive data](/agent-router-enterprise/guides/operate-and-govern/safety-and-data-protection/detect-and-redact-sensitive-data) and [Detect and block prompt injection](/agent-router-enterprise/guides/operate-and-govern/safety-and-data-protection/detect-and-block-prompt-injection) cover common check workflows. - [Protect requests with guardrails](/agent-router-enterprise/guides/operate-and-govern/safety-and-data-protection/protect-requests-with-guardrails) is the developer-side view. Where to go next Configure guardrails Set up the policies and rules the module enforces inline. Data flows Where guardrail evaluation sits on the request path through the gateway.