Guardrails

Guardrails in Agent Router Enterprise run inline in the gateway data plane as a dynamic module, inspecting the prompt on its way to a model provider and the response on its way back. A guardrail is a project-scoped policy: a set of rules, each pairing a check type (what to detect) with a provider (how it is detected) and a stage (where on the request path it runs). This page is the capability reference: guardrail and rule settings, the 23 check types, the five provider kinds with their latency and cost profiles, the full matrix of which provider supports which check type on which stage, how write-time validation rejects combinations that could never fire, the 14 built-in templates, and the design characteristics of the runtime.

---

Where guardrails run

Evaluation happens in process, as a dynamic module in the gateway data plane, with no sidecar and no extra network hop. Every request that crosses the gateway is inspected on the way in (the prompt, before it reaches any model provider) and on the way out (the model's response, before it reaches the application).

Guardrails inspect large language model (LLM) traffic. Model Context Protocol (MCP) traffic is also proxied by the gateway, but standalone MCP profile requests do not pass through content guardrails; they are governed by identity, access control, and audit. See Guardrails for the request-path detail.

Running at the gateway rather than in application code has two consequences:

• Centralised governance: one policy protects every application in the project, with no per-application safety code and no software development kit (SDK) integration.
• Cost at the edge: a prompt blocked at the gateway never reaches the model provider, so no charge is incurred for a call that would have failed policy.

The core design split drives everything below:

• A check type states what to look for (personally identifiable information, jailbreak, toxicity, and others). There are 23.
• A provider states how it is detected (regular expression engine, keyword pipeline, machine-learning classifiers, external service). There are five provider kinds.
• A stage states where on the request path the rule runs: input, output, or both.
• Any check type can in principle be served by different providers, so the cost and accuracy point is selected per rule. Providers score, the gateway decides: a provider returns a normalised score between 0.0 and 1.0, and the gateway compares it to the rule's threshold and applies the action.
• Not every combination of the three is implemented. The capability matrix is the authority, and the API validates every rule against it at write time.

Anatomy of a guardrail

A guardrail is a named, project-scoped policy: a collection of rules plus shared settings. A rule is one check type, one provider, an action, a mode, and a stage.

Guardrail-level settings

Setting	Values	Meaning
Type	compliance, security, policy, quality, custom, evaluator	What class of concern the guardrail addresses
Severity	low, medium, high, critical	Operational severity of a trigger
Category	data_privacy, financial, healthcare, government, custom	Domain grouping
Failure mode	fail_close (default), fail_open	If evaluation itself fails (provider down, timeout): fail-close blocks the request, fail-open lets it pass. The setting is per path: fail-close where safety dominates, fail-open where availability dominates
Evaluation timeout	milliseconds	Per-guardrail evaluation budget (the built-in templates use 3000 to 5000 ms)
Triggered response	message template	What the caller sees on a violation; supports {{policy_name}}, {{rule_name}}, and {{violation_details}} substitution

Rule-level settings

Setting	Values	Meaning
Check type	one of the 23 listed below	What to detect
Provider	one of the configured provider instances	The engine that scores the content
Action	block, redact	Block stops the request with HTTP 403, plus an optional message and a correlation ID for investigation. Redact masks the matched content (for example [CREDENTIAL_REDACTED]) and lets the request continue
Mode	enforce, monitor	Enforce applies the action. Monitor is shadow mode: it records what would have happened and passes content unchanged, which surfaces trigger and false-positive rates on live traffic before enforcement. Monitor rules never break the request flow and are always effectively fail-open
Stage	input, output, or both	Run against the prompt, the response, or both. Labelled Run on in the Admin Dashboard and execute_on in the API. Leaving it unset requests both stages, so a check supported on only one stage must have the stage set explicitly or the write is rejected
Content type	all, text, image, document, tool, thinking, context	Which kind of content an evaluation ran against. Carried on the evaluation and on the resulting trigger record, not set on the rule in the Admin Dashboard
Target	project	Rules apply to the guardrail's whole project
Threshold	0.0 to 1.0	Violation sensitivity: the gateway flags a violation when the provider score is at or above the threshold (deterministic engines score 1.0 or 0.0)

Streaming behaviour

• If every rule on a stage is monitor mode, evaluation is asynchronous and streaming is unaffected.
• If any rule on the output stage enforces, the response is buffered for evaluation before delivery, so enforcing output guardrails de-stream the response. This is a deliberate correctness trade-off; the operational implications are covered in Guardrails best practices and FAQ.

Check types

23 check types, in four groups. Each name below is the value used in the Admin Dashboard; the corresponding API constant is CHECK_TYPE_ followed by the upper-cased name, so pii is CHECK_TYPE_PII and factual_consistency is CHECK_TYPE_FACTUAL_CONSISTENCY.

Privacy and secrets

Check type	Detects
pii	Personally identifiable information: names, emails, phone numbers, national identifiers, payment cards, IBANs, and similar
secrets	Credentials and API keys (AWS, GitHub, Slack, generic API keys, and others)
protected_material	Copyrighted or otherwise protected content in outputs

Security

Check type	Detects
prompt_injection	Instructions smuggled into prompts or context to subvert the model, including system-prompt extraction and data exfiltration
jailbreak	Direct "ignore previous instructions" and role-play attacks in the user's own prompt
ban_code	Executable or malicious code in prompts or responses, including SQL injection payloads, detected by denylist and pattern. Available on the deterministic engines only
code	Executable or malicious code, additionally available on the tetrate classifier

Content safety

Check type	Detects
toxicity	Toxic, profane, abusive language
hate_speech	Hate speech targeting protected groups
sexual_content	Sexual content, including child-safety violations
violence	Violent content and threats
self_harm	Self-harm and suicide-related content
bias	Biased or discriminatory language (gender, age, disability, and others)

Policy, quality, and control

Check type	Detects
compliance	Regulation-specific prohibited practices (see the compliance templates below)
banned_topics	Configurable off-limits subject matter (for example medical, legal, or financial advice)
ban_competitors	Competitor names, recommendations, and comparisons
banned_substrings	Literal string denylists
regexp	Arbitrary custom patterns from the shared Patterns catalog
language	Language allow and deny lists (28 languages)
gibberish	Nonsense or garbled input
sentiment	Negative-sentiment content
relevance	Response relevance to the prompt
factual_consistency	Groundedness of a response against source material

Detection engines (providers)

Five provider kinds, compared by mechanism, deployment, and cost:

Provider kind	Detection mechanism	ML-based?	Where it runs	Data leaves the customer environment?	Latency and cost profile
regexp	RE2 pattern matching against a named, reusable Patterns catalog (custom patterns supported)	No, deterministic	In process, inside the gateway	No	Near zero, free (about 0 ms)
builtin	Five-layer keyword and regex pipeline: blocked keywords, then an allowlist gate, then subject and term co-occurrence within a sentence, then detection patterns, then category keywords, severity-banded	No, deterministic	In process, inside the gateway	No	Near zero, free (sub-millisecond)
tetrate	Tetrate Semantic Router: an embedding model plus task-specific small-model classifiers, comprising a domain classifier (violence, self-harm, hate speech, sexual content, weapons, crime), a PII classifier, a jailbreak classifier, a natural language inference (NLI) entailment fallback (ModernBERT class), a gibberish detector, and language identification. Cascading evaluation: local known patterns, then the ML classifier, then the NLI fallback, short-circuiting at the first firing layer	Yes, small-language-model and classifier based	In the customer cluster: a Helm-deployed workload in the customer data plane; CPU inference, no GPU required	No, the callout stays inside the customer network	Slower: in-cluster network hop plus inference, typically 40 to 120 ms; compute cost is the customer's own cluster capacity
azurecontentsafety	Azure AI Content Safety REST API: text analysis (hate, sexual, violence, and self-harm severity), Prompt Shields (jailbreak), protected-material detection, groundedness detection	Yes, hosted ML	Third-party SaaS (Microsoft)	Yes, content is sent to Azure	Slowest and metered: external network round trip (about 200 ms class) plus per-call vendor pricing
externalservice	Generic HTTP callout to any guardrail service the customer runs or subscribes to; its supported checks are declared per instance	Depends on the service	Wherever it is pointed	Yes, to that endpoint	Depends on the service

Provider availability and evaluation order

• regexp, builtin, and tetrate ship by default as globally seeded providers, immutable through the API. azurecontentsafety and externalservice are operator-configured integrations.
• The kinds form a cost and capability ladder: deterministic engines catch known formats and phrasings for free, ML engines catch paraphrase, novel phrasing, and semantic evasion at a latency and compute cost, and third-party engines add vendor capabilities at per-call prices and a data-egress trade-off.
• Cheap-decisive-first ordering: the gateway hoists enforced block rules on the in-process engines ahead of everything else, so a local pattern hit short-circuits evaluation before any remote or ML call runs. ML latency and vendor fees are not incurred for requests a regular expression would have blocked anyway.
• One batched call per provider: N rules on one provider cost one evaluation call, not N. Results are cached per content snapshot and invalidated only when a redaction rewrites the content.
• Some combinations of check type, provider, and stage are not implemented, and are restricted by the capability matrix below.

Provider support by check type

A rule is valid only if its provider kind supports its check type on the stage the rule requests. The constraint is enforced at write time, not at request time, and the Admin Dashboard mirrors it by scoping the check-type dropdown to the selected provider.

Each cell gives the stages the (provider kind, check type) pair supports:

• Both: the check runs on input, output, or both, so the stage may be left unset.
• Input only or Output only: the check runs on that stage alone. Because an unset stage requests both, such a rule must set Run on explicitly or it is rejected.
• No: the provider kind does not implement the check, and rules combining them are rejected.

The four kinds with fixed capabilities are shown. externalservice declares its capabilities per instance and is covered below.

Check type	regexp	builtin	tetrate	azurecontentsafety	tetrate detection mechanism
ban_code	Both	Both	No	No	Not supported
ban_competitors	Both	Both	No	No	Not supported
banned_substrings	Both	Both	Both	No	Keyword signal
banned_topics	Both	Both	Both	No	NLI entailment
bias	Both	Both	Output only	No	NLI entailment
code	Both	Both	Both	No	Classifier
compliance	Both	Both	No	No	Not supported
factual_consistency	Both	No	No	Output only	Not supported
gibberish	Both	No	Both	No	NLI or classifier
hate_speech	Both	Both	Both	Both	Signal evaluation
jailbreak	Both	Both	Input only	Input only	Security classifier, with NLI fallback
language	Both	Both	Both	No	Language signal
pii	Both	Both	Both	No	PII model, returning entities and masked text
prompt_injection	Both	Both	Input only	Input only	Known patterns, then classifier, then NLI
protected_material	Both	Both	No	Both	Not supported
regexp	Both	Both	Both	No	Catalog regex signal
relevance	Both	No	Output only	No	NLI against the original prompt
secrets	Both	Both	Input only	No	Keyword signal
self_harm	Both	Both	Both	Both	Signal evaluation
sentiment	Both	No	Both	No	NLI entailment
sexual_content	Both	Both	Both	Both	Signal evaluation
toxicity	Both	Both	Both	Both	Signal evaluation (violence and hate)
violence	Both	Both	Both	Both	Signal evaluation

Five qualifications apply when reading the table:

• regexp accepts every check type on both stages only because a regular expression is check-type-agnostic. The check type is a label, and the referenced patterns are the real definition, so a pii rule and a secrets rule on this provider behave identically given identical patterns.
• builtin runs its keyword and regex pipeline on both stages for every check it supports. It does not implement the meaning-based checks (gibberish, relevance, sentiment) or factual_consistency.
• tetrate carries stage limits on five checks: prompt_injection, jailbreak, and secrets on input only, bias and relevance on output only.
• azurecontentsafety covers the nine checks shown, with prompt_injection and jailbreak on input only and factual_consistency (groundedness) on output only.
• factual_consistency is therefore not available on any in-cluster provider except nominally through regexp. Groundedness checking requires azurecontentsafety or an external service that implements it.

The table is derived from the canonical capability declaration that the API validates against and the Admin Dashboard uses to scope its forms. Where this page and the product disagree, the API's error message is authoritative.

External guardrail services: per-instance capabilities

An externalservice provider wraps a remote guardrail service the customer operates. Its capabilities are declared per instance, not per kind:

• Checks are declared in the provider instance's configuration as a list of canonical check names. A rule write against that provider is validated against the instance's declared list rather than against a kind-wide set.
• Stages are unrestricted at write time. The gateway forwards the stage to the remote service, which decides what it evaluates.

Provider creation enforces the declaration: an externalservice provider with no valid, non-empty check list, drawn from the check-type vocabulary above, is rejected.

Write-time validation

Creating or updating a guardrail rule or provider is validated against the matrix. The API rejects the write with an invalid-argument error naming the supported set:

Write	Rejected when
Create or update a rule	The check type is not supported by the provider's kind or, for externalservice, is absent from the instance's declared check list. The error lists the checks the provider does support
Create or update a rule	The requested stage, where unset means both, falls outside the (kind, check) pair's supported stages. The error names the supported stages
Create or update a rule	The referenced provider's kind is unknown
Create a provider	The kind is not one of regexp, builtin, tetrate, azurecontentsafety, or externalservice
Create an externalservice provider	The configuration's check list is missing, empty, or contains names outside the canonical check vocabulary

Updates re-validate whenever the provider, check type, stage, or configuration changes. Edits confined to unrelated fields (name, description, active) do not re-trigger validation, which is what keeps legacy rules editable.

Because the Admin Dashboard scopes the rule form to the same matrix, these rejections surface mainly through direct API use and automation, where the error fails an apply fast instead of shipping a rule that can never fire.

Troubleshooting: rules that never fire

Rules created before write-time validation existed were never checked against the matrix, and they are grandfathered: they keep working exactly as before. A grandfathered rule whose (provider kind, check type, stage) combination is unsupported nevertheless never fires, because the provider returns a zero score for the unsupported check or stage and the rule is a silent no-op.

Where a guardrail does not fire on content that should trigger it:

1. Check the rule against the matrix above. Confirm that the provider kind supports the check type, and that the rule's stage, remembering that unset means both, falls within the supported stages.
2. Check the control-plane logs. The control plane logs a warning, including the rule ID, provider kind, check type, and stage, every time it ships a rule to the data plane that the matrix says can never fire. Searching for that warning is the fastest way to find pre-existing no-op rules.
3. Check the Admin Dashboard. A stored check type the provider does not support is shown as disabled in the rule's form.

Editing a grandfathered rule repairs it: changing its provider, check type, stage, or configuration re-triggers validation, so the edit is held to the matrix and the corrected rule is guaranteed to be a combination that can fire. Edits limited to name, description, or active status leave the rule untouched by validation.

Built-in guardrail templates

Agent Router Enterprise ships 14 ready-made template guardrails (is_template=true). Creating a guardrail from a template clones its rules into the project, where every setting can then be tuned. All template rules ship with action = redact, mode = monitor, an observe-first default: a template rule records what it would have done until it is promoted to enforce, and to block where appropriate. The rollout procedure is covered in Configure guardrails.

Data privacy and PII (regex-based: deterministic, in process, about 0 ms)

All rules use the pii and secrets check types on the regexp provider with named pattern catalogs, on both input and output, redacting matches in place (for example [CREDENTIAL_REDACTED], [{pattern_name}_REDACTED]).

Template	Severity and failure mode	Coverage
Baseline PII Protection	critical, fail-close	Minimal set for internal tools: tax identifiers, credentials and API keys (AWS, GitHub, Slack, generic), payment cards and IBAN
GDPR EU PII Protection	critical, fail-close	EU national identifiers, financial data, contact information, business identifiers (GDPR Art. 32)
Canadian PIPEDA PII Protection	critical, fail-close	SIN, OHIP and driver licenses, passports and immigration documents, institutional IDs, contact and financial data (PIPEDA, FIPPA)
Australian PII Protection	critical, fail-close	TFN, ABN, Medicare, passports, international IDs, financial data, credentials, network infrastructure identifiers, protected-class information
Singapore PDPA Compliance (PII half)	critical, fail-close	NRIC-class identifiers, contact, financial, and business identifiers

Regulatory compliance (builtin keyword pipeline: deterministic, in process, sub-millisecond)

All rules use the compliance or banned_topics check type on the builtin provider's five-layer keyword pipeline, on both input and output unless noted.

Template	Severity and failure mode	Coverage
EU AI Act Article 5 Compliance	critical, fail-close	Prohibited practices: manipulative AI, vulnerability exploitation, social scoring, emotion recognition, biometric profiling, with parallel rule sets in English and French
Singapore MAS AI Governance	critical, fail-close	MAS AI risk-management guidelines for financial institutions: fairness and bias assessment, transparency and explainability, human oversight, data governance, model security controls
Singapore PDPA Compliance (policy half)	critical, fail-close	Sensitive data categories, do-not-call registry, cross-border data transfer, profiling and automated decisions
UAE Regulatory Compliance	high, fail-close	Anti-discrimination law and cultural-sensitivity requirements
Healthcare Claims Compliance	critical, fail-close	Claims-processing safety: fraud coaching, disclosure of protected health information (PHI), prior-authorization gaming, system-override attempts, unauthorized medical advice

Security (builtin, input stage, deterministic, in process)

Template	Severity and failure mode	Coverage
Prompt Injection Detection	critical, fail-close	Input-only rules for prompt injection, system-prompt extraction, data exfiltration, jailbreaks, malicious code injection, and SQL injection (prompt_injection, jailbreak, and ban_code check types)

Content and business policy (builtin, deterministic, in process)

Template	Severity and failure mode	Coverage
NSFW Content Safety (Global)	high, fail-close	Toxicity in five language variants (English, Spanish, French, German, Australian English), plus violence, self-harm, and child-safety rules (toxicity, violence, self_harm, sexual_content)
Bias & Discrimination Protection	high, fail-open	Gender, racial, religious, sexual-orientation, age, and disability bias (bias, hate_speech)
Professional Advice Guardrails	high, fail-open	Output-only rules preventing unauthorized legal, financial, and medical advice (banned_topics)
Competitor Mention Detection	medium, fail-open	Paired input and output rules for competitor references, recommendations, and comparisons (ban_competitors)

The same check types can be re-pointed at the ML providers where semantic coverage is needed. A jailbreak rule on the tetrate provider uses its jailbreak classifier with NLI fallback instead of, or layered behind, the pattern-based template rule.

Design characteristics

Characteristic	Description
In-gateway, in-process evaluation	a dynamic module in the data plane, rather than an SDK embedded per application or a separate proxy-of-a-proxy service hop
ML detection without data egress	The tetrate provider runs embedding and classifier models inside the customer cluster on CPU, so semantic detection (jailbreak, PII, toxicity, and the NLI-backed checks) sends no prompt to a third party. Third-party engines (Azure AI Content Safety, external services) are available and are an explicit opt-in per rule
Combinations validated before they ship	The API rejects a rule whose provider kind cannot serve its check type on its stage, so a rule that could never fire cannot be created. The Admin Dashboard scopes its forms to the same matrix
A cost ladder controlled per rule	The same check type can be served by a free in-process engine or by an ML engine. The gateway auto-orders cheap decisive checks first so local hits short-circuit expensive calls, and batches all of a provider's checks into one call
Redaction as well as blocking	Rules can mask matched content in place, with configurable redaction formats, and let traffic flow, on inputs and on outputs, instead of only rejecting requests
Shadow mode on every rule	Any rule can run in audit-only mode against live traffic before enforcement. Monitor evaluation is asynchronous and never affects latency or streaming
Deliberate failure semantics	Fail-close, fail-open, and the evaluation timeout are explicit per-guardrail settings, so safety-critical paths can fail closed while availability-critical paths fail open
Investigation support	Blocked responses carry a correlation ID. Operators holding the read-only guardrail-reader role can look up which rules fired, the scores, and the offending content, and join against their own logging by correlation ID
14 ready-made templates	Region-specific and regulation-specific packs (GDPR, PIPEDA, PDPA, MAS, EU AI Act Article 5, UAE, Australian privacy, healthcare claims) plus security and content-safety packs, multilingual where relevant (EU AI Act in English and French, toxicity in five language variants), cloneable and fully editable per project

---

Where to go next

About guardrails

The object model: policies, rules, providers, patterns, and thresholds.

Configure guardrails

Create project policies and rules, and validate in monitor mode before enforcing.

Configure guardrail providers

How the scoring engines differ in detection approach, latency, and configuration.

Guardrails best practices and FAQ

Rollout order, latency budgets, failure modes, and investigation.

Guardrails architecture

Where guardrail evaluation sits on the request path through the gateway.

Guardrails API reference

Request and response detail for the guardrail and rule management API. Providers, patterns, and triggers have their own API references.