Guardrails

Guardrails in Agent Router Enterprise run inline in the gateway data plane as a dynamic module, inspecting the prompt on its way to a model provider and the response on its way back. A guardrail is a project-scoped policy: a set of rules, each pairing a check type (what to detect) with a guardrail provider (how it is detected) and a stage (where on the request path it runs).

---

Where guardrails run

Evaluation happens in process, as a dynamic module in the gateway data plane, with no sidecar and no extra network hop. Every request that crosses the gateway is inspected on the way in (the prompt, before it reaches any model provider) and on the way out (the model's response, before it reaches the application).

Guardrails inspect large language model (LLM) traffic. Model Context Protocol (MCP) traffic is also proxied by the gateway, but standalone MCP profile requests do not pass through content guardrails; they are governed by identity, access control, and audit. See Guardrails for the request-path detail.

Running at the gateway rather than in application code has two consequences:

• Centralized governance: one policy protects every application in the project, with no per-application safety code and no software development kit (SDK) integration.
• Cost at the edge: a prompt blocked at the gateway never reaches the model provider, so no charge is incurred for a call that would have failed policy.

The core design split drives everything below:

• A check type states what to look for (personally identifiable information, jailbreak, toxicity, and others). There are 23.
• A guardrail provider states how it is detected (regular expression engine, keyword pipeline, machine-learning classifiers, external service). There are five provider kinds.
• A stage states where on the request path the rule runs: input, output, or both.
• Any check type can in principle be served by different providers, so the cost and accuracy point is selected per rule. Providers score, the gateway decides: a provider returns a normalized score between 0.0 and 1.0, and the gateway compares it to the rule's threshold and applies the action.
• Not every combination of the three is implemented. The capability matrix is the authority, and the API validates every rule against it at write time.

Anatomy of a guardrail

A guardrail is a named, project-scoped policy: a collection of rules plus shared settings. A rule is one check type, one provider, an action, a mode, and a stage.

Guardrail-level settings

Setting	Values	Meaning
Type	compliance, security, policy, quality, custom, evaluator	What class of concern the guardrail addresses
Severity	low, medium, high, critical	Operational severity of a trigger
Category	data_privacy, financial, healthcare, government, custom	Domain grouping
Failure mode	fail_close (default), fail_open	If evaluation itself fails (provider down, timeout): fail-close blocks the request, fail-open lets it pass. The setting is per path: fail-close where safety dominates, fail-open where availability dominates
Evaluation timeout	milliseconds	Per-guardrail evaluation budget (the built-in templates use 3000 to 5000 ms)
Triggered response	message template	What the caller sees on a violation; supports {{policy_name}}, {{rule_name}}, and {{violation_details}} substitution

Rule-level settings

Setting	Values	Meaning
Check type	one of the 23 listed below	What to detect
Provider	one of the configured provider instances	The guardrail provider that scores the content
Action	block, redact	Block stops the request with HTTP 403, plus an optional message and a correlation ID for investigation. Redact masks the matched content (for example [CREDENTIAL_REDACTED]) and lets the request continue
Mode	enforce, monitor	Enforce applies the action. Monitor is shadow mode: it records what would have happened and passes content unchanged, which surfaces trigger and false-positive rates on live traffic before enforcement. Monitor rules never break the request flow and are always effectively fail-open
Stage	input, output, or both	Run against the prompt, the response, or both. Labeled Run on in the Admin Console and execute_on in the API. Leaving it unset requests both stages, so a check supported on only one stage must have the stage set explicitly or the write is rejected
Content type	all, text, image, document, tool, thinking, context	Which kind of content an evaluation ran against. Carried on the evaluation and on the resulting trigger record, not set on the rule in the Admin Console
Target	project	Rules apply to the guardrail's whole project
Threshold	0.0 to 1.0	Violation sensitivity: the gateway flags a violation when the provider score is at or above the threshold (deterministic engines score 1.0 or 0.0)

Streaming behavior

• If every rule on a stage is monitor mode, evaluation is asynchronous and streaming is unaffected.
• If any rule on the output stage enforces, the response is buffered for evaluation before delivery, so enforcing output guardrails de-stream the response. This is a deliberate correctness trade-off; the operational implications are covered in Guardrails best practices and FAQ.
• Buffering follows the mode, not the action. Both actions buffer when the rule enforces: redact has to rewrite content it has already seen, so it de-streams a response exactly as block does. Choosing redact over block changes what the caller receives, never whether the response streams.
• Enforcement on the input stage leaves streaming intact. Input rules run before the request is forwarded, so their cost lands on time-to-first-token rather than on the stream, which makes enforce-on-input with monitor-on-output the combination that polices prompts while keeping responses streaming.
• A blocked response is returned as a structured HTTP 403 with the error envelope, not as a dropped or truncated stream, precisely because the response was buffered before the violation was found. See Streaming behavior.

Check types

23 check types, in four groups. Each name below is the value used in the Admin Console; the corresponding API constant is CHECK_TYPE_ followed by the upper-cased name, so pii is CHECK_TYPE_PII and factual_consistency is CHECK_TYPE_FACTUAL_CONSISTENCY.

Privacy and secrets

Check type	Detects
pii	Personally identifiable information: names, emails, phone numbers, national identifiers, payment cards, IBANs, and similar
secrets	Credentials and API keys (AWS, GitHub, Slack, generic API keys, and others)
protected_material	Copyrighted or otherwise protected content in outputs

Security

Check type	Detects
prompt_injection	Instructions smuggled into prompts or context to subvert the model, including system-prompt extraction and data exfiltration
jailbreak	Direct "ignore previous instructions" and role-play attacks in the user's own prompt
ban_code	Executable or malicious code in prompts or responses, including SQL injection payloads, detected by denylist and pattern. Available on the deterministic engines only
code	Executable or malicious code, additionally available on the tetrate classifier

Content safety

Check type	Detects
toxicity	Toxic, profane, abusive language
hate_speech	Hate speech targeting protected groups
sexual_content	Sexual content, including child-safety violations
violence	Violent content and threats
self_harm	Self-harm and suicide-related content
bias	Biased or discriminatory language (gender, age, disability, and others)

Policy, quality, and control

Check type	Detects
compliance	Regulation-specific prohibited practices (see the compliance templates below)
banned_topics	Configurable off-limits subject matter (for example medical, legal, or financial advice)
ban_competitors	Competitor names, recommendations, and comparisons
banned_substrings	Literal string denylists
regexp	Arbitrary custom patterns from the shared Patterns catalog
language	Language allow and deny lists (28 languages)
gibberish	Nonsense or garbled input
sentiment	Negative-sentiment content
relevance	Response relevance to the prompt
factual_consistency	Groundedness of a response against source material

Guardrail providers

Five provider kinds, compared by mechanism, deployment, and cost:

Provider kind	Detection mechanism	ML-based?	Where it runs	Data leaves the customer environment?	Latency and cost profile
regexp	RE2 pattern matching against a named, reusable Patterns catalog (custom patterns supported)	No, deterministic	In process, inside the gateway	No	Near zero, free (about 0 ms)
builtin	Five-layer keyword and regex pipeline: blocked keywords, then an allowlist gate, then subject and term co-occurrence within a sentence, then detection patterns, then category keywords, severity-banded	No, deterministic	In process, inside the gateway	No	Near zero, free (sub-millisecond)
tetrate	Tetrate Semantic Router: an embedding model plus task-specific small-model classifiers, comprising a domain classifier (violence, self-harm, hate speech, sexual content, weapons, crime), a PII classifier, a jailbreak classifier, a natural language inference (NLI) entailment fallback (ModernBERT class), a gibberish detector, and language identification. Cascading evaluation: local known patterns, then the ML classifier, then the NLI fallback, short-circuiting at the first firing layer	Yes, small-language-model and classifier based	In the customer cluster: a Helm-deployed workload in the customer data plane; CPU inference, no GPU required	No, the callout stays inside the customer network	Slower: in-cluster network hop plus inference, typically 40 to 120 ms; compute cost is the customer's own cluster capacity
azurecontentsafety	Azure AI Content Safety REST API: text analysis (hate, sexual, violence, and self-harm severity), Prompt Shields (jailbreak), protected-material detection, groundedness detection	Yes, hosted ML	Third-party SaaS (Microsoft)	Yes, content is sent to Azure	Slowest and metered: external network round trip (about 200 ms class) plus per-call vendor pricing
externalservice	Generic HTTP callout to any guardrail service the customer runs or subscribes to; its supported checks are declared per instance	Depends on the service	Wherever it is pointed	Yes, to that endpoint	Depends on the service

Provider availability and evaluation order

• Providers are owned by a project (since 0.4.0). A provider is an instance with its own identifier, and only an instance the project owns can be referenced by one of its rules. Those instances are editable like any other row, so one kind can be configured differently from project to project.
• Four of the five kinds also exist as catalog templates: regexp, builtin, tetrate, and azurecontentsafety. A template is global, read-only, and not usable by a rule. It holds a starting configuration that an operator copies into a project to create an instance. The previously shared, deployment-wide regexp, builtin, and tetrate rows became these templates. externalservice has no template and is configured from scratch.
• A newly created project owns a regexp, a builtin, and a tetrate provider, copied from those templates when the project is created. azurecontentsafety is not among them, because it cannot evaluate anything without a credential; it stays a template until an operator creates an instance and configures one. externalservice instances are likewise created deliberately.
• A rule binds only to an enabled provider its own project owns. An identifier naming another project's provider, a catalog template, or nothing at all is refused identically, so provider identifiers cannot be probed across projects; a disabled provider is refused separately, with an error saying it is disabled. A provider that is disabled after its rules exist is skipped at evaluation, and the rules bound to it stop enforcing. Listing a project's providers returns the project's own rows unless templates are asked for, and reading providers requires the guardrail_providers.read permission or a scope that grants guardrail reads.
• A provider's credential is never part of its configuration. It enters through the write-only SetGuardrailProviderSecret call, is stored encrypted, and is returned by no read: a provider reports whether a credential is stored, not what it is. Calling the operation again rotates the credential in place, and deleting the provider or its project destroys it. Creating or updating a provider with a credential inside the configuration is refused with an error naming the operation.
• The kinds form a cost and capability ladder: deterministic engines catch known formats and phrasings for free, ML engines catch paraphrase, novel phrasing, and semantic evasion at a latency and compute cost, and third-party engines add vendor capabilities at per-call prices and a data-egress trade-off.
• Cheap-decisive-first ordering: the gateway hoists enforced block rules on the in-process engines ahead of everything else, so a local pattern hit short-circuits evaluation before any remote or ML call runs. ML latency and vendor fees are not incurred for requests a regular expression would have blocked anyway.
• One call per guardrail, per stage, per provider, per string: the rules of one guardrail that share a provider cost one evaluation call on each stage, not one per rule, and two guardrails on the same provider cost two. An externalservice provider takes every string of the request in one call instead, plus one call per item after a redaction. Results are cached per content snapshot and invalidated only when a redaction rewrites the content.
• Some combinations of check type, provider, and stage are not implemented, and are restricted by the capability matrix below.

Provider support by check type

A rule is valid only if its provider kind supports its check type on the stage the rule requests. The constraint is enforced at write time, not at request time, and the Admin Console mirrors it by scoping the check-type dropdown to the selected provider.

Each cell gives the stages the (provider kind, check type) pair supports:

• Both: the check runs on input, output, or both, so the stage may be left unset.
• Input only or Output only: the check runs on that stage alone. Because an unset stage requests both, such a rule must set Run on explicitly or it is rejected.
• No: the provider kind does not implement the check, and rules combining them are rejected.

The four kinds with fixed capabilities are shown. externalservice declares its capabilities per instance and is covered below.

Check type	regexp	builtin	tetrate	azurecontentsafety	tetrate detection mechanism
ban_code	Both	Both	No	No	Not supported
ban_competitors	Both	Both	No	No	Not supported
banned_substrings	Both	Both	Both	No	Keyword signal
banned_topics	Both	Both	Both	No	NLI entailment
bias	Both	Both	Output only	No	NLI entailment
code	Both	Both	Both	No	Classifier
compliance	Both	Both	No	No	Not supported
factual_consistency	Both	No	No	Output only	Not supported
gibberish	Both	No	Both	No	NLI or classifier
hate_speech	Both	Both	Both	Both	Signal evaluation
jailbreak	Both	Both	Input only	Input only	Security classifier, with NLI fallback
language	Both	Both	Both	No	Language signal
pii	Both	Both	Both	No	PII model, returning entities and masked text
prompt_injection	Both	Both	Input only	Input only	Known patterns, then classifier, then NLI
protected_material	Both	Both	No	Both	Not supported
regexp	Both	Both	Both	No	Catalog regex signal
relevance	Both	No	Output only	No	NLI against the original prompt
secrets	Both	Both	Input only	No	Keyword signal
self_harm	Both	Both	Both	Both	Signal evaluation
sentiment	Both	No	Both	No	NLI entailment
sexual_content	Both	Both	Both	Both	Signal evaluation
toxicity	Both	Both	Both	Both	Signal evaluation (violence and hate)
violence	Both	Both	Both	Both	Signal evaluation

Five qualifications apply when reading the table:

• regexp accepts every check type on both stages only because a regular expression is check-type-agnostic. The check type is a label, and the referenced patterns are the real definition, so a pii rule and a secrets rule on this provider behave identically given identical patterns.
• builtin runs its keyword and regex pipeline on both stages for every check it supports. It does not implement the meaning-based checks (gibberish, relevance, sentiment) or factual_consistency.
• tetrate carries stage limits on five checks: prompt_injection, jailbreak, and secrets on input only, bias and relevance on output only.
• azurecontentsafety covers the nine checks shown, with prompt_injection and jailbreak on input only and factual_consistency (groundedness) on output only.
• factual_consistency is therefore not available on any in-cluster provider except nominally through regexp. Groundedness checking requires azurecontentsafety or an external service that implements it.

The table is derived from the canonical capability declaration that the API validates against and the Admin Console uses to scope its forms. Where this page and the product disagree, the API's error message is authoritative.

Azure AI Content Safety: configuration and credential

An azurecontentsafety provider is configured with four settings. Only the endpoint is required; each of the other three is applied at its default when left blank.

Setting	Required	Default	Description
endpoint	Yes	none	The Azure resource endpoint, as an absolute http or https URL, for example https://<resource>.cognitiveservices.azure.com. A value with no scheme is rejected at write time. A trailing slash is stripped; a path segment is accepted and then prefixed onto every Azure request, which fails at call time rather than at write time
api_version	No	2024-09-01	The Azure API version for severity analysis, Prompt Shields, and protected-material detection. Passed through unvalidated
groundedness_api_version	No	2024-09-15-preview	The Azure API version for groundedness detection, which Azure versions separately. Passed through unvalidated
timeout	No	10s	Per-request budget for the call to Azure, written as a duration such as 10s, 1500ms, or 1m30s. A bare number is not a duration. The value is not validated at write time, and the gateway cannot build a provider whose timeout it cannot parse, so a malformed one leaves every rule bound to that provider evaluating nothing

The credential is not one of these settings and is never part of a provider's configuration:

• It is set through a dedicated write-only operation, a POST to the provider's secret sub-resource, which is the only way a credential enters. The equivalent command is tare api guardrails providers set-secret.
• A configuration carrying one is rejected. A create or update that places api_key in an azurecontentsafety provider's configuration, or auth_token in an externalservice provider's, fails with an invalid-argument error naming the credential operation. Nothing is silently consumed and nothing is silently discarded.
• No read returns it. Listing providers, reading one, and the configuration delivered to the data plane all omit it. A read reports whether a credential is stored, never its value.
• Calling the operation again rotates in place. The provider keeps its identifier, its configuration, and every rule bound to it. The operation is audit-logged, recording who set a credential on which provider and never the value.
• There is no operation that clears one. A stored credential is destroyed by deleting the provider or the project that owns it.
• A template holds none. The operation is refused on a catalog template, which is read-only.

How the credential reaches the gateway, and what a rotation costs in propagation time, is described in Configuration propagation.

External guardrail services: per-instance capabilities

An externalservice provider wraps a remote guardrail service the customer operates. Its capabilities are declared per instance, not per kind:

• Checks are declared in the provider instance's configuration as a list of canonical check names. A rule write against that provider is validated against the instance's declared list rather than against a kind-wide set.
• Stages are unrestricted at write time. The gateway forwards the stage to the remote service, which decides what it evaluates.

The service implements the version 1 service contract: the gateway sends one POST carrying the stage, a list of checks, and a list of content items, and the service answers with exactly one result per pair of check and item, a score between 0.0 and 1.0 or a statement that the pair was skipped. The contract has no verdict field, because providers score and the gateway decides. Any answer other than HTTP 200 with a well-formed body is an evaluation failure, handled by the guardrail's failure mode. A vendor product that does not implement the contract natively is reached through an adapter that does.

Provider creation enforces the declaration: an externalservice provider with no valid, non-empty check list, drawn from the check-type vocabulary above, is rejected.

The service only ever sees content as it stands after every earlier rule: content an earlier rule blocked is never sent, and content an earlier rule redacted is sent masked. The wire format is the guardrail service contract, and the provider is set up by following Connect an external guardrail service.

Write-time validation

Creating or updating a guardrail rule or provider is validated against the matrix. Most rejections are an invalid-argument error naming the supported set. Three are not, and automation that branches on the code should expect them: a rule naming a provider that is a template, belongs to another project, does not exist, or is disabled is a failed-precondition, and setting a credential on a catalog template is a permission-denied.

Write	Rejected when
Create or update a rule	The check type is not supported by the provider's kind or, for externalservice, is absent from the instance's declared check list. The error lists the checks the provider does support
Create or update a rule	The requested stage, where unset means both, falls outside the (kind, check) pair's supported stages. The error names the supported stages
Create or update a rule	The referenced provider's kind is unknown
Create or update a rule	The referenced provider is a catalog template, belongs to another project, or does not exist. All three are refused identically, stating the provider is not available in this project
Create or update a rule	The referenced provider is disabled
Create a provider	The kind is missing, or names no engine the data plane implements. The error lists the accepted set, which is the authority: it is a superset of the five kinds documented above, because it also carries engines that are present in the data plane but not offered. The kind is fixed at creation and cannot be changed afterwards
Create or update a provider	The configuration carries a credential for the kind: api_key on azurecontentsafety, auth_token on externalservice. The error names the credential operation to use instead
Create or update a provider	The configuration carries the key the data plane reads the credential file from: api_key_file on azurecontentsafety, auth_token_file on externalservice. Only the management plane sets it; the error names the credential operation to use instead
Create or update an azurecontentsafety provider	The configuration's endpoint is missing, empty, or is not an absolute http or https URL. The error names the expected shape
Create or update an externalservice provider	The configuration's check list is missing, empty, or contains names outside the canonical check vocabulary
Create or update an externalservice provider	The configuration carries a key outside service_url, checks, endpoint, timeout, ca_cert_pem, and name, or a value other than checks that is not a string
Create or update an externalservice provider	service_url is missing, is not an absolute http or https URL, carries userinfo, a query, a fragment, or a trailing slash, or names a link-local or cloud-metadata address
Create or update an externalservice provider	endpoint is blank or does not start with /; timeout is blank, is not a duration, or is not greater than 0s and at most 30s. To take the default, leave the key out
Create or update an externalservice provider	ca_cert_pem is empty, is larger than 8 KiB, holds more than 4 certificates, or holds anything other than plain CERTIFICATE blocks. The error never repeats the value
Create or update a rule on an externalservice provider	The raw configuration's threshold is not a number, or is not greater than 0 and at most 1
Create or update a rule	The configuration uses the deprecated externalService variant, for any provider kind. A stored rule that carries it stays editable as long as the write does not send it back; per-check parameters belong in the raw configuration
Set a credential	The target is a catalog template, which is read-only, or its kind takes no credential

Updates re-validate whenever the provider, check type, stage, or configuration changes. Edits confined to unrelated fields (name, description, active) do not re-trigger validation, which is what keeps legacy rules editable. The credential check is the exception: every update that carries a configuration is screened for a credential key, whether or not the configuration changed.

Because the Admin Console scopes the rule form to the same matrix, these rejections surface mainly through direct API use and automation, where the error fails an apply fast instead of shipping a rule that can never fire.

Troubleshooting: rules that never fire

Rules created before write-time validation existed were never checked against the matrix, and they are grandfathered: they keep working exactly as before. A grandfathered rule whose (provider kind, check type, stage) combination is unsupported nevertheless never fires, because the provider returns a zero score for the unsupported check or stage and the rule is a silent no-op.

Where a guardrail does not fire on content that should trigger it:

1. Check the rule against the matrix above. Confirm that the provider kind supports the check type, and that the rule's stage, remembering that unset means both, falls within the supported stages.
2. Check the management-plane logs. The management plane logs a warning, including the rule ID, provider kind, check type, and stage, every time it ships a rule to the data plane that the matrix says can never fire. Searching for that warning is the fastest way to find pre-existing no-op rules.
3. Check the Admin Console. A stored check type the provider does not support is shown as disabled in the rule's form.

Editing a grandfathered rule repairs it: changing its provider, check type, stage, or configuration re-triggers validation, so the edit is held to the matrix and the corrected rule is guaranteed to be a combination that can fire. Edits limited to name, description, or active status leave the rule untouched by validation.

Built-in guardrail templates

Agent Router Enterprise ships 14 ready-made template guardrails (is_template=true). Creating a guardrail from a template clones its rules into the project, where every setting can then be tuned. All template rules ship with action = redact, mode = monitor, an observe-first default: a template rule records what it would have done until it is promoted to enforce, and to block where appropriate. The rollout procedure is covered in Configure guardrails.

Data privacy and PII (regex-based: deterministic, in process, about 0 ms)

All rules use the pii and secrets check types on the regexp provider with named pattern catalogs, on both input and output, redacting matches in place (for example [CREDENTIAL_REDACTED], [{pattern_name}_REDACTED]).

Template	Severity and failure mode	Coverage
Baseline PII Protection	critical, fail-close	Minimal set for internal tools: tax identifiers, credentials and API keys (AWS, GitHub, Slack, generic), payment cards and IBAN
GDPR EU PII Protection	critical, fail-close	EU national identifiers, financial data, contact information, business identifiers (GDPR Art. 32)
Canadian PIPEDA PII Protection	critical, fail-close	SIN, OHIP and driver licenses, passports and immigration documents, institutional IDs, contact and financial data (PIPEDA, FIPPA)
Australian PII Protection	critical, fail-close	TFN, ABN, Medicare, passports, international IDs, financial data, credentials, network infrastructure identifiers, protected-class information
Singapore PDPA Compliance (PII half)	critical, fail-close	NRIC-class identifiers, contact, financial, and business identifiers

Regulatory compliance (builtin keyword pipeline: deterministic, in process, sub-millisecond)

All rules use the compliance or banned_topics check type on the builtin provider's five-layer keyword pipeline, on both input and output unless noted.

Template	Severity and failure mode	Coverage
EU AI Act Article 5 Compliance	critical, fail-close	Prohibited practices: manipulative AI, vulnerability exploitation, social scoring, emotion recognition, biometric profiling, with parallel rule sets in English and French
Singapore MAS AI Governance	critical, fail-close	MAS AI risk-management guidelines for financial institutions: fairness and bias assessment, transparency and explainability, human oversight, data governance, model security controls
Singapore PDPA Compliance (policy half)	critical, fail-close	Sensitive data categories, do-not-call registry, cross-border data transfer, profiling and automated decisions
UAE Regulatory Compliance	high, fail-close	Anti-discrimination law and cultural-sensitivity requirements
Healthcare Claims Compliance	critical, fail-close	Claims-processing safety: fraud coaching, disclosure of protected health information (PHI), prior-authorization gaming, system-override attempts, unauthorized medical advice

Security (builtin, input stage, deterministic, in process)

Template	Severity and failure mode	Coverage
Prompt Injection Detection	critical, fail-close	Input-only rules for prompt injection, system-prompt extraction, data exfiltration, jailbreaks, malicious code injection, and SQL injection (prompt_injection, jailbreak, and ban_code check types)

Content and business policy (builtin, deterministic, in process)

Template	Severity and failure mode	Coverage
NSFW Content Safety (Global)	high, fail-close	Toxicity in five language variants (English, Spanish, French, German, Australian English), plus violence, self-harm, and child-safety rules (toxicity, violence, self_harm, sexual_content)
Bias & Discrimination Protection	high, fail-open	Gender, racial, religious, sexual-orientation, age, and disability bias (bias, hate_speech)
Professional Advice Guardrails	high, fail-open	Output-only rules preventing unauthorized legal, financial, and medical advice (banned_topics)
Competitor Mention Detection	medium, fail-open	Paired input and output rules for competitor references, recommendations, and comparisons (ban_competitors)

The same check types can be re-pointed at the ML providers where semantic coverage is needed. A jailbreak rule on the tetrate provider uses its jailbreak classifier with NLI fallback instead of, or layered behind, the pattern-based template rule.

Design characteristics

Characteristic	Description
In-gateway, in-process evaluation	a dynamic module in the data plane, rather than an SDK embedded per application or a separate proxy-of-a-proxy service hop
ML detection without data egress	The tetrate provider runs embedding and classifier models inside the customer cluster on CPU, so semantic detection (jailbreak, PII, toxicity, and the NLI-backed checks) sends no prompt to a third party. Third-party engines (Azure AI Content Safety, external services) are available and are an explicit opt-in per rule
Combinations validated before they ship	The API rejects a rule whose provider kind cannot serve its check type on its stage, so a rule that could never fire cannot be created. The Admin Console scopes its forms to the same matrix
A cost ladder controlled per rule	The same check type can be served by a free in-process engine or by an ML engine. The gateway auto-orders cheap decisive checks first so local hits short-circuit expensive calls, and batches all of a provider's checks into one call
Redaction as well as blocking	Rules can mask matched content in place, with configurable redaction formats, and let traffic flow, on inputs and on outputs, instead of only rejecting requests
Shadow mode on every rule	Any rule can run in monitor mode against live traffic before enforcement. Monitor evaluation is asynchronous and never affects latency or streaming
Deliberate failure semantics	Fail-close, fail-open, and the evaluation timeout are explicit per-guardrail settings, so safety-critical paths can fail closed while availability-critical paths fail open
Investigation support	Blocked responses carry a correlation ID. Operators holding a read-only grant of the guardrail_providers.read permission can look up which rules fired, the scores, and the offending content, and join against their own logging by correlation ID
14 ready-made templates	Region-specific and regulation-specific packs (GDPR, PIPEDA, PDPA, MAS, EU AI Act Article 5, UAE, Australian privacy, healthcare claims) plus security and content-safety packs, multilingual where relevant (EU AI Act in English and French, toxicity in five language variants), cloneable and fully editable per project

---

Where to go next

About guardrails

The object model: policies, rules, providers, patterns, and thresholds.

Configure guardrails

Create project policies and rules, and validate in monitor mode before enforcing.

Configure guardrail providers

How the scoring engines differ in detection approach, latency, and configuration.

Guardrails best practices and FAQ

Rollout order, latency budgets, failure modes, and investigation.

Guardrails architecture

Where guardrail evaluation sits on the request path through the gateway.

Guardrails API reference

Request and response detail for the guardrail and rule management API. Providers, patterns, and triggers have their own API references.