Connect an external guardrail service

Enterprise Tier

An external guardrail service is a detection service the customer operates, reached through a provider of kind externalservice. The provider is created in the Admin Console from the External guardrail service catalog template, or with the tare CLI; rules are bound to it in the Admin Console like any other provider, and the connection counts as working only once a request the service is known to flag produces a trigger record.

Persona: Platform operator with a Super Admin session, working in the Admin Console, and in a terminal for the CLI route.

Estimated time: 30 to 45 minutes once the adapter is deployed and reachable, plus the time the adapter team needs to supply a known-positive test input.

When this guide applies

Situation	What it covers
A detection product outside Agent Router has to score prompts or responses inline	Registering the service as an `externalservice` provider and binding rules to it
A vendor product has been wrapped in an adapter that implements the contract	Pointing the provider at the adapter and trusting its certificate
A provider was created but nothing appears on the Events tab	The positive test, the log line, and the metric that separate "not triggered" from "not working"
A rule bound to the service blocks every request	The first-delivery window, the failure-mode defaults, and the old-proxy `400`

The call to the service leaves from the data plane, so where the data plane runs decides who owns the network path:

Self-hosted data plane. The call leaves from the egress gateway pods in the customer's cluster. DNS, firewalling, and the certificate the adapter presents are the customer's to arrange, and so is the data plane upgrade to 0.5.0 that this feature needs.
SaaS data plane (Fully Managed). The call leaves from Tetrate's network, so the adapter has to be reachable from there, and Tetrate operates the data plane version. Whether the adapter can restrict its callers by source address is covered under Allow the gateway's source address (SaaS data plane only).

Content evaluated by the service leaves the gateway for that endpoint on every request the bound rules cover, which is an explicit per-rule choice. On output-stage calls the gateway also sends the user's prompt when it has it, so that checks such as relevance or groundedness can judge the response against it. A service bound only to output-stage rules therefore receives the user's prompt as well as the model's response. The prompt is sent as the model received it, so text that an input-stage rule redacted is sent masked. For which checks the service can perform and how those are declared per instance, see External guardrail services in the guardrails reference.

Outcomes

By the end of this guide:

An externalservice provider exists in the project, with its bearer token stored write-only and its certificate authority trusted.
A guardrail with at least one rule bound to the provider is enforcing, with a deliberate failure mode and an evaluation timeout.
A request the service is known to flag has been blocked and recorded on the Events tab, which is the only proof accepted here.
The log line and metric that expose a failing adapter are known, so an outage of the service is not mistaken for clean traffic.

Prerequisites

Each of these is a gate rather than a convenience: the steps below fail, or worse succeed silently, without it.

A data plane at 0.5.0 or later. Earlier data planes do not read the provider's credential or certificate keys, call the service unauthenticated with the previous request shape, and block every covered request under fail-close. See Troubleshooting.
A deployed adapter, or a service that implements the contract natively. The gateway sends one POST per guardrail, per stage, per provider, plus one per item after a redaction, and expects one score per check and item. The request and response shapes, the transport rules, and a reference adapter are on the Guardrail service contract page, which is written for the implementer; this page is written for the operator. Ask the adapter team for two things before starting: the base URL, and an input the service is known to score above the threshold, which the verification step depends on.
The TLS decision. For an https URL the gateway verifies the server certificate against the system trust store; verification cannot be switched off. A certificate issued by a private authority is trusted by storing that authority's PEM in the provider's CA certificates (ca_cert_pem): at most 8 KiB and 4 certificates, plain CERTIFICATE blocks with nothing else around them, and never empty. A certificate from a public authority needs nothing, and the field is then left blank. Plain http is accepted but sends content and the bearer token in clear text.
The bearer token, if the adapter enforces one. The gateway sends Authorization: Bearer <token> when a credential is stored and no header at all otherwise. The token is optional at the gateway; whether it is required is the adapter's decision.
Permission on guardrail_providers. Creating, updating, and storing a secret for a provider require the guardrail_providers permissions, in the Admin Console and the CLI alike. The built-in Super Admin role holds them and the admin API key scope carries them. The built-in Guardrails Admin role holds only guardrails.*: it is enough for the rule steps, not for registering the provider, where the Admin Console withholds Use this template and the CLI fails with a permission error. Two ways to satisfy it from the CLI:
- A Super Admin session: tare api login --base-url https://<management-plane> stores a session token in the active profile.
- An admin-scoped API key: AGENTROUTER_API_KEY and AGENTROUTER_BASE_URL in the environment, used when the active profile holds no credential. See Which API key for which surface.
The target project, selected in the Admin Console. Guardrail providers are project-scoped and a rule can only reference a provider of its own project. The Admin Console creates the provider in the active project. The CLI route passes --project-id on every command; the value is the project's id as shown under its path in the Admin Console (/projects/<id>), and it must be the project that is active in the Admin Console when the rules are bound in Step 2.
jq, for the CLI route only, used to place the PEM inside the provider JSON without escaping it by hand.

Step 1: register the provider

The provider is registered through either route below. Both produce the same provider, checked by the same server-side validation, and both store the token before the provider is enabled. The configuration is the same on both:

Admin Console field	Configuration key	Value
Service URL	`service_url`	The adapter's base URL. It must be reachable from where the gateway runs, not from the operator's workstation.
Checks	`checks`	The canonical checks the service performs, at least one. The rule form later offers only these.
Endpoint (optional)	`endpoint`	The evaluation path, starting with `/`. Defaults to `/v1/evaluate`.
Timeout (optional)	`timeout`	A duration greater than `0` and at most `30s`. Defaults to `10s`.
CA certificates (optional)	`ca_cert_pem`	The private authority's PEM, as described under Prerequisites.
Bearer token (optional)	none	Stored write-only, outside the configuration.

To take a default, leave the field blank or, on the CLI, leave the key out: a key present with an empty value is refused. Two constraints on the URL: it is a bare http or https base with nothing after the path (no query, fragment, userinfo, or trailing slash), and it must not name a link-local or cloud-metadata address. A literal address of that kind, or a well-known metadata host name such as metadata.google.internal, is refused at save time. A host name that only resolves to a link-local address is refused when the gateway dials; one that resolves to another metadata address, such as 100.100.100.200, is not refused at dial time, so name resolution for the adapter's host has to be controlled.

A plain http URL sends the token unencrypted

The stored token is sent as Authorization: Bearer <token> on every call, and over an http URL it travels in clear text. Use http only for an adapter reached over a trusted network, such as one in the same cluster or behind mesh mTLS. For anything else, use https, with CA certificates for a private certificate authority.

In the Admin Console

Confirm the active project is the target project, then open Guardrails → Guardrail Providers.
In Provider catalog, find External guardrail service and choose Use this template.
In the Add provider from External guardrail service drawer, give the provider a Name. Names are unique within the project.
Fill in the connection form from the table above. The CA certificates are pasted or loaded from a file, and a counter shows their size against the 8 KiB limit. Each field is checked as the server checks it, and every error is shown beside its field before anything is saved.
Leave Enabled on and choose Create provider.

With a bearer token entered, the Admin Console creates the provider disabled, stores the token, and only then enables it, so the provider is never enabled without its credential. If the token cannot be stored after the provider exists, the drawer locks the settings and offers Store credential to retry; if the enable fails, it offers Enable provider, and the provider can also be enabled from the provider list.

The provider's own page shows the same form. The configuration is edited there, and entering a new token rotates the stored one in place. The token is never shown again: the page reads Credential stored or No credential (optional). Clearing a check that a rule still uses shows a warning naming those rules, because a rule whose check the provider no longer declares can never fire.

There is no connection test in the Admin Console. The gateway calls the service, not the Admin Console, so a probe from there would test the wrong network path; Step 3 is the test.

With the tare CLI

The CLI route runs the same sequence as the Admin Console in three commands: create the provider disabled, store the token, then enable it.

Create the provider, disabled. The --provider flag takes the provider as JSON: name, kind, enabled, and config, where config holds the configuration keys from the table above.

export PROJECT_ID="<project id>"

tare api guardrails providers create \
  --project-id "${PROJECT_ID}" \
  --provider "$(jq -n --rawfile ca ca.pem '{
    name: "content-adapter",
    kind: "externalservice",
    enabled: false,
    config: {
      service_url: "https://guardrails.example.com",
      checks: ["pii", "toxicity"],
      timeout: "5s",
      ca_cert_pem: $ca
    }
  }')"

jq --rawfile ca ca.pem reads the PEM file into the $ca variable as one string, with its line breaks intact and escaped for JSON, which a hand-typed flag value cannot do reliably. For a certificate from a public authority, drop the ca_cert_pem line and the --rawfile option.

The response carries the new provider, including its server-assigned id, which the next two commands need:

export PROVIDER_ID="<the id from the create response>"

A create that carries auth_token or any other credential key inside config is refused with an error naming SetGuardrailProviderSecret; the token only enters through the next command.

Store the bearer token. The flag is named --api-key for every provider kind; for this kind the value is the bearer token the adapter expects:

tare api guardrails providers set-secret "${PROVIDER_ID}" \
  --project-id "${PROJECT_ID}" \
  --api-key "$(cat adapter-token.txt)"

The token travels on the command line

--api-key is the only route for the credential on the CLI, and a command-line argument is visible in the process list while the command runs and, when the value is typed literally, in the shell history afterwards. Read it from a file as above rather than pasting it, and keep the file out of version control. A token that has reached a shell history is rotated on the adapter and stored again with the same command.

Skip this command for an adapter that does not require a token.

Enable the provider. The JSON carries only the id and the change; a config that is absent leaves the stored configuration untouched, whereas a config that is present replaces it wholesale.

tare api guardrails providers update \
  --project-id "${PROJECT_ID}" \
  --provider "{\"id\": \"${PROVIDER_ID}\", \"enabled\": true}"

How the token reaches the gateway

The token is stored encrypted and never returned: the provider only reports has_api_key. On the data plane it reaches the gateway as a file mounted from a Kubernetes Secret. The provider configuration delivered to the gateway carries only that file's path, under auth_token_file, a key the management plane sets itself and refuses from a caller, so the token appears in no ConfigMap and no configuration dump. Storing it again rotates it in place, and the gateway reads the new value once the rotated Secret is projected into the proxy pod and its next poll sees the file, with no restart and no traffic interruption. A provider with no token stored shows no credential, which is a valid state for this kind.

Nothing evaluates yet: an enabled provider with no rule bound to it receives no traffic.

The first delivery after enabling can block briefly

Configuration and credential reach the gateway by two paths. The provider configuration is pushed to the proxy at once, while the credential is projected into the proxy pod as a file, and the gateway polls that file every 10 seconds. Between the two, every evaluation the provider runs fails with credential not yet available, and under a fail-close guardrail those requests are blocked. Both routes store the token before enabling, so the window is at most the projection delay plus one poll; expect a handful of blocked requests on a busy project if the provider is enabled with rules already bound.

Step 2: bind rules in the Admin Console

The rule work is the same as for any provider and is covered in Configure guardrails. What differs for this kind is what the defaults do when the service is unreachable.

In the Admin Console, confirm the active project is the one the provider was created in, then open Guardrails → Guardrail Providers. The new provider is listed under Your Providers with the kind shown as External guardrail service, and the credential reads Credential stored or No credential (optional).
Open Guardrails → Rules and add a guardrail, or edit an existing one. Set Evaluation timeout (ms). It has no default: an unset value means every covered request can wait the provider's full timeout on an adapter that hangs. The effective budget per call is the shorter of the provider's timeout and this value.
Add a rule: Provider is the new instance, Check type is one of the checks declared in Step 1, Run on is the stage the service evaluates. A disabled provider is listed as disabled and cannot be picked. Configuration shows a Threshold field: a score at or above it is a violation, an empty field means the data plane default of 0.5, and a value must be greater than 0 and at most 1: anything else, 0 included, is refused when the rule is saved. The field is stored as threshold in the rule's configuration. threshold and failOpen are the two keys the gateway keeps for itself; everything else under Edit as JSON (advanced) is forwarded to the service as that check's parameters. A rule written through the API instead carries these keys in its raw configuration; the externalService configuration variant is deprecated and refused on every rule write.
Save, and enable the guardrail.

The defaults turn an unreachable adapter into an outage

The guardrail form defaults to fail-close, and the rule form to block and enforce. Together they mean that a provider error (a refused connection, a bad certificate, a 400 from the adapter, a timeout, or a pending credential) blocks the covered request, with no trigger record to explain it. That is the right default for a control that must not be bypassed; it is chosen here deliberately, with the evaluation timeout set, rather than inherited. Monitor mode is not a safe way to test the connection either: monitor rules never block on a provider error, so a broken adapter in monitor mode passes traffic and records no trigger. Only the fail-open log line and the provider_error metric named in Step 3 tell it apart from a healthy adapter on clean traffic.

Step 3: verify with a positive test

The proof is a request the service is known to flag being blocked and recorded. A request that passes proves nothing: clean content, a broken adapter in monitor mode, and a fail-open provider error all look the same from the caller's side.

With a rule on the provider set to block and enforce, send the known-positive input from the prerequisites through the project gateway, with a client API key of the project:

curl -sS -X POST "https://<project gateway>/v1/chat/completions" \
  -H "Authorization: Bearer ${CLIENT_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "<a model the project can route>",
    "messages": [{"role": "user", "content": "<the input the service scores high>"}]
  }'

Require a 400 whose body carries the error code content_policy_violation and a message that starts with Guardrail triggered:. The rest of the message is the blocked-response message set on the rule or, failing that, on the guardrail; with neither set it reads Request blocked by guardrail <guardrail id> rule <rule id> in <stage> stage.
Open Guardrails → Rules → Events in the Admin Console and require a trigger record for the rule, carrying the score the service returned.

A 400 whose message reads Guardrail triggered: guardrail evaluation failed, with no trigger record, is a provider error under fail-close, not a detection. A 200 is either clean content or a silent failure. In both cases the connection is not verified, and the two places that say why are on the data plane:

The proxy log. A fail-close block caused by the service logs one line, Guardrail: blocked due to provider error (fail-close), with the provider_id, the rule_id, and the error the call returned. On a self-hosted data plane: kubectl logs -n tars-dataplane -l gateway.envoyproxy.io/owning-gateway-name=egress -c envoy --tail=-1 | grep "provider error". A fail-open or monitor rule logs the first error per provider as Guardrail: provider error on a fail-open rule; further errors from this provider are suppressed for this config generation and, as the line says, suppresses the rest until the next configuration push.
The metric. tars.aidiscovery.guardrail.evaluations with error.type=provider_error counts every evaluation the service failed, by guardrail, whether the rule failed open or closed. A non-zero rate on a freshly connected provider is the signal to read the log.

The error value names the cause: credential not yet available is the first-delivery window from Step 1; a certificate error means ca_cert_pem does not cover the chain the adapter presents; a 400 from the adapter on every request is the contract mismatch under Troubleshooting.

Allow the gateway's source address (SaaS data plane only)

On a self-hosted data plane the source address of the call is the cluster's own egress, and nothing on this page changes it. On a SaaS data plane the adapter is called from Tetrate's network.

Source address restriction on a SaaS data plane

A stable source address that an adapter can allow-list is not published yet. Until it is, an adapter reached from a SaaS data plane authenticates its caller with the bearer token from Step 1 rather than by source address; contact Tetrate Support for the current egress arrangement of the region the data plane runs in.

Troubleshooting

Symptom	Cause	Fix
Use this template is withheld in the Admin Console, or `create`, `update`, or `set-secret` fails with a permission error, while the Guardrails pages show normally	The session or key holds `guardrails.` (Guardrails Admin) but not `guardrail_providers.`	Register the provider with a Super Admin session, or on the CLI with an `admin`-scoped key
Saving the provider, or a change to its configuration, is refused naming `checks`, `service_url`, `ca_cert_pem`, `timeout`, `endpoint`, or an unknown key	Write-time validation: `checks` is missing, empty, or names a check outside the canonical vocabulary; the URL is not a bare absolute base; the PEM is empty, holds something other than certificate blocks, or exceeds 8 KiB or 4 certificates; the timeout is blank or not a duration greater than `0` and at most `30s`; the endpoint is blank or lacks a leading `/`; a value other than `checks` is not a string; or the key is outside the set for this kind	Correct the value the error names; the closed key set is `service_url`, `checks`, `endpoint`, `timeout`, `ca_cert_pem`, and `name`
The rule form does not list the provider, or lists it as disabled	The provider belongs to a different project than the one active in the Admin Console, or it is still disabled	Create the provider in the active project (on the CLI, match `--project-id` to it), and enable it
Every covered request is blocked, no trigger record, log shows `credential not yet available`	The first-delivery window: configuration reached the proxy before the credential file	Wait for the next 10 second poll; store the token before enabling in future
Every covered request is blocked, log shows a `400` from the adapter on a request that carries a single `content` field and no `version` (the reference adapter answers with `unsupported_request` and "the calling gateway predates it"; a customer adapter names the cause in its own words)	The data plane predates 0.5.0: it posts the previous single-`content` request shape to `/evaluate`, without credentials, and a contract v1 adapter answers `400`	Upgrade the data plane to 0.5.0 or later. On a data plane that cannot be upgraded yet, disable the provider or set the guardrail to fail-open until it can
Every covered request is blocked, log shows a certificate error	The adapter's chain is not covered by the system roots plus `ca_cert_pem`	Store the issuing authority's certificate in `ca_cert_pem`; it is appended to the system roots, never substituted for them
Every covered request is blocked, log shows a refused connection or a deadline	The adapter is unreachable from the data plane, or slower than the shorter of `timeout` and the evaluation timeout	Check reachability from the cluster the data plane runs in, not from a workstation; on a SaaS data plane the adapter must be reachable from Tetrate's network
The known-positive input passes with a `200` and no trigger record	The rule is in monitor mode (provider errors are ignored), the guardrail is fail-open, or the threshold is above the score the service returns	Test with block and enforce; read the metric for `provider_error`; compare the threshold with the score the adapter team quoted
Saving a rule is refused with `rule configuration variant "externalService" is deprecated and no longer accepted`	The write carries the deprecated variant. A stored rule that has it stays editable only while the write does not send it back, and an Admin Console older than the management plane sends it back on every edit	Move the rule's per-check parameters into its `raw` configuration; upgrade the Admin Console together with the management plane
A `3xx` from the adapter is logged as an evaluation failure	The gateway never follows redirects	Point `service_url` at the final address

Where to go next

Guardrail service contract

The request and response the adapter implements, with the worked example and reference adapter.

Configure guardrails

Guardrail settings, rule fields, and the Patterns library.

Configure guardrail providers

The providers every project owns, and the Azure AI Content Safety setup.

Guardrails best practices and FAQ

Failure modes, rule ordering, and investigating a block from its correlation id.

When this guide applies​

Outcomes​

Prerequisites​

Step 1: register the provider​

In the Admin Console​

With the tare CLI​

How the token reaches the gateway​

Step 2: bind rules in the Admin Console​

Step 3: verify with a positive test​

Allow the gateway's source address (SaaS data plane only)​

Troubleshooting​