Connect an external guardrail service

Enterprise Tier

An external guardrail service is a detection service the customer operates, reached through a provider of kind externalservice. The provider is registered with the tare CLI (the Admin Console cannot create one yet), rules are bound to it in the Admin Console like any other provider, and the connection counts as working only once a request the service is known to flag produces a trigger record.

Persona: Platform operator with a Super Admin session, working in a terminal and the Admin Console.

Estimated time: 30 to 45 minutes once the adapter is deployed and reachable, plus the time the adapter team needs to supply a known-positive test input.

When this guide applies

Situation	What it covers
A detection product outside Agent Router has to score prompts or responses inline	Registering the service as an `externalservice` provider and binding rules to it
A vendor product has been wrapped in an adapter that implements the contract	Pointing the provider at the adapter and trusting its certificate
A provider was created but nothing appears on the Events tab	The positive test, the log line, and the metric that separate "not triggered" from "not working"
A rule bound to the service blocks every request	The first-delivery window, the failure-mode defaults, and the old-proxy `400`

The call to the service leaves from the data plane, so where the data plane runs decides who owns the network path:

Self-hosted data plane. The call leaves from the egress gateway pods in the customer's cluster. DNS, firewalling, and the certificate the adapter presents are the customer's to arrange, and so is the data plane upgrade to 0.5.0 that this feature needs.
SaaS data plane (Fully Managed). The call leaves from Tetrate's network, so the adapter has to be reachable from there, and Tetrate operates the data plane version. Whether the adapter can restrict its callers by source address is covered under Allow the gateway's source address (SaaS data plane only).

Content evaluated by the service leaves the gateway for that endpoint on every request the bound rules cover, which is an explicit per-rule choice. For which checks the service can perform and how those are declared per instance, see External guardrail services in the guardrails reference.

Outcomes

By the end of this guide:

An externalservice provider exists in the project, with its bearer token stored write-only and its certificate authority trusted.
A guardrail with at least one rule bound to the provider is enforcing, with a deliberate failure mode and an evaluation timeout.
A request the service is known to flag has been blocked and recorded on the Events tab, which is the only proof accepted here.
The log line and metric that expose a failing adapter are known, so an outage of the service is not mistaken for clean traffic.

Prerequisites

Each of these is a gate rather than a convenience: the steps below fail, or worse succeed silently, without it.

A data plane at 0.5.0 or later. Earlier data planes do not read the provider's credential or certificate keys, call the service unauthenticated with the previous request shape, and block every covered request under fail-close. See Troubleshooting.
A deployed adapter, or a service that implements the contract natively. The gateway sends one POST per guardrail, per stage, per provider, plus one per item after a redaction, and expects one score per check and item. The request and response shapes, the transport rules, and a reference adapter are on the Guardrail service contract page, which is written for the implementer; this page is written for the operator. Ask the adapter team for two things before starting: the base URL, and an input the service is known to score above the threshold, which the verification step depends on.
The TLS decision. For an https URL the gateway verifies the server certificate against the system trust store; verification cannot be switched off. A certificate issued by a private authority is trusted by storing that authority's PEM in the provider's ca_cert_pem: at most 8 KiB and 4 certificates, plain CERTIFICATE blocks with nothing else around them, and never empty. A certificate from a public authority needs nothing, and the key is then left out altogether. Plain http is accepted but sends content and the bearer token in clear text.
The bearer token, if the adapter enforces one. The gateway sends Authorization: Bearer <token> when a credential is stored and no header at all otherwise. The token is optional at the gateway; whether it is required is the adapter's decision.
Permission on guardrail_providers. Creating, updating, and storing a secret for a provider require the guardrail_providers permissions, which the built-in Super Admin role holds and the admin API key scope carries. The built-in Guardrails Admin role holds only guardrails.* and is enough for the rule steps in the Admin Console, not for the provider steps in the CLI: those fail with a permission error. Two ways to satisfy it from the CLI:
- A Super Admin session: tare api login --base-url https://<management-plane> stores a session token in the active profile.
- An admin-scoped API key: AGENTROUTER_API_KEY and AGENTROUTER_BASE_URL in the environment, used when the active profile holds no credential. See Which API key for which surface.
The project id, and the same project selected in the Admin Console. Guardrail providers are project-scoped and a rule can only reference a provider of its own project. Every command below passes --project-id; the value is the project's id as shown under its path in the Admin Console (/projects/<id>), and it must be the project that is active in the Admin Console when the rules are bound in Step 4.
jq, used to place the PEM inside the provider JSON without escaping it by hand.

Step 1: register the provider, disabled

Create the provider with enabled: false, so that the token can be stored before any rule can route traffic through it. The --provider flag takes the provider as JSON: name, kind, enabled, and config, where config holds the keys the data plane reads for this kind. service_url is the adapter's base URL, checks is the list of canonical check names the service performs (the rule form later offers only these), and endpoint (default /v1/evaluate) and timeout (default 10s, greater than 0 and at most 30s) are optional. To take a default, leave the key out: a key present with an empty value is refused.

export PROJECT_ID="<project id>"

tare api guardrails providers create \
  --project-id "${PROJECT_ID}" \
  --provider "$(jq -n --rawfile ca ca.pem '{
    name: "content-adapter",
    kind: "externalservice",
    enabled: false,
    config: {
      service_url: "https://guardrails.example.com",
      checks: ["pii", "toxicity"],
      timeout: "5s",
      ca_cert_pem: $ca
    }
  }')"

jq --rawfile ca ca.pem reads the PEM file into the $ca variable as one string, with its line breaks intact and escaped for JSON, which a hand-typed flag value cannot do reliably. For a certificate from a public authority, drop the ca_cert_pem line and the --rawfile option. Two constraints on the URL: it is a bare http or https base with nothing after the path (no query, fragment, userinfo, or trailing slash), and it must not name a link-local or cloud-metadata address. A literal address of that kind, or a well-known metadata host name such as metadata.google.internal, is refused at save time. A host name that only resolves to a link-local address is refused when the gateway dials; one that resolves to another metadata address, such as 100.100.100.200, is not refused at dial time, so name resolution for the adapter's host has to be controlled.

The response carries the new provider, including its server-assigned id. It is needed by every later step:

export PROVIDER_ID="<the id from the create response>"

A create that carries auth_token or any other credential key inside config is refused with an error naming SetGuardrailProviderSecret; the token only enters through the next step.

Step 2: store the bearer token

Store the token with set-secret. The flag is named --api-key for every provider kind; for this kind the value is the bearer token the adapter expects:

tare api guardrails providers set-secret "${PROVIDER_ID}" \
  --project-id "${PROJECT_ID}" \
  --api-key "$(cat adapter-token.txt)"

The token travels on the command line

--api-key is the only route for the credential, and a command-line argument is visible in the process list while the command runs and, when the value is typed literally, in the shell history afterwards. Read it from a file as above rather than pasting it, and keep the file out of version control. A token that has reached a shell history is rotated on the adapter and stored again with the same command.

The token is stored encrypted and never returned: the provider only reports has_api_key. On the data plane it reaches the gateway as a file mounted from a Kubernetes Secret. The provider configuration delivered to the gateway carries only that file's path, under auth_token_file, a key the management plane sets itself and refuses from a caller, so the token appears in no ConfigMap and no configuration dump. Storing it again rotates it in place, and the gateway reads the new value once the rotated Secret is projected into the proxy pod and its next poll sees the file, with no restart and no traffic interruption. Skip this step for an adapter that does not require a token; the provider then shows no credential, which is a valid state for this kind.

Step 3: enable the provider

Enable the provider with update. The JSON carries only the id and the change; a config that is absent leaves the stored configuration untouched, whereas a config that is present replaces it wholesale.

tare api guardrails providers update \
  --project-id "${PROJECT_ID}" \
  --provider "{\"id\": \"${PROVIDER_ID}\", \"enabled\": true}"

Nothing evaluates yet: an enabled provider with no rule bound to it receives no traffic.

The first delivery after enabling can block briefly

Configuration and credential reach the gateway by two paths. The provider configuration is pushed to the proxy at once, while the credential is projected into the proxy pod as a file, and the gateway polls that file every 10 seconds. Between the two, every evaluation the provider runs fails with credential not yet available, and under a fail-close guardrail those requests are blocked. Store the token before enabling, as above, so that the window is at most the projection delay plus one poll; and expect a handful of blocked requests on a busy project if the provider is enabled with rules already bound.

Step 4: bind rules in the Admin Console

The rule work is the same as for any provider and is covered in Configure guardrails. What differs for this kind is what the defaults do when the service is unreachable.

In the Admin Console, confirm the active project is the one --project-id named, then open Guardrails → Guardrail Providers. The new provider is listed under Your Providers with the kind shown as External guardrail service; the certificate value is shown read-only and the credential reads as stored or as optional.
Open Guardrails → Rules and add a guardrail, or edit an existing one. Set Evaluation timeout (ms). It has no default: an unset value means every covered request can wait the provider's full timeout on an adapter that hangs. The effective budget per call is the shorter of the provider's timeout and this value.
Add a rule: Provider is the new instance, Check type is one of the checks declared in Step 1, Run on is the stage the service evaluates. Configuration shows a Threshold field: a score at or above it is a violation, an empty field means the data plane default of 0.5, and a value must be greater than 0 and at most 1: anything else, 0 included, is refused when the rule is saved. The field is stored as threshold in the rule's configuration. threshold and failOpen are the two keys the gateway keeps for itself; everything else under Edit as JSON (advanced) is forwarded to the service as that check's parameters. A rule written through the API instead carries these keys in its raw configuration; the externalService configuration variant is deprecated and refused on every rule write.
Save, and enable the guardrail.

The defaults turn an unreachable adapter into an outage

The guardrail form defaults to fail-close, and the rule form to block and enforce. Together they mean that a provider error (a refused connection, a bad certificate, a 400 from the adapter, a timeout, or a pending credential) blocks the covered request, with no trigger record to explain it. That is the right default for a control that must not be bypassed; it is chosen here deliberately, with the evaluation timeout set, rather than inherited. Monitor mode is not a safe way to test the connection either: monitor rules never block on a provider error, so a broken adapter in monitor mode passes traffic and records no trigger. Only the fail-open log line and the provider_error metric named in Step 5 tell it apart from a healthy adapter on clean traffic.

Step 5: verify with a positive test

The proof is a request the service is known to flag being blocked and recorded. A request that passes proves nothing: clean content, a broken adapter in monitor mode, and a fail-open provider error all look the same from the caller's side.

With a rule on the provider set to block and enforce, send the known-positive input from the prerequisites through the project gateway, with a client API key of the project:

curl -sS -X POST "https://<project gateway>/v1/chat/completions" \
  -H "Authorization: Bearer ${CLIENT_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "<a model the project can route>",
    "messages": [{"role": "user", "content": "<the input the service scores high>"}]
  }'

Require a 400 whose body carries the error code content_policy_violation and a message that starts with Guardrail triggered:. The rest of the message is the blocked-response message set on the rule or, failing that, on the guardrail; with neither set it reads Request blocked by guardrail <guardrail id> rule <rule id> in <stage> stage.
Open Guardrails → Rules → Events in the Admin Console and require a trigger record for the rule, carrying the score the service returned.

A 400 whose message reads Guardrail triggered: guardrail evaluation failed, with no trigger record, is a provider error under fail-close, not a detection. A 200 is either clean content or a silent failure. In both cases the connection is not verified, and the two places that say why are on the data plane:

The proxy log. A fail-close block caused by the service logs one line, Guardrail: blocked due to provider error (fail-close), with the provider_id, the rule_id, and the error the call returned. On a self-hosted data plane: kubectl logs -n tars-dataplane -l gateway.envoyproxy.io/owning-gateway-name=egress -c envoy --tail=-1 | grep "provider error". A fail-open or monitor rule logs the first error per provider as Guardrail: provider error on a fail-open rule; further errors from this provider are suppressed for this config generation and, as the line says, suppresses the rest until the next configuration push.
The metric. tars.aidiscovery.guardrail.evaluations with error.type=provider_error counts every evaluation the service failed, by guardrail, whether the rule failed open or closed. A non-zero rate on a freshly connected provider is the signal to read the log.

The error value names the cause: credential not yet available is the first-delivery window from Step 3; a certificate error means ca_cert_pem does not cover the chain the adapter presents; a 400 from the adapter on every request is the contract mismatch under Troubleshooting.

Allow the gateway's source address (SaaS data plane only)

On a self-hosted data plane the source address of the call is the cluster's own egress, and nothing on this page changes it. On a SaaS data plane the adapter is called from Tetrate's network.

Source address restriction on a SaaS data plane

A stable source address that an adapter can allow-list is not published yet. Until it is, an adapter reached from a SaaS data plane authenticates its caller with the bearer token from Step 2 rather than by source address; contact Tetrate Support for the current egress arrangement of the region the data plane runs in.

Troubleshooting

Symptom	Cause	Fix
`create`, `update`, or `set-secret` fails with a permission error while the Admin Console shows the Guardrails pages normally	The session or key holds `guardrails.` (Guardrails Admin) but not `guardrail_providers.`	Use a Super Admin session or an `admin`-scoped key for the provider steps
`create` or an `update` that changes `config` is refused naming `checks`, `service_url`, `ca_cert_pem`, `timeout`, `endpoint`, or an unknown key	Write-time validation: `checks` is missing, empty, or names a check outside the canonical vocabulary; the URL is not a bare absolute base; the PEM is empty, holds something other than certificate blocks, or exceeds 8 KiB or 4 certificates; the timeout is blank or not a duration greater than `0` and at most `30s`; the endpoint is blank or lacks a leading `/`; a value other than `checks` is not a string; or the key is outside the set for this kind	Correct the value the error names; the closed key set is `service_url`, `checks`, `endpoint`, `timeout`, `ca_cert_pem`, and `name`
The rule form does not list the provider	The provider belongs to a different project than the one active in the Admin Console, or it is still disabled	Match `--project-id` to the active project; run Step 3
Every covered request is blocked, no trigger record, log shows `credential not yet available`	The first-delivery window: configuration reached the proxy before the credential file	Wait for the next 10 second poll; store the token before enabling in future
Every covered request is blocked, log shows a `400` from the adapter on a request that carries a single `content` field and no `version` (the reference adapter answers with `unsupported_request` and "the calling gateway predates it"; a customer adapter names the cause in its own words)	The data plane predates 0.5.0: it posts the previous single-`content` request shape to `/evaluate`, without credentials, and a contract v1 adapter answers `400`	Upgrade the data plane to 0.5.0 or later. On a data plane that cannot be upgraded yet, disable the provider or set the guardrail to fail-open until it can
Every covered request is blocked, log shows a certificate error	The adapter's chain is not covered by the system roots plus `ca_cert_pem`	Store the issuing authority's certificate in `ca_cert_pem`; it is appended to the system roots, never substituted for them
Every covered request is blocked, log shows a refused connection or a deadline	The adapter is unreachable from the data plane, or slower than the shorter of `timeout` and the evaluation timeout	Check reachability from the cluster the data plane runs in, not from a workstation; on a SaaS data plane the adapter must be reachable from Tetrate's network
The known-positive input passes with a `200` and no trigger record	The rule is in monitor mode (provider errors are ignored), the guardrail is fail-open, or the threshold is above the score the service returns	Test with block and enforce; read the metric for `provider_error`; compare the threshold with the score the adapter team quoted
Saving a rule is refused with `rule configuration variant "externalService" is deprecated and no longer accepted`	The write carries the deprecated variant. A stored rule that has it stays editable only while the write does not send it back, and an Admin Console older than the management plane sends it back on every edit	Move the rule's per-check parameters into its `raw` configuration; upgrade the Admin Console together with the management plane
A `3xx` from the adapter is logged as an evaluation failure	The gateway never follows redirects	Point `service_url` at the final address

Where to go next

Guardrail service contract

The request and response the adapter implements, with the worked example and reference adapter.

Configure guardrails

Guardrail settings, rule fields, and the Patterns library.

Configure guardrail providers

The providers every project owns, and the Azure AI Content Safety setup.

Guardrails best practices and FAQ

Failure modes, rule ordering, and investigating a block from its correlation id.

When this guide applies​

Outcomes​

Prerequisites​

Step 1: register the provider, disabled​

Step 2: store the bearer token​

Step 3: enable the provider​

Step 4: bind rules in the Admin Console​

Step 5: verify with a positive test​

Allow the gateway's source address (SaaS data plane only)​

Troubleshooting​