Control request log data

Enterprise Tier

Prompt and response bodies routed through the gateway routinely contain the most sensitive data an organization handles: customer records, source code, internal documents, regulated personal data. For many operators, the question is not whether that content is useful in the dashboard, it plainly is, but whether it is permitted to leave the data-plane cluster at all. Data-residency rules, privacy commitments, and compliance regimes frequently require prompt and completion content to stay inside the customer's own infrastructure, while usage and cost still need to be visible centrally for billing and capacity planning.

The Request logs setting governs exactly that boundary. It controls what reaches the Tetrate management plane in request_log records, so the amount of request detail that leaves the cluster becomes a deliberate operator decision rather than a fixed default. The richest setting forwards full prompt and response content for inspection in the dashboard; the most restrictive setting keeps that content out of the management plane entirely. The setting accepts three values:

Mode	What reaches the management plane	What's dropped
Full (default)	Envelope, counters, costs, headers, `llm_parameters`, request body, response body.	Nothing.
Metadata only	Envelope, counters, costs, headers, `llm_parameters`.	`request_body`, `response_body`.
Off	Nothing.	The entire `request_log` record.

Each mode is a strict superset of the one below it. Full gives the dashboard's per-request views their full detail. Metadata only keeps usage and cost visible while prompt and completion content remains inside the cluster. Off suppresses the request_log record at the management plane altogether.

The setting is system-wide for the deployment. Switching takes effect across the data plane within a few seconds of saving: no restart, no reinstall.

Persona: Platform operator working in the Admin Console, often in coordination with security or compliance stakeholders.

Estimated time: 5 to 10 minutes to choose, set, and verify a mode; longer if an external collector is also being wired up.

When this guide applies

The setting is relevant whenever a decision has to be made about how much request content may cross the cluster boundary toward Tetrate. Typical cases:

Situation	Suggested mode
Default evaluation or development, where full per-request detail in the dashboard is wanted	Full
Prompt and completion content must stay inside the cluster, but usage and cost visibility is still required centrally	Metadata only
No `request_log` record may leave the cluster at all, for the strictest data-residency posture	Off
Full content is required for observability, but in a self-managed stack rather than the dashboard	Metadata only or Off, paired with an external OTEL destination

Outcomes

By the end of this guide:

The three modes are understood, including exactly what each one forwards and what it drops.
A mode has been set from the dashboard and confirmed active with a smoke test.
The audit trail for the change has been located.
Where required, the full record has been confirmed arriving at a self-managed OTEL collector while content is withheld from the management plane.

What each mode affects

The mode affects only the records stored at the management plane. Its effect on each dashboard surface and downstream path is shown below:

Surface	Full	Metadata only	Off
Dashboard Request logs view (per-request rows)	rows with bodies	rows, body panels empty	no rows
Dashboard Usage view (tokens, costs over time)	populated	populated	populated
Dashboard Audit logs view	populated	populated	populated
Billing / transactions table	populated	populated	populated
Customer-attached OTEL destination on the data plane	full record	full record (mode applies only to MP)	full record

Two things are never affected by this setting:

Billing. Transactions are written through a separate path. Every request still produces a transaction regardless of mode, so monthly billing remains accurate.
Customer-attached destinations. A self-managed OTEL receiver (see the External destination section below) receives the full record under every mode; the toggle scopes only what crosses the cluster boundary toward Tetrate.

How to change the mode

The mode is changed entirely from the dashboard. No command-line access, redeployment, or pod restart is required.

Sign in to the dashboard with an admin account.
Go to Settings → Request logs.
Pick a mode from the dropdown. The card shows a short description of what each mode stores.
Click Save.

The save action:

Persists the new mode to system settings.
Pushes the change to the data plane via the existing self-heal channel.
Writes an audit log entry recording who changed the mode and to what value.

Propagation takes a few seconds. Confirm it by sending a prompt through the gateway and checking the Request logs view (see the next section).

How to verify the mode is active

Because the change takes effect within seconds, the active mode is confirmed by sending a known prompt through the gateway and observing how it appears in the dashboard. This smoke test uses the dashboard alone; no kubectl is required.

Setup once

In the dashboard, go to API Keys → Create key. Copy the sk-... value. Note the data plane's gateway URL (under Settings → Workspace).

Smoke test per mode

Set the mode in Settings → Request logs.

Send a prompt with a distinctive word:

curl -X POST https://<your-gateway>/v1/chat/completions \
  -H "Authorization: Bearer sk-<your-key>" \
  -H "Content-Type: application/json" \
  -d '{"model":"claude-haiku-4-5","messages":[{"role":"user","content":"Say MANGO once and stop."}],"max_tokens":50}'

Alternatively, send a prompt from the playground.

Wait ~10 seconds, then check the dashboard.

Expected result by mode:

Mode	Request logs view	Usage view
Full	New row for MANGO. Detail panel shows prompt + response bodies.	MANGO's tokens + cost reflected.
Metadata only	New row for MANGO. Detail panel shows tokens + headers; body panels say "No request body available" / "No response body available".	MANGO's tokens + cost reflected.
Off	No row for MANGO.	MANGO's tokens + cost still reflected (comes from the transaction path).

Repeat for each mode under test, picking a different keyword each time (MANGO / PAPAYA / DURIAN / KIWI, etc.) so the rows are easy to spot.

Audit log check

After any change, Audit logs in the dashboard shows a new row:

Resource type: system_settings
Resource ID: request_logs.mp_mode
Action: UPDATE
Body: {"mode":"<the saved value>"}
Actor: the acting user account
Source IP, User agent, Correlation ID populated

External destination: sending the full record to a self-managed backend

To keep prompt and completion content inside a self-managed observability stack while still letting Tetrate see usage and costs, configure Metadata only or Off mode on the management-plane path and route the full record to an operator-run backend.

The data plane always emits the full record to any local destination, regardless of mode. The typical operator pattern is:

Mode = metadata_only or off (depending on what reaches Tetrate)
Local destination = a backend subscribed to the requestlog signal, as below

There are two ways to attach the local destination. The collector-routed path (subscribing a custom backend to the requestlog signal in Helm values) is the supported primary path. The older direct-attach method (hand-setting OTLP environment variables on the egress container) still works and is documented below as legacy, with a migration path.

Primary path: subscribe a custom backend to the `requestlog` signal

In the data-plane Helm values (the file that carries global.observability), add a backends.custom[] entry subscribed to requestlog:

global:
  observability:
    enabled: true
    signals:
      requestlog:
        enabled: true              # chart default
    backends:
      custom:
        - name: request-log-sink
          endpoint: otel-collector.observability.svc.cluster.local:4317
          signals: [requestlog]    # explicit subscription: an omitted list never includes requestlog
          tls: { insecure: true }  # in-cluster plaintext hop; omit for TLS
          sendingQueue: { queueSize: 64 }

Apply with helm upgrade (or the normal chart-management flow). The subscription is the only switch, from it, the chart wires the whole path automatically:

The data plane's local OpenTelemetry collector renders a dedicated request-log lane: an OTLP/gRPC receiver on port 3501 (with a raised message-size cap) plus an OTLP/HTTP receiver on 3502, and a pipeline whose destinations are exactly the subscribed backends.
The gateway's egress container is injected with the emission configuration (OTEL_LOGS_EXPORTER=otlp, OTEL_EXPORTER_OTLP_LOGS_ENDPOINT=http://otel-gateway.<system-namespace>:3501, OTEL_EXPORTER_OTLP_LOGS_PROTOCOL=grpc) so the module pushes request logs to the collector over OTLP/gRPC (the default transport on this path). These variables are chart-managed: do not set them by hand alongside a subscription (the render fails, see the migration section).

Emission and the collector lane appear and disappear together: remove the subscription and, on the next upgrade, the receiver, the pipeline, and the injected variables are all gone.

The endpoint shape, protocol: otlphttp for backends behind HTTP load balancers or with records past a gRPC size cap, per-backend TLS, header auth from the otel-gateway-backend-credentials Secret, and queue sizing (the sendingQueue: { queueSize: 64 } above is the recommended request-log sizing) are covered field-by-field in Custom observability backends, including how to fan the same records out to several destinations by adding more entries.

Legacy path: direct-attach via container environment variables

Legacy

This is the pre-fan-out mechanism: it supports exactly one destination, speaks OTLP/HTTP, and bypasses the data plane's local collector. Existing setups keep working unchanged, but new destinations should use the collector-routed path above.

Two environment variables on the egress proxy container:

Variable	Value
`OTEL_LOGS_EXPORTER`	`otlp`
`OTEL_EXPORTER_OTLP_LOGS_ENDPOINT`	the full URL of the collector's OTLP/HTTP logs endpoint, including the `/v1/logs` path

For example, a collector listening at https://collector.observability.svc.cluster.local:4318 requires the endpoint https://collector.observability.svc.cluster.local:4318/v1/logs.

The /v1/logs suffix is mandatory on this variable. Per the OTLP exporter specification, the signal-specific OTEL_EXPORTER_OTLP_LOGS_ENDPOINT is used verbatim, and most collectors only respond on that path. Omitting it is the single most common misconfiguration. (Alternatively, set the base variable OTEL_EXPORTER_OTLP_ENDPOINT to the collector URL without a path; the exporter appends /v1/logs itself. On data planes released before July 2026, only the signal-specific variable works, the base form was posted verbatim and returned 404.)

When OTEL_EXPORTER_OTLP_LOGS_PROTOCOL is unset, the transport stays OTLP/HTTP (http/protobuf). This has always been the direct-attach behavior, and upgrades preserve it. An explicit OTEL_EXPORTER_OTLP_LOGS_PROTOCOL=grpc is honored on current data planes; with gRPC the endpoint carries no /v1/logs path and the URL scheme selects TLS (http:// plaintext, https:// TLS against system CAs).

In the Helm values for the egress chart, the variables go on the egress container's env block:

envoyProxy:
  provider:
    kubernetes:
      envoyDeployment:
        container:
          env:
            - name: OTEL_LOGS_EXPORTER
              value: otlp
            - name: OTEL_EXPORTER_OTLP_LOGS_ENDPOINT
              value: https://<your-collector>/v1/logs

Apply with helm upgrade. The egress pod restarts and the new endpoint becomes the local destination.

If the collector requires an auth header, add it via OTEL_EXPORTER_OTLP_HEADERS ("Authorization=Bearer <token>"); for sensitive headers, bind from a Kubernetes Secret with valueFrom.secretKeyRef instead of inlining.

Migrating from direct-attach to the collector-routed path

The collector-routed path replaces hand-managed env with a declared backend: it fans out to any number of destinations, moves header credentials from env strings into a Kubernetes Secret, and adds collector-side queueing and byte-aware batching in front of the backend. Migration is a single values change:

Add the backend entry (as in the primary-path example above) pointing at the same collector. For OTLP/gRPC (protocol: otlp, the default) the endpoint is bare host:port, no scheme, no /v1/logs. To keep talking OTLP/HTTP to a collector that only listens on 4318, set protocol: otlphttp and use the full http(s):// URL without the /v1/logs suffix (the collector exporter appends signal paths itself).
In the same change, remove the operator-set variables from envoyProxy.provider.kubernetes.envoyDeployment.container.env: OTEL_LOGS_EXPORTER, OTEL_EXPORTER_OTLP_LOGS_ENDPOINT, and OTEL_EXPORTER_OTLP_LOGS_PROTOCOL if present. Move any OTEL_EXPORTER_OTLP_HEADERS auth to the backend's auth.headers with the otel-gateway-backend-credentials Secret (see Custom observability backends).
Apply, and re-run the verification below.

The render fails on collision: by design. As soon as any backend subscribes to requestlog, those three variables become chart-managed, and the chart refuses to render a values file that also sets any of them on the container env. helm template / helm upgrade aborts before anything rolls out, with an error naming the colliding key:

egress-helm: OTEL_LOGS_EXPORTER is chart-managed while the requestlog fan-out lane is active
(global.observability.signals.requestlog.enabled plus a subscribed
observability.backends.custom[] entry), but it is also set via
envoyProxy.provider.kubernetes.envoyDeployment.container.env (legacy direct-attach).
Remove the operator-supplied key and configure the destination as a custom backend instead —
see the request-logs migration guide: https://docs.tetrate.ai/guides/ops/cost-and-quota/configuring-request-logs

The fix is exactly what the message says: delete the named key (and its siblings) from the container env, keeping the destination only as a backends.custom[] entry. There is no silent precedence: the chart never lets the two mechanisms configure the same variables at once. Conversely, while no backend subscribes to requestlog, operator-set variables pass through untouched, which is why existing direct-attach setups keep working until they opt in. (The same rule protects the guardrail-trigger lane's AIDISCOVERY_OTEL_EXPORTER_OTLP_LOGS_ENDPOINT.)

What changes in behavior after migrating:

	Direct-attach (legacy)	Collector-routed
Destinations	Exactly one	Any number of subscribed backends
Transport from the module	OTLP/HTTP (unset protocol)	OTLP/gRPC to the local collector; per-backend `otlp` or `otlphttp` onward
Auth	Header string in container env	Per-backend headers from the `otel-gateway-backend-credentials` Secret
Buffering	SDK exporter only	Plus a sized in-memory queue and byte-aware batching per backend in the collector
Lifecycle	Hand-managed env	Rendered and removed with the subscription

Verifying the external destination

The external destination is verified by setting the management-plane path to drop everything, then confirming the full record still arrives at the local collector.

Stand up (or reuse) an OTEL collector reachable from the data-plane cluster on the URL above.
Apply the Helm values change. Wait for the egress pod to roll.
Set the dashboard mode to Off (the strongest case: Tetrate gets nothing).
Send a test prompt via curl as above.
Within ~10 seconds, the collector logs a record with:
- Body: the full request-log record as a plain JSON document (request_id, model, tokens, costs, and the request and response bodies inline). No decompression or decoding is needed; any OTLP-compatible backend can display and index it directly.
- Attribute event.type: request_log, plus gen_ai.* and tars.* dimensions (tars.request.id joins these records with guardrail-trigger records shipped to the same stack).
- Resource attribute log.type: request_log
Request logs are the only records on this stream. Internal management-plane records (billing transactions, access-log forwards, metrics) never reach the local destination.

Older data planes

Data planes released before July 2026 sent the local destination the management-plane transport envelope instead: records with body Request log or ALS access log data export and an opaque gzip-compressed payload attribute marked json+gzip / protobuf+gzip. If the collector shows those, upgrade the data plane; the record content was unreadable by standard OTLP backends.

The dashboard's Request logs view shows no new row for this prompt (off mode at MP). The dashboard's Usage view does show the tokens + cost (transaction path).

If the collector receives nothing, on the collector-routed path:

Confirm the emission variables were injected and the collector lane rendered:
```
kubectl get envoyproxy -n <egress-namespace> -o yaml | grep -A1 OTEL_
kubectl get cm otel-gateway-config -n <system-namespace> -o yaml | grep -A5 requestlog
```
If both are absent, one leg of the gate is off: global.observability.enabled, signals.requestlog.enabled, or the backend's explicit signals: [requestlog] subscription (an omitted signals list never includes requestlog).
Check the otel-gateway collector pod's logs for export errors toward the backend, and see the delivery-semantics section of Custom observability backends for queue-overflow and oversized-record failure modes.
Confirm the backend is reachable from the data-plane cluster's pod network. If it is outside the cluster (for example, behind a tunnel or external load balancer), test reachability with curl from a debug pod first.

On the legacy direct-attach path:

Confirm OTEL_EXPORTER_OTLP_LOGS_ENDPOINT ends in /v1/logs.
Confirm OTEL_LOGS_EXPORTER=otlp is set; without it the local destination is disabled.
Check the egress pod's container logs for OTEL SDK error lines; they include the failing URL and error code.

Frequently asked questions

Does switching to `off` break monthly billing?

No. Transactions are written through a separate code path and arrive regardless of mode. The Usage view and billing reports keep working.

Does switching to `off` lose audit information about who ran which prompt?

The request_log row is the per-request record at the management plane. Under off it is not stored at MP. If that detail is required, run metadata_only (envelope + tokens reach MP, bodies do not) or wire an external destination as above.

Does switching to `metadata_only` retroactively strip bodies from rows already in the database?

No. The setting affects only records produced from the moment it takes effect onward. Existing rows are unchanged.

Can different workspaces have different modes?

Not in this version. The setting is system-wide for the deployment. A per-workspace control is on the roadmap if customers need it.

What happens when an unknown mode value is saved via a direct database write?

The data plane treats anything it does not recognize as full, so a misconfigured value can never silently drop data. The dashboard only ever writes one of the three known values.

Why is the body panel empty in the dashboard?

Either the request genuinely had no body, the response was empty (for example, an error before a generation finished), or the mode when the request was processed was metadata_only. The dashboard does not currently distinguish these cases in the UI.

Where to go next

Custom observability backends

The full field reference for backends.custom[], more signals to subscribe, and the delivery guarantees behind the collector-routed path.

Manage log retention and purge

Govern how long the records each mode produces are kept and how they are removed.

Configure data residency and no-retention

Keep regulated content in-region and unretained at the provider.

When this guide applies​

Outcomes​

What each mode affects​

How to change the mode​

How to verify the mode is active​

Setup once​

Smoke test per mode​

Audit log check​

External destination: sending the full record to a self-managed backend​

Primary path: subscribe a custom backend to the requestlog signal​

Legacy path: direct-attach via container environment variables​

Migrating from direct-attach to the collector-routed path​

Verifying the external destination​

Frequently asked questions​

Does switching to off break monthly billing?​

Does switching to off lose audit information about who ran which prompt?​

Does switching to metadata_only retroactively strip bodies from rows already in the database?​

Can different workspaces have different modes?​

What happens when an unknown mode value is saved via a direct database write?​

Why is the body panel empty in the dashboard?​