# OpenTelemetry traces and metrics

> Reference for Agent Router observability: the span model, span attributes, and events emitted by the ai-gateway extproc, and the metric instruments it registers.

The gateway emits two complementary streams of observability data. Trace data flows over OTLP to any OpenTelemetry-compatible backend; metrics are exposed by each data plane component on a Prometheus-compatible scrape endpoint, where they can be pulled by an existing metrics agent and forwarded as OTLP metrics by a collector if the destination requires it. This page documents the structure of both streams: the span model used for traces, the attributes and events actually stamped on those spans, and the metric families exposed by the gateway. For the configuration mechanics of trace export and the supported authentication modes, see <Link to="/agent-router-enterprise/guides/observability-and-analytics/export-telemetry-to-an-observability-stack/">Export Telemetry to an Observability Stack</Link>.

:::note
Every span name, attribute, event, and metric instrument on this page corresponds to a specific construct in the shipped data plane, based on Envoy AI Gateway v0.7.0 with Tetrate patches applied. Instrumentation changes between data plane releases. Where a dashboard or alert rule depends on an exact string, confirm it against the running deployment: spans through a test request to the configured backend, metrics through the scrape endpoint described under [Metric names on the Prometheus endpoint](#metric-names-on-the-prometheus-endpoint).
:::

## Trace structure

Spans are emitted by a single component: the **ai-gateway extproc**, the external-processing service that Envoy calls for each AI request. Each request produces **one flat span**. There is no nesting.

Three consequences follow, and each matters when dashboards or alert rules are being authored against this data:

- **There are no gateway, routing, or inference child spans.** The extproc span covers the request end to end. A fallback walk across several backends does not add spans, and the individual attempts are not separately visible in the trace stream.
- **There are no Envoy proxy spans.** The `EnvoyProxy` resource rendered by the data plane does not configure a tracing provider, so the Envoy hop is not instrumented. The extproc span is the whole of the gateway's contribution to the trace.
- **Client trace context is honoured.** Incoming request headers are run through the configured propagator (W3C `traceparent` by default) before the span is started, so the extproc span attaches to the caller's trace as a child rather than starting a new one. Where a client is instrumented, the gateway span appears inside the application's existing trace.

The span carries the OTel span kind `Internal` for all inference endpoints, and `Client` for MCP spans.

### Span names

The span name identifies the endpoint that served the request. Names are fixed strings supplied by the per-endpoint recorder, not derived from the request path:

| Span name | Endpoint |
| --- | --- |
| `ChatCompletion` | Chat completions |
| `Completion` | Legacy text completions |
| `Responses` | Responses API |
| `CreateEmbeddings` | Embeddings |
| `ImagesResponse` | Image generation |
| `AudioSpeech` | Speech synthesis |
| `Transcription` | Audio transcription |
| `Translation` | Audio translation |
| `Rerank` | Rerank (Cohere schema) |
| `Message` | Anthropic Messages, including token-counting requests, which share the Messages recorder |

MCP traffic is named from the JSON-RPC method rather than the endpoint: `Initialize`, `ListTools`, `CallTool`, `ListPrompts`, `GetPrompt`, `ListResources`, `ReadResource`, `Subscribe`, `Unsubscribe`, `ListResourceTemplates`, `SetLoggingLevel`, `Complete`, and `Ping`. Any method without a mapping is used verbatim as the span name.

### Resource attributes

Resource attributes are set from the environment by the data plane chart and are identical on every span:

| Attribute | Value |
| --- | --- |
| `service.name` | `ai-gateway-extproc` |
| `service.layer` | `ENVOY_AI_GATEWAY` |
| `job_name` | `envoy-ai-gateway` |

### Span attributes

Attributes follow the [OpenInference semantic conventions](https://github.com/Arize-ai/openinference/blob/main/spec/semantic_conventions.md), the open standard for large-language-model trace attributes. This is what allows an LLM-aware backend, such as Arize Phoenix or any other OpenInference-compatible viewer, to render the span as a model invocation rather than a generic HTTP span.

Identity and parameters:

| Attribute | Type | Description |
| --- | --- | --- |
| `openinference.span.kind` | string | `LLM` for inference endpoints, `EMBEDDING` for embeddings, `RERANKER` for rerank |
| `llm.system` | string | `openai`, `anthropic`, or `cohere`, reflecting the request schema. Not set on embedding spans, where the convention excludes it |
| `llm.model_name` | string | The model name. Set from the request at span start, then overwritten from the response model where the provider returns one |
| `llm.invocation_parameters` | string | JSON of the request parameters excluding messages and tools, which have their own attributes |
| `embedding.model_name` | string | The embeddings equivalent of `llm.model_name`, on `CreateEmbeddings` spans |
| `embedding.invocation_parameters` | string | JSON of `model`, `encoding_format`, `dimensions`, and `user`, on `CreateEmbeddings` spans |

Input and output payloads:

| Attribute | Type | Description |
| --- | --- | --- |
| `input.value` | string | The raw request body as JSON, or `__REDACTED__` when input capture is disabled |
| `input.mime_type` | string | `application/json`, set alongside a captured `input.value`. On `Responses` spans it is set unconditionally |
| `output.value` | string | The serialised response body, or `__REDACTED__` when output capture is disabled |
| `output.mime_type` | string | `application/json`. Omitted when output capture is disabled |
| `output.audio_duration` | double | Audio duration reported by a transcription response, where present |
| `output.language` | string | Language detected by a transcription response, where present |

Token counts. These are treated as metadata and are still recorded when prompt and response capture is disabled:

| Attribute | Type | Description |
| --- | --- | --- |
| `llm.token_count.prompt` | integer | Prompt (input) tokens |
| `llm.token_count.completion` | integer | Completion (output) tokens |
| `llm.token_count.total` | integer | Combined prompt and completion tokens |
| `llm.token_count.prompt_details.cache_read` | integer | Prompt tokens served from the provider's prompt cache |
| `llm.token_count.prompt_details.cache_creation` | integer | Prompt tokens written to the provider's prompt cache |
| `llm.token_count.prompt_details.audio` | integer | Audio tokens in the prompt |
| `llm.token_count.completion_details.reasoning` | integer | Tokens spent on reasoning or chain-of-thought |
| `llm.token_count.completion_details.audio` | integer | Audio tokens in the completion |

Each count is recorded only when the provider reports a value greater than zero, so absence of an attribute indicates absence of usage rather than a gap in instrumentation.

Message content. These are indexed families rather than single keys, and they are present only when the active logging mode permits prompt and response capture. See [Configuring Request Logs](/agent-router-enterprise/guides/operate-and-govern/cost-and-quota-management/configuring-request-logs/) for the controls:

| Attribute family | Description |
| --- | --- |
| `llm.input_messages.{i}.message.role`, `.message.content` | Role and content per input message |
| `llm.input_messages.{i}.message.contents.{j}.message_content.{text,type,image.image.url}` | Multi-part input content, including images |
| `llm.output_messages.{i}.message.role`, `.message.content` | Role and content per output message |
| `llm.output_messages.{i}.message.tool_calls.{j}.tool_call.{id,function.name,function.arguments}` | Tool calls returned by the model |
| `llm.tools.{i}.tool.json_schema` | JSON schema of each tool offered in the request |
| `llm.prompts.{i}.prompt.text`, `llm.choices.{i}.completion.text` | Prompt and choice text on legacy `Completion` spans |
| `embedding.embeddings.{i}.embedding.text`, `.embedding.vector` | Embedding input text and output vector on `CreateEmbeddings` spans |

Because indexed message attributes scale with conversation length, the default OTel attribute count limit of 128 is lifted when message capture is enabled, and retained otherwise.

MCP spans carry their own set instead, including `mcp.protocol.version`, `mcp.transport`, `mcp.request.id`, `mcp.method.name`, and, according to the method, `mcp.tool.name`, `mcp.prompt.name`, `mcp.resource.uri`, `mcp.session.id`, and `mcp.client.{name,title,version}`.

### Header-mapped attributes

The data plane configures a header-to-attribute mapping on the extproc. Each listed request header, when present, is copied onto the span under the mapped attribute name:

| Attribute | Source header | Description |
| --- | --- | --- |
| `request.id` | `x-request-id` | The per-request correlation ID |
| `tars.user` | `x-tars-user` | The identity that issued the request |
| `tars.customer` | `x-tars-customer` | The owning customer |
| `tars.workspace` | `x-tars-workspace` | The owning workspace |
| `tars.router` | `x-tars-router` | The router that served the request |

Two further internal mappings (`tars.signature` and `bellhop.user.id`) are configured alongside these and are not intended as a query surface.

This mapping is applied to every endpoint span except `ImagesResponse`, whose tracer is constructed without a header mapping. Image-generation spans therefore carry no `request.id` and no `tars.*` attributes, which is worth accounting for in any dashboard that groups by workspace or customer.

### Span events

The event set is small: lifecycle markers on streaming responses, the standard OTel error event, and one event specific to MCP.

| Event | Recorded when |
| --- | --- |
| `First Token Stream Event` | The first chunk of a streaming response is observed. Emitted on `ChatCompletion`, `Completion`, `Responses`, and `Message` spans |
| `Response Completed Event` | A streaming Responses API request reaches its completion event. Emitted on `Responses` spans only |
| `exception` | The upstream returned a non-success status. Carries `exception.type` (`BadRequestError`, `AuthenticationError`, `PermissionDeniedError`, `NotFoundError`, `RateLimitError`, `InternalServerError`, or `Error`, mapped from the status code) and `exception.message`. The span status is also set to error |
| `route to backend` | An MCP request is routed to a backend. Carries `mcp.backend.name` and `mcp.session.id`. Emitted on MCP spans only |

There is no first-byte, request-received, routing-resolved, or backend-attempted event. Non-streaming requests emit no lifecycle event at all: the span's start and end timestamps are the only timing signal.

### Fields not present on spans

The following are absent from the trace stream and cannot be queried, alerted on, or charted from span data. Several are available elsewhere, as noted:

| Expected field | Reality |
| --- | --- |
| `gateway.*` (`request_id`, `api_key_id`, `requested_model`, `resolved_model`, `resolved_provider`, `fallback_attempts`) | No `gateway.*` namespace is emitted. Correlation is available as `request.id`; the model appears as `llm.model_name` |
| `llm.usage.*` | Not emitted. Token counts use the `llm.token_count.*` keys above |
| `llm.provider` | Not emitted. The OpenInference embedding spec excludes it, and no recorder sets it. The nearest equivalent is `llm.system`, which names the request schema rather than the resolved upstream |
| `gateway.latency_ms`, `gateway.time_to_first_token_ms` | Not emitted as attributes. Latency is derivable from span duration, and both are available as metrics (`gen_ai.server.request.duration`, `gen_ai.server.time_to_first_token`) |
| `gen_ai.*` | These are metric attributes only. No `gen_ai.*` attribute is set on any span |
| `http.method`, `http.target`, `http.status_code` | Not set by the extproc recorders. The endpoint is identifiable from the span name, and failures from the `exception` event |
| Fallback attempt detail | Not represented in the trace stream. Fallback behaviour is observable through metrics and Request Logs |

For the export configuration that delivers these spans to a backend, see [Export Telemetry to an Observability Stack](/agent-router-enterprise/guides/observability-and-analytics/export-telemetry-to-an-observability-stack/).

---

## Metric families

Metrics are registered by the ai-gateway extproc, the same component that emits spans. Two delivery paths exist and are independent of each other:

- **Prometheus scrape.** The extproc admin listener serves `/metrics` in Prometheus exposition format, on port 1064 by default. The Prometheus reader is always registered, irrespective of the configured OTLP metrics exporter, so this endpoint is available whenever the extproc is running.
- **OTLP push.** When metric export is enabled, the same instruments are also pushed over OTLP to the configured collector.

Envoy's own connection and HTTP statistics are a separate stream, published by the proxy's Prometheus stats sink and documented by Envoy rather than here.

The instrument set is small and follows the [OpenTelemetry semantic conventions for generative AI metrics](https://opentelemetry.io/docs/specs/semconv/gen-ai/gen-ai-metrics/). Every instrument is a histogram or a counter; **no gauges are registered**, so there is no in-flight or active-request metric to chart.

### Inference metrics

Four instruments cover all inference traffic. All four are histograms:

| Metric | Unit | Records |
| --- | --- | --- |
| `gen_ai.client.token.usage` | token | Token counts, one observation per token category. Partitioned by `gen_ai.token.type` |
| `gen_ai.server.request.duration` | s | Total request duration, measured from receipt of request headers in the extproc to the end of response-body processing. Recorded for every request, successful or not |
| `gen_ai.server.time_to_first_token` | s | Time from request-header receipt to the first token of the response. Streaming responses only |
| `gen_ai.server.time_per_output_token` | s | Mean time per output token after the first, calculated as `(request_duration - time_to_first_token) / (output_tokens - 1)`. Recorded once at end of stream, and only when more than one output token was produced |

Because `gen_ai.server.request.duration` spans the extproc's whole involvement, it does not separate gateway-internal processing from time spent waiting on the upstream provider. There is no backend-latency instrument, so that split is not available from the metric stream.

`gen_ai.client.token.usage` records a separate observation per token category rather than a single total. Summing across all values of `gen_ai.token.type` double counts, because cached and cache-creation input tokens overlap with the input total as the provider reports them:

| `gen_ai.token.type` | Meaning |
| --- | --- |
| `input` | Prompt tokens |
| `output` | Completion tokens |
| `cached_input` | Prompt tokens served from the provider's cache |
| `cache_creation_input` | Prompt tokens written to the provider's cache |
| `reasoning` | Tokens spent on reasoning |

The `cached_input` and `cache_creation_input` values are not part of the upstream specification yet; they are permitted as custom values and are used here pending standardisation.

### MCP metrics

MCP traffic is instrumented separately:

| Metric | Type | Records |
| --- | --- | --- |
| `mcp.request.duration` | Histogram | Duration of an MCP request. Carries `error.type` on the error path |
| `mcp.initialization.duration` | Histogram | Duration of MCP session initialisation |
| `mcp.method.count` | Counter | MCP method invocations. Carries `mcp.method.name` and `status` (`success`, `failed`, or `error`) |
| `mcp.capabilities.negotiated` | Counter | Capabilities agreed during initialisation. Carries `capability.type` (`tools`, `resources`, `prompts`, `sampling`, `roots`, `experimental`, `elicitation`, `completions`, `logging`) and `capability.side` (`client` or `server`) |
| `mcp.progress.notifications` | Counter | Progress notifications relayed |

MCP instruments additionally carry `mcp.backend`, identifying the upstream server that handled the request.

:::note
`mcp.initialization.duration` is registered with the unit `token` despite measuring seconds. The value is a duration; only the declared unit is wrong. This matters because the Prometheus exporter appends the unit to the metric name, so the scraped series is named for tokens rather than seconds. Confirm the rendered name against a live scrape before writing a query against it.
:::

### Metric names on the Prometheus endpoint

The names above are the OTLP instrument names. The Prometheus exporter rewrites them on the way out: dots become underscores, the unit is appended as a suffix, and histograms expand into `_bucket`, `_sum`, and `_count` series. `gen_ai.server.request.duration` is therefore not the string a PromQL query uses.

Rather than reproduce the mapping here, where it would drift, query the live endpoint for the exact series names. The extproc runs as a sidecar container in the Envoy proxy pods, so the admin listener is reached from inside one of those pods:

```bash
kubectl port-forward -n NAMESPACE POD_NAME 1064:1064
curl -s localhost:1064/metrics | grep -E '^# (HELP|TYPE)'
```

Substitute the namespace and pod of the gateway being inspected; the proxy namespace is set at install time and is not fixed. The `# TYPE` lines give the exposed series name and instrument type for each metric.

---

## Metric attributes

Every inference metric carries the same five base attributes, plus the header-mapped set below. Two further attributes are conditional.

| Attribute | Cardinality | Description |
| --- | --- | --- |
| `gen_ai.operation.name` | Low | The endpoint that served the request: `chat`, `completion`, `embeddings`, `messages`, `image_generation`, `responses`, `speech`, `transcription`, `translation`, or `rerank` |
| `gen_ai.provider.name` | Low | The backend's API schema: `openai`, `azure.openai`, `aws.bedrock`, `aws.anthropic`, `gcp.vertex_ai`, `gcp.anthropic`, `anthropic`, or `cohere`. Any other schema falls back to the configured backend name |
| `gen_ai.original.model` | Medium | The model named in the incoming request body, before any virtualisation is applied |
| `gen_ai.request.model` | Medium | The model sent upstream after resolution |
| `gen_ai.response.model` | Medium | The model the provider reports as having generated the response, which is often a dated build of the requested model |
| `gen_ai.token.type` | Low | Token category. Present on `gen_ai.client.token.usage` only |
| `error.type` | Low | Present on `gen_ai.server.request.duration` only, and only for failed requests. The value is always the `_OTHER` placeholder; the gateway does not yet classify error types on this metric |

The three model attributes are the requested-versus-resolved distinction: `gen_ai.original.model` is what the caller asked for, `gen_ai.request.model` is what routing selected, and `gen_ai.response.model` is what actually answered. Any of the three reports `unknown` when the value could not be determined, which is the expected reading for requests that failed before model resolution.

Because `error.type` is the only failure dimension and it carries a single placeholder value, the metric stream distinguishes failed from successful requests but not one failure mode from another. Error classification comes from Request Logs, or from the `exception.type` attribute on the corresponding span.

### Header-mapped attributes {#metric-header-mapped-attributes}

As with spans, request headers are copied onto metrics under mapped attribute names. **The metric mapping is not the same as the span mapping**, which is worth attention when building queries that span both:

| Attribute | Source header | On spans | On metrics |
| --- | --- | :---: | :---: |
| `tars.user` | `x-tars-user` | Yes | Yes |
| `tars.customer` | `x-tars-customer` | Yes | Yes |
| `tars.workspace` | `x-tars-workspace` | Yes | Yes |
| `tars.router` | `x-tars-router` | Yes | Yes |
| `router.project.id` | `x-router-project-id` | No | Yes |
| `router.gateway.id` | `x-router-gateway-id` | No | Yes |
| `request.id` | `x-request-id` | Yes | No |

Two consequences follow:

- **There is no per-request join between metrics and traces.** `request.id` is a span attribute only, by design: it would be an unbounded label on a metric. Per-request investigation belongs to the trace stream and Request Logs.
- **Chargeback and per-tenant reporting are available from metrics**, at customer, workspace, project, and router granularity. There is no API key dimension on any metric, so per-key attribution is not obtainable from the metric stream; Request Logs and the in-Console Usage Analytics surface are the sources for that.

---

## Sampling

The data plane does not set a sampler, so the OpenTelemetry SDK default applies: **`parentbased_always_on`**. The consequences differ according to whether the caller is instrumented, and the second case is a common source of confusion:

- **Unparented requests**, where the client sends no `traceparent` header, are always sampled. Every such request produces a span.
- **Parented requests**, where the client does send trace context, inherit the caller's sampling decision. If the calling application samples at 1 %, then 99 % of its gateway requests arrive marked as not sampled, and the gateway records no span for them.

The second case means trace volume is partly outside the gateway's control. An instrumented application that samples aggressively will appear to be missing from the trace stream, and no gateway-side setting recovers those spans: the decision was made upstream and propagated. Where a service's gateway traffic is expected in the trace stream but absent, the caller's own sampling configuration is the first thing to check.

Metrics are not sampled. Every request is recorded in every applicable instrument regardless of the trace sampling decision, so the metric stream is a complete record where the trace stream may not be.

This shapes which stream answers which question:

| Question | Stream |
| --- | --- |
| What did this specific request send and receive? | Traces, or Request Logs |
| Why did this request fail, and with what error type? | Traces (`exception` event), or Request Logs |
| What is the p95 latency for this model? | Metrics |
| How many tokens did this workspace consume? | Metrics |
| Did every request get counted for billing? | Metrics or Request Logs, never traces |

Because the gateway contributes a single flat span, the trace stream does not break latency down by stage and does not show which backends a fallback chain attempted. Neither is available from metrics either; fallback behaviour is visible only in Request Logs.

---

## Related

- [Gateway Behavior](/reference/gateway-behavior/): correlation IDs and per-request data
- [Audit Log Events](/reference/audit-log-events/): event schema for administrative actions
- [Export Telemetry to an Observability Stack](/agent-router-enterprise/guides/observability-and-analytics/export-telemetry-to-an-observability-stack/): configuration walkthrough for trace and metric export

Where to go next

  <Link to="/agent-router-enterprise/guides/observability-and-analytics/export-telemetry-to-an-observability-stack/" className="tare-nav-card">
    Export telemetry to an observability stack
    Configure trace and metric export to your backend.
  </Link>
  <Link to="/agent-router-service/guides/monitor-traffic-and-usage/" className="tare-nav-card">
    Monitor traffic and usage
    Read request volume, latency, and token usage in the Console.
  </Link>