Skip to main content

Manage log retention and purge

Request logs are useful for debugging, cost attribution, and incident response, but every record retained is also a record that has to be governed. Prompt and response bodies routed through Agent Router Enterprise routinely carry the most sensitive data an organisation handles, so the longer those records persist, the larger the surface that a compliance review, a breach, or a subject-access request has to account for. Retention is therefore a control in its own right: it minimises the volume of stored sensitive data, it satisfies a documented retention policy rather than letting records accumulate by default, and it supports right-to-erasure obligations by providing a defined way to remove records on demand.

This guide covers the retention side of logging on the platform: how long records are kept, how they are archived to long-term storage before removal where that is required, and how records are purged and the deletion verified. It does not re-document the logging modes themselves. The choice of how much request detail is captured in the first place (from metadata-only, through metadata plus prompt, through metadata plus prompt and response, down to zero-retention) is made in Control what request data leaves the cluster, and that mode can be scoped per app or route and per data classification. Retention picks up from there: given whatever records a mode produces, this guide governs how long they live and how they are removed.

Persona: Platform operator working in the Admin Dashboard, in coordination with compliance and data-governance stakeholders.

Estimated time: 20--30 minutes to set a retention window, configure archival where required, and run a verified purge.

When this guide applies​

Retention controls are relevant whenever the question shifts from "what is captured" to "how long is it kept and how is it removed". Typical cases:

SituationWhat this guide provides
A retention policy specifies a maximum age for request recordsA retention window so records age out automatically
Sensitive records must be preserved in long-term storage before they are removed from the active storeAn archival step that runs ahead of purge
A right-to-erasure or subject-access request requires specific records to be deleted nowAn on-demand purge workflow with deletion verification
Different apps, routes, or data classifications carry different retention obligationsScoping retention alongside the per-app and per-classification logging modes set in the request-logs guide
A periodic compliance attestation must show that stored data is bounded and removableA documented, tested purge workflow plus the audit trail it produces

Outcomes​

By the end of this guide:

  • A retention window has been set so request records age out automatically once they exceed the configured maximum age.
  • Where required, an archival path to long-term storage has been configured to run before records are purged.
  • A purge has been run on demand and the deletion confirmed against the active store.
  • The relationship between request-log retention, audit-trail retention, and any external SIEM copies is understood, so a claim about "where the data lives and for how long" can be made accurately.

Prerequisites​

  • Administrator access to the Admin Dashboard, typically the super_admin role for retention and purge controls.
  • A logging mode already chosen and active for the apps, routes, or classifications in scope. Retention governs whatever records a mode produces; if a route is set to zero-retention, no request records are stored for it and there is nothing to age out or purge. See Control what request data leaves the cluster.
  • Agreement with compliance stakeholders on the retention window per classification, on whether archival is required before purge, and on the destination and retention of any archive.
  • Where archival is required, a long-term storage destination that is reachable from the deployment and governed by its own retention policy.

Step 1: confirm the logging mode and its scope​

Retention only governs records that exist, so the first step is to confirm what each app, route, or classification in scope is configured to capture. A route running in metadata-only mode has no prompt or response bodies to age out; a route running in full mode has the largest record and the strongest case for a short retention window.

  1. Review the active logging mode for each app, route, or data classification in scope. The mode and its scoping are configured in Control what request data leaves the cluster.
  2. Note which scopes capture prompt or response bodies. These carry the most sensitive content and usually warrant the shortest retention.
  3. Note which scopes are set to zero-retention. No request records are stored for these, so they need no retention window and produce nothing to purge.
  4. Record the resulting picture (scope, mode, and the sensitivity of what each scope stores) as the basis for the retention windows set in the next step.

This mapping is what allows retention to be reasoned about per app and per classification rather than as a single global number. A scope that stores full bodies and a scope that stores metadata only can carry different retention windows even though they share the same store.

Step 2: set a retention window so records age out​

A retention window defines the maximum age of a request record. Once a record exceeds that age, it is removed automatically without operator action, which keeps the active store bounded and enforces the retention policy continuously rather than relying on periodic manual cleanup.

  1. Sign in to the Admin Dashboard with an administrator account.
  2. Open the request-log retention settings.
  3. Set the retention window for the scope being configured, the maximum age after which records are removed. Shorter windows minimise stored sensitive data; longer windows preserve more history for investigation.
  4. Where the deployment supports per-scope retention, set a distinct window for each app, route, or classification that carries a different obligation, using the mapping from Step 1. The most sensitive scopes, those capturing prompt or response bodies, generally take the shortest windows.
  5. Save the change.

Once set, records age out on their own schedule. Aging out is continuous and applies going forward; it does not retroactively change the content of records already stored, only how long they survive. A change to the window takes effect for records evaluated after the change, so shortening a window can bring older records within range of removal at the next cycle.

Step 3: archive records before purge where required​

Some retention policies require records to be preserved in long-term storage before they leave the active store, so that an auditable copy survives even after the operational record is removed. Where this applies, archival runs ahead of both automatic aging-out and on-demand purge.

  1. Confirm with compliance stakeholders whether archival is required for the scope, and where the archive must be held.
  2. Configure the long-term storage destination for the records in scope. The destination is governed by its own retention policy, which is typically longer than the active store's window.
  3. Confirm that records are written to the archive before they are removed from the active store, so that no record in scope leaves the active store without an archived copy first existing.
  4. Record the archive location and its retention policy alongside the active-store retention window, so the full lifecycle of a record (active store, then archive, then final removal) is documented in one place.

Archival and active-store retention are separate controls with separate clocks. A record can be removed from the active store on the active window while its archived copy persists for far longer under the archive's own policy. Both windows belong in any statement about how long the data is retained.

Step 4: run a purge and verify the deletion​

Automatic aging-out handles routine retention. A purge handles the on-demand case: a right-to-erasure request, a subject-access deletion, or any situation where specific records must be removed now rather than at their natural expiry. A purge is only complete once the deletion has been verified against the active store.

  1. Identify the records to be purged: by scope, by time range, or by the subject the deletion request concerns.
  2. Where archival is required for the scope, confirm that the archival step in Step 3 has already preserved any copy that policy requires before the records are removed. A purge removes records from the active store; it does not reach into the archive.
  3. Run the purge for the identified records.
  4. Verify the deletion. Confirm that the records no longer appear in the request-logs view for the scope and time range concerned, and that a query for the subject of an erasure request returns no remaining records in the active store.
  5. Record the purge: what was removed, when, on whose authority, and the verification result. This record is what demonstrates that a right-to-erasure obligation was met.

Verification is not optional. A deletion that is requested but not confirmed cannot be attested to, and the most common failure is assuming a purge completed without checking the store afterward. Test the purge workflow end to end (run it, then verify) before relying on it for a live erasure request.

Step 5: account for the audit trail and external SIEM copies​

Request-log retention governs request records. It does not govern two other places the same activity may be recorded, and a complete retention statement has to account for all three.

  • The audit trail has its own retention. Administrative actions, including changes to retention settings and the purge operations run in Step 4, are recorded in the platform's immutable audit log, which is retained under its own policy rather than the request-log window. The audit log cannot be edited or deleted, so a purge of request records does not remove the audit record that the purge occurred. See Audit platform activity for the audit surface and Audit log events reference for the event vocabulary.
  • Exported copies live under the SIEM's retention. Where request or audit records are exported to an external SIEM, those copies are governed by the SIEM's retention policy, not the platform's. Purging a record from the active store does not remove any copy already delivered to the SIEM; that copy ages out, or is purged, on the SIEM's own terms. See Export audit and policy decisions to a SIEM.

The practical consequence is that "the record has been deleted" is only true for the store it names. A right-to-erasure response that has to cover every copy must address the active store, the archive (Step 3), and any external SIEM separately, each on its own retention clock.

What to do next​

Where to go next

Configure data residency and no-retention

Where a scope should store nothing, zero-retention removes the record at source.

Export audit and policy decisions to a SIEM

Retain audit records beyond the in-platform window and govern them centrally.