Skip to main content

Classify callers as people or applications

Enterprise Tier

From 0.12.0 every request log records whether the caller is a person or an application (the caller kind) and how the caller proved who it is (the authentication method). Usage statistics, request logs, and a caller adoption count filter and group by both.


How a request is classified​

Each request log carries these fields:

FieldValuesSet by
caller_kindperson, application, unknownThe management plane, when it stores the log. It never changes the kind it has stamped.
authentication_methodapi_key, idp_token, unknownThe data plane. It is empty on logs stored before 0.12.0 and for requests served by a data plane older than 0.12.0.
app_idThe azp, appid, client_id, or cid claim of a verified identity provider tokenThe data plane. It is empty for an API key call.
end_user_idThe x-litellm-customer-id header, else the x-litellm-end-user-id header, else the user field of the request bodyThe data plane, from what the caller sends.
agent_nameThe x-tars-agent-name headerThe data plane, from what the caller sends.

The caller chooses end_user_id and agent_name, so no policy reads them, and they describe what a caller claims, not an identity the gateway verified. See Attribution headers are caller-asserted. A claim that names a person is never stored as app_id, a value over 128 bytes is left out, and end_user_id and agent_name are cut to 128 bytes. A data plane that sends request logs in the metadata_only mode sends app_id and leaves out end_user_id and agent_name.

authentication_method is api_key for an Agent Router API key and idp_token for an identity provider token, including a caller admitted through a project's OIDC contract.

How the caller kind is chosen​

The management plane takes the first rule that applies:

  1. The API key is linked to a principal that has a kind: that kind, person or application.
  2. A user owns the API key: person. This includes an agent or a CI job that runs with that user's key.
  3. Otherwise: unknown.

An application is therefore a caller that signs in with an identity provider token that a mapping rule or an OIDC contract maps to an application. An agent that runs with a person's API key counts as a person.

For a log stored with no kind, such as one stored before 0.12.0, a read returns the kind that the same rule gives from its key today, or an empty value when no kind applies.

Set the default kind for new callers​

A caller kind default is set where a principal is created, and the management plane copies it into each principal created there.

  • An identity mapping rule takes caller_kind_default (person, application, or unknown). A rule created without it holds unknown, and a pre_registered rule accepts only unknown, because it creates no principal. A rule cannot be edited, so a new default means deleting the rule and creating it again. The principals the old rule created keep their kind.
  • A project's OIDC contract takes caller_kind_default with the same values. A contract written without it holds unknown, and a rewrite that omits it keeps the stored value. A changed default applies only to callers seen for the first time after the change, because a principal that already exists keeps its kind.

person or application is set only on a rule or a contract whose callers are all of that kind. The rule and the contract are described in Authenticate gateway requests.

Correct a principal​

A default applies only to principals created after it. The 0.12.0 upgrade sets every existing principal of type human to person and leaves every other principal unknown, so the kind of a principal that already exists is corrected one by one.

  • In the Admin Console. Directory → Principals lists the organization's principals with their caller kind and source, and filters them by kind. A holder of principals.read sees the page, and a holder of principals.edit corrects a kind. A reason is required.
  • Through the API. ListPrincipals and GetPrincipal read principals, and UpdatePrincipalCallerKind corrects the kind and requires a reason.

Either way, the correction is recorded in the audit log as UPDATE_PRINCIPAL_CALLER_KIND with the old and the new kind. It applies to requests made afterwards: the management plane stamps the corrected kind on the request logs it stores from about 30 seconds after the correction, and it never changes a kind it already stamped. Super Admin holds both permissions, and the admin management key holds the two principals permissions. See Manage roles and permissions.

Read usage and request logs by caller kind and sign-in method​

The filters and groups need a management plane on 0.12.0. An earlier management plane ignores the new fields and returns every request or unfiltered usage.

In the Admin Console​

The Cost, Tokens, and Usage pages filter by Caller kind (All, Person, Application, Unknown) and by Sign-in method (All, API key, Identity provider token, Unknown). Either value can also be the grouping, and the tables show Caller and Sign-in method columns that follow the filters and are carried in the CSV export. See See where AI spend is going.

Through the API​

  • Usage statistics. GET /v1/usage/stats takes caller_kind (person, application, or unknown) and authentication_method (api_key, idp_token, or unknown) as filters and as group_by and subgroup_by values. The groups add up to the totals. caller_type keeps idp and api_key.
  • Request logs. QueryRequestLogs, GetRequestLogStats, and GetCustomerRequestLogStats take the same two filters, and the two stats calls group by both. For both fields, unknown also matches requests that carry no value and, for the kind, requests whose kind is not stored and cannot be resolved from the API key, so the groups add up to the ungrouped totals.
  • Caller adoption. GET /v1/usage/caller-adoption counts, from the usage rollups, the distinct users recorded as people, as applications, and as unknown, in a time range of up to 400 days. It needs the metrics_reader scope and the usage.read permission, as GET /v1/usage/stats does.

How unknown arises in usage figures:

  • Usage recorded before 0.12.0 counts as unknown: for the method for good, and for the kind until its rollups are rebuilt.
  • So does a key's usage in an hour in which its kind or its method changed. Daily figures skip unknown hours, and are unknown only if the other hours show two values.
  • Requests served by a data plane older than 0.12.0 carry no authentication method and count as unknown.

How the adoption count is made:

  • A user counts once across their API keys, and once in each organization they call.
  • A person who also signs in with an identity provider token counts again under that token's principal, once per project.
  • An identity whose kind an admin changed during the range counts under each kind it had.
  • Usage by a deleted key counts as a group of its own.

After the upgrade​

The usage pages count requests recorded before the upgrade as unknown until the rollups of that period are rebuilt once. The steps are in the Upgrading to 0.12.0 section of the data plane upgrade guide.