Classify callers as people or applications
From 0.12.0 every request log records whether the caller is a person or an application (the caller kind) and how the caller proved who it is (the authentication method). Usage statistics, request logs, and a caller adoption count filter and group by both.
How a request is classified
Each request log carries these fields:
| Field | Values | Set by |
|---|---|---|
caller_kind | person, application, unknown | The management plane, when it stores the log. It never changes the kind it has stamped. |
authentication_method | api_key, idp_token, unknown | The data plane. It is empty on logs stored before 0.12.0 and for requests served by a data plane older than 0.12.0. |
app_id | The azp, appid, client_id, or cid claim of a verified identity provider token | The data plane. It is empty for an API key call. |
end_user_id | The x-litellm-customer-id header, else the x-litellm-end-user-id header, else the user field of the request body | The data plane, from what the caller sends. |
agent_name | The x-tars-agent-name header | The data plane, from what the caller sends. |
The caller chooses end_user_id and agent_name, so no policy reads them, and they describe what a caller claims, not an identity the gateway verified. See Attribution headers are caller-asserted. A claim that names a person is never stored as app_id, a value over 128 bytes is left out, and end_user_id and agent_name are cut to 128 bytes. A data plane that sends request logs in the metadata_only mode sends app_id and leaves out end_user_id and agent_name.
authentication_method is api_key for an Agent Router API key and idp_token for an identity provider token, including a caller admitted through a project's OIDC contract.
How the caller kind is chosen
The management plane takes the first rule that applies:
- The API key is linked to a principal that has a kind: that kind,
personorapplication. - A user owns the API key:
person. This includes an agent or a CI job that runs with that user's key. - Otherwise:
unknown.
An application is therefore a caller that signs in with an identity provider token that a mapping rule or an OIDC contract maps to an application. An agent that runs with a person's API key counts as a person.
For a log stored with no kind, such as one stored before 0.12.0, a read returns the kind that the same rule gives from its key today, or an empty value when no kind applies.
Set the default kind for new callers
A caller kind default is set where a principal is created, and the management plane copies it into each principal created there.
- An identity mapping rule takes
caller_kind_default(person,application, orunknown). A rule created without it holdsunknown, and apre_registeredrule accepts onlyunknown, because it creates no principal. A rule cannot be edited, so a new default means deleting the rule and creating it again. The principals the old rule created keep their kind. - A project's OIDC contract takes
caller_kind_defaultwith the same values. A contract written without it holdsunknown, and a rewrite that omits it keeps the stored value. A changed default applies only to callers seen for the first time after the change, because a principal that already exists keeps its kind.
person or application is set only on a rule or a contract whose callers are all of that kind. The rule and the contract are described in Authenticate gateway requests.
Correct a principal
A default applies only to principals created after it. The 0.12.0 upgrade sets every existing principal of type human to person and leaves every other principal unknown, so the kind of a principal that already exists is corrected one by one.
- In the Admin Console. Directory → Principals lists the organization's principals with their caller kind and source, and filters them by kind. A holder of
principals.readsees the page, and a holder ofprincipals.editcorrects a kind. A reason is required. - Through the API.
ListPrincipalsandGetPrincipalread principals, andUpdatePrincipalCallerKindcorrects the kind and requires a reason.
Either way, the correction is recorded in the audit log as UPDATE_PRINCIPAL_CALLER_KIND with the old and the new kind. It applies to requests made afterwards: the management plane stamps the corrected kind on the request logs it stores from about 30 seconds after the correction, and it never changes a kind it already stamped. Super Admin holds both permissions, and the admin management key holds the two principals permissions. See Manage roles and permissions.
Read usage and request logs by caller kind and sign-in method
The filters and groups need a management plane on 0.12.0. An earlier management plane ignores the new fields and returns every request or unfiltered usage.
In the Admin Console
The Cost, Tokens, and Usage pages filter by Caller kind (All, Person, Application, Unknown) and by Sign-in method (All, API key, Identity provider token, Unknown). Either value can also be the grouping, and the tables show Caller and Sign-in method columns that follow the filters and are carried in the CSV export. See See where AI spend is going.
Through the API
- Usage statistics.
GET /v1/usage/statstakescaller_kind(person,application, orunknown) andauthentication_method(api_key,idp_token, orunknown) as filters and asgroup_byandsubgroup_byvalues. The groups add up to the totals.caller_typekeepsidpandapi_key. - Request logs.
QueryRequestLogs,GetRequestLogStats, andGetCustomerRequestLogStatstake the same two filters, and the two stats calls group by both. For both fields,unknownalso matches requests that carry no value and, for the kind, requests whose kind is not stored and cannot be resolved from the API key, so the groups add up to the ungrouped totals. - Caller adoption.
GET /v1/usage/caller-adoptioncounts, from the usage rollups, the distinct users recorded as people, as applications, and as unknown, in a time range of up to 400 days. It needs themetrics_readerscope and theusage.readpermission, asGET /v1/usage/statsdoes.
How unknown arises in usage figures:
- Usage recorded before 0.12.0 counts as
unknown: for the method for good, and for the kind until its rollups are rebuilt. - So does a key's usage in an hour in which its kind or its method changed. Daily figures skip
unknownhours, and areunknownonly if the other hours show two values. - Requests served by a data plane older than 0.12.0 carry no authentication method and count as
unknown.
How the adoption count is made:
- A user counts once across their API keys, and once in each organization they call.
- A person who also signs in with an identity provider token counts again under that token's principal, once per project.
- An identity whose kind an admin changed during the range counts under each kind it had.
- Usage by a deleted key counts as a group of its own.
After the upgrade
The usage pages count requests recorded before the upgrade as unknown until the rollups of that period are rebuilt once. The steps are in the Upgrading to 0.12.0 section of the data plane upgrade guide.
Where to go next