Skip to main content

CreateRateLimitPolicy

POST 
/v1/customers/:customer_id/projects/:project_id/policy/rate-limit-policies

CreateRateLimitPolicy creates a new rate-limit policy.

A policy sets any mix of six limits, and each one applies on its own. A dimension left at 0 is unset and has no limit.

  • input_tokens_per_hour: input (prompt) tokens per rolling hour.
  • output_tokens_per_hour: output (completion) tokens per rolling hour.
  • total_tokens_per_hour: input plus output tokens per rolling hour. This is the "tokens per hour" limit.
  • total_tokens_per_minute: input plus output tokens per rolling minute.
  • requests_per_minute: requests per rolling minute. Each request counts 1 when it arrives.
  • requests_per_hour: requests per rolling hour.

requests_per_minute, requests_per_hour and total_tokens_per_minute are accepted only with scope_type PROJECT or API_KEY and action BLOCK. The server refuses them on any other scope or action with InvalidArgument. Tag, system and monitor policies keep the first three dimensions.

Every dimension that is set must be between 1 and 1,000,000,000. The server refuses any other value with InvalidArgument.

A project-scope limit is counted for each API key in the project on its own, not shared across the project.

The period field has no effect: the server stores and returns it, and each dimension's window comes from the dimension's name.

The server refuses the policy with FailedPrecondition when it sets one of the three newer dimensions and a data plane serving the target project reports a build older than 0.8.0, or no build.

Request​

Responses​

Success