Skip to main content

Configuration propagation

A configuration change saved in the Console or through the admin API reaches the data plane in under a minute. Nothing is redeployed and no request is dropped: the data plane controller collects pending configuration changes every ten seconds and applies them to the running gateway. Two slower periodic syncs sit behind that as a backstop, so thirty minutes is the outer bound before a change that has not taken effect is worth investigating.

How long a change takes​

PathWhen it runsHow long
NormalEvery save. The management plane records the change and the data plane controller collects it on its next pass.A few seconds; under a minute
BackstopPeriodic reconciliation, whether or not anything changed. Catches a change that did not reach this data plane on the normal path.5 minutes for the drift sweep, 30 minutes for the full sync

The backstop is not the expected path. It exists so that a data plane which was restarting, unreachable, or newly joined converges on its own without an operator re-saving anything.

This applies to every change held in the management plane: providers and their credentials, the model catalog, model assignments to a project, routing policies, fallback chains, traffic-splitting weights, budgets, rate limits, guardrails, API keys and their tags, MCP profiles, and request-log settings. It does not apply to a change to the data plane release itself. Changing the gateway image or its replica count is a Helm deployment, not a configuration change, and follows the data plane upgrade path instead.

What happens when a change is saved​

  1. The change is written through the Console, the admin API, or an infrastructure-as-code apply, and the management plane stores it.
  2. The management plane records a configuration change for every data plane and every project gateway the change affects.
  3. The controller in the data plane collects pending changes on its next pass, by default every ten seconds, and writes the resulting gateway configuration.
  4. The gateway begins serving the new configuration. In-flight requests are not dropped and no pod restarts.

The propagation settings​

Three settings on the data plane controller govern the timings above. The defaults are what a standard installation runs; there is no reason to change them outside of a support engagement.

Helm value / environment variableDefaultWhat it controls
workerConfig.pollInterval10sHow often the controller collects pending configuration changes. This is the normal path.
WORKER_DRIFT_SWEEP_INTERVAL5mHow often the controller asks the management plane to re-send anything this data plane has not converged on.
WORKER_FULL_SYNC_EVERY_N_DRIFT_TICKS6How many drift sweeps pass between full syncs. At the defaults this is one full sync every 30 minutes. A full sync also runs when the controller starts.

Lowering the poll interval does not make a change arrive materially sooner and adds load on the management plane. Raising the sweep intervals widens the window in which an unnoticed gap goes uncorrected.

What the gateway returns while a change is in flight​

A model whose configuration is still landing returns one of three errors rather than serving traffic. All three use the standard error format.

CodeStatusMeaningWhat to do
model_not_ready503The route for this model exists but is not active yet. The response carries Retry-After: 30.Retry. Most SDKs retry a 503 with Retry-After automatically.
model_not_found400The data plane does not have this model in its catalog yet.If the model was enabled in the last minute, retry. Otherwise check the spelling against GET /v1/models.
model_not_routed404No route is deployed for this model on this data plane.If the model was enabled in the last minute, retry. Otherwise check that it is assigned to the project this key belongs to.

Two things about this window are worth knowing before it is mistaken for a misconfiguration:

  • The gateway's model list is delivered configuration, not the catalog. GET /v1/models on the gateway returns what that data plane has been given, so a model enabled a moment ago is absent from it until the change lands, and a request for it returns model_not_found. The Console shows the model immediately, because the Console reads the catalog. The two disagreeing is the normal appearance of a change in flight, not a fault.
  • Routes can exist before the backends they point at. On a self-hosted data plane, kubectl get aigatewayroutes -A can briefly show rows that are not Accepted, with a message naming a backend that does not exist yet. A provider's routes and its backends are written as separate steps, and a snapshot taken between them looks like a permanent failure. Re-check before acting on it.

When a change has not taken effect​

Work through this in order. Steps 1 and 2 cover almost every case.

  1. Wait a full minute and retry. A retry inside the first few seconds is the most common reason a change looks like it did not apply.
  2. Confirm the change is actually saved. Reload the page in the Console rather than trusting the form state. The Console shows a saved value immediately, whether or not the data plane has it yet.
  3. Check the error code. model_not_ready means the change is still landing and the only correct action is to retry. model_not_routed after a minute means the model has no route on this data plane, which is a configuration question rather than a timing one: confirm the model is enabled in the catalog and assigned to the project.
  4. On a self-hosted data plane, confirm the routes are accepted. kubectl get aigatewayroutes -A should show Accepted for the affected rows. Rows that stay unaccepted for more than a few minutes, naming a backend that does not exist, indicate the data plane has part of the change and not the rest.
  5. Give it thirty minutes before opening a case. That is one full sync cycle. A change that has not taken effect after a full sync is not a propagation delay, and the support case should say so, with the time of the save and the time of the last request that used the old configuration.
note

Nothing in the Console indicates that a rollout is in progress. A saved value is displayed as soon as it is stored, so the Console cannot be used to tell "applied" from "saving". Confirm a change by sending a request through the gateway, not by reading it back in the Console.