Skip to main content

Configuration propagation

A configuration change saved in the Console or through the admin API reaches the data plane in under a minute. Nothing is redeployed and no request is dropped: the data plane controller collects pending configuration changes every ten seconds and applies them to the running gateway. Two slower periodic syncs sit behind that as a backstop, so thirty minutes is the outer bound before a change that has not taken effect is worth investigating.

How long a change takes​

PathWhen it runsHow long
NormalEvery save. The management plane records the change and the data plane controller collects it on its next pass.A few seconds; under a minute
BackstopPeriodic reconciliation, whether or not anything changed. Catches a change that did not reach this data plane on the normal path.5 minutes for the drift sweep, 30 minutes for the full sync

The backstop is not the expected path. It exists so that a data plane which was restarting, unreachable, or newly joined converges on its own without an operator re-saving anything.

This applies to every change held in the management plane: providers and their credentials, the model catalog, model assignments to a project, routing policies, fallback chains, traffic-splitting weights, budgets, rate limits, guardrails, API keys and their tags, MCP profiles, and request-log settings. It does not apply to a change to the data plane release itself. Changing the gateway image or its replica count is a Helm deployment, not a configuration change, and follows the data plane upgrade path instead.

What happens when a change is saved​

  1. The change is written through the Console, the admin API, or an infrastructure-as-code apply, and the management plane stores it.
  2. The management plane records a configuration change for every data plane and every project gateway the change affects.
  3. The controller in the data plane collects pending changes on its next pass, by default every ten seconds, and writes the resulting gateway configuration.
  4. The gateway begins serving the new configuration. In-flight requests are not dropped and no pod restarts.

The propagation settings​

Three settings on the data plane controller govern the timings above. The defaults are what a standard installation runs; there is no reason to change them outside of a support engagement.

Helm value / environment variableDefaultWhat it controls
workerConfig.pollInterval10sHow often the controller collects pending configuration changes. This is the normal path.
WORKER_DRIFT_SWEEP_INTERVAL5mHow often the controller asks the management plane to re-send anything this data plane has not converged on.
WORKER_FULL_SYNC_EVERY_N_DRIFT_TICKS6How many drift sweeps pass between full syncs. At the defaults this is one full sync every 30 minutes. A full sync also runs when the controller starts.

Lowering the poll interval does not make a change arrive materially sooner and adds load on the management plane. Raising the sweep intervals widens the window in which an unnoticed gap goes uncorrected.

Guardrail provider credentials​

A guardrail provider's credential does not travel inside the configuration. It is delivered to the gateway as a mounted Kubernetes Secret, one entry per provider, which is why it never appears in the tars-config ConfigMap, in the EnvoyPatchPolicy, or in the proxy's configuration dump.

The proxy is rolled once, on the upgrade that adds the credential volume. Adding a volume is a change to the proxy's pod template, so the egress gateway pods are replaced when the release carrying it is installed. This happens on that upgrade whether or not any guardrail provider holds a credential, and it happens only once. Pod replacement observes the proxy's drain timeout, 300 seconds by default, so in-flight requests and open streams finish rather than being cut. A PodDisruptionBudget is not enabled by default; where a capacity gap during the roll is unacceptable, confirm the replica count before upgrading.

Nothing after that upgrade changes the pod template again. The volume projects the whole Secret, so no provider identifier appears in the pod spec. Adding a credential and rotating one write to the Secret's data alone, and so does the pruning that follows deleting a provider or its project, which is the only way a stored credential goes away. Kubelet projects the new content into the already-running pods. The proxy is not replaced again, whatever the number of providers holding one.

Rotation is picked up by the periodic sweep, not the normal path. Rotating a credential replaces it in place and leaves every byte of the delivered configuration unchanged, so there is nothing for the normal ten-second path to notice. The drift sweep carries it instead: on each pass the data plane's stored credentials are re-resolved and any that have changed are rewritten. At the defaults that is within five minutes, plus the time kubelet takes to project the new file and the gateway takes to re-read it. Keep the previous credential valid until traffic is observed succeeding on the new one. Because each sweep re-resolves from scratch, a rotation that has not landed yet is carried by the next pass with no operator action.

Adding a credential differs from rotating one: the delivered configuration does change, because the provider gains its reference to the credential. So does deleting a provider that holds one. Both follow the normal path in the table above and produce a gateway configuration update, still without a pod restart.

A disabled drift sweep strands every rotation

The drift sweep is the only mechanism that carries a rotated guardrail provider credential to the data plane. Where WORKER_DRIFT_SWEEP_INTERVAL has been set to a negative value, which disables the sweep, rotations never arrive and the gateway keeps authenticating with the retired credential indefinitely. Leave the sweep enabled.

Configuring the providers themselves is covered in Configure guardrail providers.

What the gateway returns while a change is in flight​

A model whose configuration is still landing returns one of three errors rather than serving traffic. All three use the standard error format.

CodeStatusMeaningWhat to do
model_not_ready503The route for this model exists but is not active yet. The response carries Retry-After: 30.Retry. Most SDKs retry a 503 with Retry-After automatically.
model_not_found400The data plane does not have this model in its catalog yet.If the model was enabled in the last minute, retry. Otherwise check the spelling against GET /v1/models.
model_not_routed404No route is deployed for this model on this data plane.If the model was enabled in the last minute, retry. Otherwise check that it is assigned to the project this key belongs to.
model_not_available400The gateway's GET /v1/models lists the model, but its catalog entry is still propagating to the serving path. Returned by data planes on 0.4.0 or later.Retry. The window closes on its own; earlier data planes report this case as model_not_found.

This window is easily mistaken for a misconfiguration:

  • The gateway's model list is delivered configuration, not the catalog. GET /v1/models on the gateway returns what that data plane has been given, so a model enabled a moment ago is absent from it until the change lands, and a request for it returns model_not_found. The Console shows the model immediately, because the Console reads the catalog. The two disagreeing is the normal appearance of a change in flight, not a fault. Since 0.4.0 the catalog is published to the gateway earlier, so a just-enabled model appears in GET /v1/models sooner and answers model_not_ready while its route deploys, instead of being absent until the whole update lands.
  • Routes can exist before the backends they point at. On a self-hosted data plane, kubectl get aigatewayroutes -A can briefly show rows that are not Accepted, with a message naming a backend that does not exist yet. A provider's routes and its backends are written as separate steps, and a snapshot taken between them looks like a permanent failure. Re-check before acting on it.
  • A route deployment reports partial and lost routes. Since 0.4.0, a route deployment for an API key does not become active while only part of the routes from its last render exist, and an active deployment whose routes later go missing raises a Warning condition on the RouteDeployment, naming how many are gone, alongside a warning log. The condition sits beside the phase rather than replacing it, clears itself once the set is whole, and reports the loss rather than re-creating the routes. Deployments that were already active before the 0.4.0 upgrade carry no recorded render count and are reported on from their next render onward.

When a change has not taken effect​

Work through this in order. Steps 1 and 2 cover almost every case.

  1. Wait a full minute and retry. A retry inside the first few seconds is the most common reason a change looks like it did not apply.
  2. Confirm the change is actually saved. Reload the page in the Console rather than trusting the form state. The Console shows a saved value immediately, whether or not the data plane has it yet.
  3. Check the error code. model_not_ready and model_not_available mean the change is still landing and the only correct action is to retry. model_not_routed after a minute means the model has no route on this data plane, which is a configuration question rather than a timing one: confirm the model is enabled in the catalog and assigned to the project.
  4. On a self-hosted data plane, confirm the routes are accepted. kubectl get aigatewayroutes -A should show Accepted for the affected rows. Rows that stay unaccepted for more than a few minutes, naming a backend that does not exist, indicate the data plane has part of the change and not the rest. On a 0.4.0 data plane, kubectl describe routedeployment also says when an active deployment has lost routes: a Warning condition names how many are missing.
  5. Give it thirty minutes before opening a case. That is one full sync cycle. A change that has not taken effect after a full sync is not a propagation delay, and the support case should say so, with the time of the save and the time of the last request that used the old configuration.
note

Nothing in the Console indicates that a rollout is in progress. A saved value is displayed as soon as it is stored, so the Console cannot be used to tell "applied" from "saving". Confirm a change by sending a request through the gateway, not by reading it back in the Console.