Skip to main content

Move a data plane to declared hostnames

Enterprise Tier

Before 0.7.0 the management plane stored a data plane's address, written through SetDataplaneURL or the Admin Console, and each project gateway carried its own URL. From 0.7.0 a data plane declares its hostnames in its serve-helm values instead, and the stored calls are deprecated. A data plane crosses over the moment its chart lists the first name, so the list has to be complete in that one change.


Persona: Platform operator with access to the data plane's Helm values or tetrate identity file and to the Admin Console.

Estimated time: 15 to 30 minutes per data plane, plus one configuration poll.

Outcomes​

By the end of this guide:

  • The data plane's chart declares every hostname it serves, and the Admin Console lists each one as declared in the chart.
  • No project gateway on the data plane carries a hostname the chart does not declare, so later gateway edits are not refused.
  • The rollback path is known: emptying the list and then removing the key returns the data plane to its stored address.

Prerequisites​

  • The data plane runs 0.7.0 or later. Upgrading to 0.7.0 with untouched values keeps it on the stored address and changes nothing about where it serves, so the upgrade itself is safe to take first. See Upgrade a self-hosted data plane.
  • The values file the data plane was installed with, for a Helm or GitOps install. CLI installs carry their values in the release.

What changes when the first name is declared​

There are 3 things that change on the data plane the moment its chart declares at least one hostname:

  1. The declared names replace the stored address everywhere: in the Admin Console, in the readiness probe, and in the address the console prints.
  2. CreateProjectGateway, UpdateProjectGateway, and SetProjectGatewayURL accept only a hostname the data plane declared. Any other hostname is refused with FailedPrecondition and a message that starts data plane <id> does not declare "<hostname>"; a hostname is declared in the data plane's own chart under gateway.hostnames and reported from there. A console or API edit that resends a gateway's own unchanged URL is refused the same way when that hostname is missing from the list.
  3. SetDataplaneURL no longer copies the data plane's address onto its default gateway. It still records the address, which the data plane uses again only after a rollback.

A data plane whose chart declares nothing keeps the pre-0.7 behavior for all three, so each data plane crosses over on its own schedule.

Step 1: inventory the names in use​

  1. In the Admin Console, open Directory → Data Planes and select the data plane.
  2. Read the addresses it lists. While the chart declares nothing, the page lists the stored address first, then the URL of each project gateway on the data plane, each tagged set through the management plane. The page also offers the serve-helm values that declare the same addresses.
  3. Add any hostname callers reach the data plane on that the page does not show, such as a client-facing failover name that DNS points at this data plane.
  4. Choose the name callers should use and put it first. The console prints the first name wherever it shows a single address.

The same list is available from the API, through the call that lists a data plane's hostnames. Each entry carries a source of chart or management-plane.

Step 2: declare the complete list in one change​

Declare every name from Step 1 at once. A partial list turns the gate on for the names left out, and the next edit of a gateway carrying one of them is refused.

With the tetrate CLI:

tetrate agentrouter dataplane upgrade /path/to/data-plane-credentials.json \
--gateway-hostname=ai.acme.example.com \
--gateway-hostname=proxy.acme.example.com \
--gateway-hostname=bedrock-team.acme.example.com

With Helm or GitOps, add the list to the values file and apply it with that file:

global:
gateway:
hostnames:
- ai.acme.example.com
- proxy.acme.example.com
- bedrock-team.acme.example.com
helm upgrade tars "oci://${PRIVATE_IMAGE_REGISTRY}/serve-helm" -f values.yaml

Each name keeps the DNS record, certificate, and load balancer route it already has. A name added for the first time needs all three; see Declare the hostnames a data plane serves.

Step 3: confirm the crossover​

  1. Read the rendered declaration:

    kubectl -n tars-system get configmap tars-gateway-declaration -o jsonpath='{.data.gateway-hostnames}'
  2. After the data plane's next configuration poll, reload its page in the Admin Console. The rows no longer carry the set through the management plane tag.

  3. Check the readiness probe on each name. On a data plane on 0.7.0 or later the probe answers for the whole data plane:

    curl -s https://ai.acme.example.com/healthz/membership
    {"status":"ready","dataplane":"dp-prod-use1"}
  4. Run tetrate agentrouter doctor /path/to/data-plane-credentials.json --only gateway and clear any warning that the certificate does not cover a declared name.

Serve every project on every hostname​

Declaring hostnames does not change which projects a hostname serves. While the data plane carries two or more project gateways, each request is still matched on its hostname:

  • Each gateway's hostname serves only that gateway's project.
  • A declared name that no gateway carries serves no project. Every request on it is refused with 403.

A data plane serves its projects on every hostname only once a single project gateway remains on it. That gateway then serves every project labelled on the data plane, whatever hostname a request arrives on.

Keep the per-project gateways for now

In 0.7.0 and 0.8.0, two known issues affect a data plane that is reduced to one project gateway. The remaining gateway's hostname, usually the one callers already use, answers 404 with model_not_routed for most API keys and does not recover on its own. And once that gateway serves several projects, an API key can call models that only another project on the data plane enables. Until both are fixed, keep the per-project gateways and do not move several projects onto one gateway. See the known issues in the 0.8.0 release notes.

Getting there means labelling the other gateways' projects on the data plane, as in Manage a data plane's addresses and projects, and then deleting the other project gateways.

Draining a data plane​

An explicit empty list is different from leaving the key out. It declares that the data plane serves no hostname: the management plane retires every name it recorded for the data plane, the readiness probe answers 404 on every address, and the three gateway URL calls go back to the pre-0.7 behavior for that data plane.

tetrate agentrouter dataplane upgrade /path/to/data-plane-credentials.json --gateway-hostname=""

Use it only to take a data plane out of DNS rotation on purpose. Draining stops the readiness probe, not inference: the data plane keeps serving callers that still reach it, so a health checker polling /healthz/membership takes it out of rotation while DNS pointed straight at it keeps sending traffic. Shift DNS away from the data plane before draining it.

Rollback​

A missing or wrong name is fixed forward: correct the list and upgrade again, as in Step 2. That takes one upgrade and leaves the readiness probe answering. Roll back only to leave declared hostnames altogether.

Removing global.gateway.hostnames alone is not a rollback. A data plane whose chart carries no key reports nothing, and the management plane reads that as no change, so it keeps the names it last recorded. An empty list is the opposite half:

BehaviorKey removedEmpty list
Names the management plane recordsKept as last declaredCleared
Addresses the Admin Console listsThe declared namesThe stored address, tagged set through the management plane
A gateway URL call with an undeclared hostnameStill refusedAccepted, as before 0.7.0
Readiness probe (/healthz/membership)Answers on the stored address and the project gateway hostnamesAnswers 404 on every address

Neither change affects inference. A full rollback takes both, in this order:

  1. Empty the list: set hostnames: [] in the values file and apply it, or run tetrate agentrouter dataplane upgrade with --gateway-hostname="" alone, as in Draining a data plane.
  2. After the data plane's next configuration poll, reload its page in the Admin Console. The page lists the stored address again, tagged set through the management plane.
  3. Remove the key from the values file and apply it with Helm. The readiness probe answers on the stored address and the project gateway hostnames again, and the management plane keeps the cleared record.

From the first step until the last, the readiness probe answers 404 everywhere, so a health checker polling it takes the data plane out of rotation. Run the three steps back to back, or shift DNS away first.

tetrate agentrouter dataplane upgrade cannot remove the key: without --gateway-hostname it carries the installed list forward, an empty one included. On a CLI install the last step is therefore a Helm upgrade of the tars release with its current values minus the key, which helm get values tars -n tars-system -o yaml prints.

Declaring hostnames never deletes the address stored through SetDataplaneURL, so nothing has to be re-entered.

Keep the list on every upgrade​

tetrate agentrouter dataplane upgrade without --gateway-hostname carries the installed list forward. A helm upgrade that passes no values at all keeps it too, because Helm then reuses the release's previous values. The list is lost when an upgrade passes values that do not carry it: a values file rewritten without the key, --set flags alone, or --reset-values. That removes the key, with the effect described in the table above. Keep the list in the values file applied on every upgrade.

Deprecation timeline​

SetDataplaneURL, its SetWorkspaceURL alias, SetProjectGatewayURL, and the url field of CreateProjectGateway and UpdateProjectGateway are deprecated in 0.7.0. They keep working until no supported data plane predates 0.7.0, because a data plane before 0.7.0 cannot declare hostnames. Keep using SetDataplaneURL for such a data plane until it is upgraded. The Admin Console on 0.7.0 no longer has a field that writes either URL. See Deprecations.

From 0.8.0, SetDataplaneURL and SetWorkspaceURL refuse to set or clear a URL with FailedPrecondition when another organization uses the same data plane ID, because the URL is stored under the ID alone and the write would also change the other organization's data plane. For the same reason, the inference API no longer returns a URL stored under a shared ID: GetDataplaneURL, ListDataplaneURLs, and Prompt reach such a data plane only through the hostnames it declares under global.gateway.hostnames. A data plane whose ID is shared therefore either declares its hostnames, which needs 0.7.0 or later, or is installed under an ID no other organization uses. The URL stored under the shared ID stays as it is.