Skip to main content

Declare the hostnames a data plane serves

Enterprise Tier

From 0.7.0 a data plane declares the hostnames it serves in its own chart, under global.gateway.hostnames, and reports them to the management plane on every configuration poll. The Admin Console lists them read-only. No console page or API call writes a declared hostname: adding or retiring one is a values change followed by tetrate agentrouter dataplane upgrade or helm upgrade.


Persona: Platform operator with access to the data plane's Helm values or tetrate identity file, the DNS zone, and the load balancer in front of the cluster.

Estimated time: 10 minutes for the values change, plus DNS propagation and certificate issuance.

Outcomes​

By the end of this guide:

  • The data plane's serve-helm values list every hostname callers use to reach it, with the name callers should use first.
  • Each declared hostname resolves to the data plane's load balancer, presents a certificate that covers it, and routes to the data plane.
  • The Admin Console lists the hostnames on the data plane's page, and tetrate agentrouter doctor confirms the certificate covers each one.

Prerequisites​

  • A self-hosted data plane on 0.7.0 or later. A data plane before 0.7.0 cannot declare hostnames and keeps the address stored for it through the admin API.
  • Control of the DNS zone for each hostname, and of the certificate the load balancer presents.
  • For a data plane that already serves traffic on a stored address, read Move a data plane to declared hostnames first. A partial list refuses later edits to gateway URLs that are missing from it.

Step 1: list the hostnames​

Add the list to the data plane's serve-helm values:

global:
gateway:
hostnames:
- eu.acme.example.com # first: the address the console prints
- dp.acme.example.com:8443 # a port, for callers that do not use 443

tetrate agentrouter dataplane install takes the same list as a repeatable flag:

# Declare two names at install time.
tetrate agentrouter dataplane install /path/to/data-plane-credentials.json \
--gateway-hostname=eu.acme.example.com \
--gateway-hostname=dp.acme.example.com:8443

# Upgrade and keep the installed list. tetrate agentrouter dataplane upgrade reads it from the release.
tetrate agentrouter dataplane upgrade /path/to/data-plane-credentials.json

# Upgrade and replace the list.
tetrate agentrouter dataplane upgrade /path/to/data-plane-credentials.json --gateway-hostname=eu.acme.example.com

There are 5 rules for the entries:

  1. An entry is a hostname, optionally with a port. The management plane stores the port apart from the name and joins them again when it prints an address. A scheme such as https:// is accepted and dropped.
  2. The first entry is the address the console prints wherever it shows a single address, such as the Base URL panel and the Playground. List the name callers should use first.
  3. Two data planes in one organization may declare the same name, which is how a pair behind a global server load balancer serves one hostname. The console lists both data planes under that name. Two gateways on one data plane still cannot share one.
  4. Leaving the key out declares nothing. The data plane keeps the address stored for it through SetDataplaneURL and answers its readiness probe there. serve-helm leaves the key out by default, so upgrading to 0.7.0 with untouched values changes nothing about where the data plane serves.
  5. An empty list is a declaration. It says the data plane serves no hostname: the management plane retires every name it recorded for the data plane, and the readiness probe stops answering. Write hostnames: [], or run tetrate agentrouter dataplane upgrade with --gateway-hostname="" alone, only to drain a data plane.
An upgrade that passes other values can drop the list

A helm upgrade that passes values without the key, such as a values file rewritten without it, --set flags alone, or --reset-values, drops the key. The readiness probe then falls back to the stored address, while the management plane keeps the names it last recorded and keeps refusing gateway URLs outside them. A helm upgrade that passes no values reuses the release's previous values and keeps the list, and tetrate agentrouter dataplane upgrade carries the installed list forward. The reverse also happens: --reuse-values with --set-json global.gateway.hostnames=null drops the key from helm get values but keeps rendering the last list, as described under Rollback. Read tars-gateway-declaration to see what the data plane actually declares. Keep the list in the values file applied on every upgrade.

Step 2: apply and confirm the declaration​

  1. Run tetrate agentrouter dataplane upgrade or helm upgrade with the new values.

  2. Read what the chart rendered. The chart writes the list into the tars-gateway-declaration ConfigMap:

    kubectl -n tars-system get configmap tars-gateway-declaration -o jsonpath='{.data.gateway-hostnames}'
    ["eu.acme.example.com","dp.acme.example.com:8443"]

    An empty list renders []. A chart with the key left out renders no gateway-hostnames field at all, which the management plane reads as "declares nothing" rather than as an empty list.

  3. In the Admin Console, open Directory → Data Planes and select the data plane. The hostnames appear on its page once the data plane polls for configuration. A row tagged from the management plane is a stored address, which means the chart still declares nothing.

Step 3: make each name serve​

Declaring a name tells the management plane that the data plane serves it. The chart creates no DNS record, certificate, or load balancer route. There are 3 things each declared name needs before a caller reaches it:

  1. A DNS record that points the name at the data plane's load balancer.
  2. A certificate on the load balancer, or on the cluster's own listener, that covers the name.
  3. A route at the load balancer that sends requests for the name to the data plane. Without it the TLS handshake succeeds and the request answers 404.

GKE with the gateway-serve-helm chart​

The gateway-serve-helm chart does not read global.gateway.hostnames. Pass the same names to it as gatewayHostnames, so its HTTPRoute matches each one beside serveUrl:

# gateway-serve-helm values (gateway.yaml)
customer: acme
serveUrl: proxy.acme.example.com
certificateMap:
name: acme-serve-certmap
gatewayHostnames:
- eu.acme.example.com
- dp.acme.example.com

The chart drops a port, a scheme, and a path from each entry, because a Gateway API hostname carries no port. Confirm the route after the upgrade:

kubectl -n tars-gateway get httproute proxy-route -o jsonpath='{.spec.hostnames}'

The chart names the certificate map and never creates its entries. tetrate agentrouter dataplane ingress install creates one map entry, for the serve host only. Create a certificate and a map entry for every further name:

HOST=eu.acme.example.com
SLUG=$(echo "$HOST" | tr . -)
MAP=acme-serve-certmap

gcloud certificate-manager dns-authorizations create "$SLUG-dns-auth" --domain="$HOST"
# Publish the CNAME this prints in the zone that holds $HOST.
gcloud certificate-manager dns-authorizations describe "$SLUG-dns-auth" --format='value(dnsResourceRecord)'

gcloud certificate-manager certificates create "$SLUG" \
--domains="$HOST" --dns-authorizations="$SLUG-dns-auth"
gcloud certificate-manager maps entries create "$SLUG-entry" \
--map="$MAP" --certificates="$SLUG" --hostname="$HOST"

The certificate stays PROVISIONING until the DNS authorization record is published. Point the name's A record at the Gateway's address once the certificate is ACTIVE.

TLS on the data plane​

From 0.8.0 a declared name can be published on 443 by the data plane itself: list it under egressResources.egress.gateway.httpsListeners with the Secret that holds its certificate, and egress-lb passes 443 through to it. The data plane presents the certificate and routes the name, so only the DNS record is left to the operator. Publication is per hostname: a name listed there is published for every project the data plane serves, and a declared name left out of the list gets no listener on 443. See Terminate TLS on the data plane.

AWS, Azure, and other load balancers​

No chart provisions the edge on AWS or Azure. The DNS record, the certificate (an ACM certificate on an ALB or NLB, or a Key Vault certificate on an Application Gateway), and the listener or Ingress rule for each name are the operator's own. Where the Ingress or listener uses host-based rules, add a rule for each declared name. See Gateway installation on EKS and Gateway installation on AKS.

Step 4: check the certificate the data plane reads​

A data plane on 0.7.0 or later reads the certificate serving each declared name and reports it beside the name: the subject alternative names, the issuer, the fingerprint, the validity window, the address the handshake reached, and where the reading was taken from. By default it dials each declared name over HTTPS, honouring HTTPS_PROXY and NO_PROXY, so it reads what a load balancer in front of the cluster presents as well as a certificate on the cluster's own listener.

global.gateway.certificateProbe changes where the data plane dials. There are 3 shapes:

  1. A private load balancer the cluster reaches but cannot resolve by name. Name its address, and optionally a port, an SNI, and a CA bundle that decides whether the chain reads as trusted:

    global:
    gateway:
    certificateProbe:
    address: dp-internal.eu-west-1.elb.amazonaws.com
    port: 8443
    sni: eu.acme.example.com # defaults to each declared name
  2. A GKE global external load balancer. Name the forwarding rule's address and leave sni out, so each declared name is sent in turn and the certificate map answers with the certificate it holds for that name:

    global:
    gateway:
    certificateProbe:
    address: 34.120.0.1
  3. An endpoint nothing inside the trust boundary can reach, such as a DMZ appliance or a CDN whose origin is unroutable. Turn the dial off:

    global:
    gateway:
    certificateProbe:
    enabled: false

    Every address then reads as not observed and says the chart disabled the probe. The data plane stays healthy.

certificateProbe says where to look. Nothing in it changes what the certificate says, so an expired certificate never reads as healthy.

Run the same reading from the command line:

tetrate agentrouter doctor /path/to/data-plane-credentials.json --only gateway

tetrate agentrouter doctor reads the listener certificate where a listener carries one, and otherwise dials each declared address the same way the data plane does, honouring certificateProbe. It reports:

ReadingSeverity
The certificate does not cover a declared nameWARN, naming the hostname
The certificate has expired, or is not valid yetERROR, with the time validity starts or ended. A not-yet-valid certificate points at the cluster clock or the issuing CA
The certificate expires soonReported with the time left
The address could not be reachedINFO, with the openssl s_client command to read it by hand. The data plane stays healthy

Troubleshooting​

SymptomCauseFix
A declared name appears in the console but requests to it answer 404The load balancer has no route for the name. TLS completes, then nothing matchesAdd the name to the load balancer's routes. On GKE, add it to gatewayHostnames on gateway-serve-helm
The GKE certificate for a new name stays PROVISIONINGThe DNS authorization record is not publishedPublish the record that gcloud certificate-manager dns-authorizations describe prints, then wait for ACTIVE
CreateProjectGateway, UpdateProjectGateway, or SetProjectGatewayURL fails with FailedPrecondition and data plane <id> does not declare "<hostname>", including a console edit that resends an unchanged URLThe data plane declares hostnames, and the URL's hostname is not one of themAdd the hostname to the list and upgrade the chart. See Move a data plane to declared hostnames
/healthz/membership answers 404 on one declared name while others answer 200The name is missing from the rendered ConfigMap, or the request's Host carries a non-standard portRead tars-gateway-declaration as in Step 2. The probe answers on port 443 only
After an upgrade, /healthz/membership stops answering on some declared names while the console still lists themThe upgrade passed values without global.gateway.hostnames, so the key was dropped and the probe fell back to the stored addressRe-run the upgrade with the values file that carries the list, or use tetrate agentrouter dataplane upgrade, which carries the list forward
From 0.10.0, tetrate agentrouter doctor warns on its Readiness probe row that the data plane has no readiness probe route, and /healthz/membership answers 404 on every addressThe data plane declares no hostnames and the management plane stores no address for it, so no probe route is written. Traffic keeps working, but DNS failover treats the data plane as downList the names the data plane serves under global.gateway.hostnames and upgrade the chart. When names are declared and the warning stays, the liaison worker log is checked for Data plane probe. An empty list is a deliberate drain, which doctor reports as info. --only dataplane-probe runs the check alone
tetrate agentrouter doctor warns that the certificate does not cover a nameThe certificate's subject alternative names lack the declared nameReissue the certificate with the name, or remove the name from the list
From 0.8.0, GetDataplaneURL and ListDataplaneURLs return no URL for the data plane, or Prompt is refusedAnother organization uses the same data plane ID, and this data plane declares no hostnames. For a data plane whose ID is shared, the inference API returns only the first hostname the data plane declares, and never the URL stored under the shared IDList the data plane's hostnames under global.gateway.hostnames and upgrade the chart, as in Step 1
From 0.8.0, SetDataplaneURL or SetWorkspaceURL fails with FailedPrecondition, or CreateServiceAccount with a serve URL is refusedAnother organization uses the same data plane ID. The URL is stored under the ID alone, so the write would also change the other organization's data plane. CreateServiceAccount is refused only while no URL is stored for that IDInstall the data plane under an ID no other organization uses, then repeat the request for the new ID. The URL stored under the shared ID stays as it is, because it serves the other organization too

Move a data plane to declared hostnames​

Before 0.7.0 the management plane stored a data plane's address, written through SetDataplaneURL or the Admin Console, and each project gateway carried its own URL. From 0.7.0 a data plane declares its hostnames in its serve-helm values instead, and the stored calls are deprecated. A data plane crosses over the moment its chart lists the first name, so the list has to be complete in that one change.

Persona: Platform operator with access to the data plane's Helm values or tetrate identity file and to the Admin Console.

Estimated time: 15 to 30 minutes per data plane, plus one configuration poll.

Outcomes of the move​

By the end of this section:

  • The data plane's chart declares every hostname it serves, and the Admin Console lists each one as declared in the chart.
  • No project gateway on the data plane carries a hostname the chart does not declare, so later gateway edits are not refused.
  • The rollback path is known: emptying the list and then removing the key returns the data plane to its stored address.

Prerequisites for the move​

  • The data plane runs 0.7.0 or later. Upgrading to 0.7.0 with untouched values keeps it on the stored address and changes nothing about where it serves, so the upgrade itself is safe to take first. See Upgrade a self-hosted data plane.
  • The values file the data plane was installed with, for a Helm or GitOps install. CLI installs carry their values in the release.

What changes when the first name is declared​

There are 3 things that change on the data plane the moment its chart declares at least one hostname:

  1. The declared names replace the stored address everywhere: in the Admin Console, in the readiness probe, and in the address the console prints.
  2. CreateProjectGateway, UpdateProjectGateway, and SetProjectGatewayURL accept only a hostname the data plane declared. Any other hostname is refused with FailedPrecondition and a message that starts data plane <id> does not declare "<hostname>"; a hostname is declared in the data plane's own chart under gateway.hostnames and reported from there. A console or API edit that resends a gateway's own unchanged URL is refused the same way when that hostname is missing from the list.
  3. SetDataplaneURL no longer copies the data plane's address onto its default gateway. It still records the address, which the data plane uses again only after a rollback.

A data plane whose chart declares nothing keeps the pre-0.7 behavior for all three, so each data plane crosses over on its own schedule.

Step 1: inventory the hostnames in use​

This step only reads and records. Nothing is changed in the console.

  1. In the Admin Console, open Directory → Data Planes and select the data plane.
  2. Read the addresses it lists, one row each. While the chart declares nothing, the page lists the stored address first, then the URL of each project gateway on the data plane, each tagged from the management plane. The tag is shown on the data plane's page and in the Directory → Data Planes list, not in a project's hostname view. Note which rows carry it, because Step 3 compares against them. The page also offers the serve-helm values that declare the same addresses.
  3. Write down any other hostname callers use to reach the data plane that the page does not list, such as a client-facing failover name that DNS points at this data plane. A project's Data Planes tab lists the endpoints serving that project, which helps where callers use different hostnames per project.
  4. Decide which hostname callers should use. It goes first in the list written in Step 2, because the console prints the first hostname wherever it shows a single address.

The same list is available from the API, through the call that lists a data plane's hostnames. Each entry carries a source of chart or management-plane.

Step 2: declare the complete list in one change​

Declare every name from Step 1 at once. A partial list turns the gate on for the names left out, and the next edit of a gateway carrying one of them is refused.

With the tetrate CLI:

tetrate agentrouter dataplane upgrade /path/to/data-plane-credentials.json \
--gateway-hostname=ai.acme.example.com \
--gateway-hostname=proxy.acme.example.com \
--gateway-hostname=bedrock-team.acme.example.com

With Helm or GitOps, add the list to the values file and apply it with that file:

global:
gateway:
hostnames:
- ai.acme.example.com
- proxy.acme.example.com
- bedrock-team.acme.example.com
helm upgrade tars "oci://${PRIVATE_IMAGE_REGISTRY}/serve-helm" -f values.yaml

Each name keeps the DNS record, certificate, and load balancer route it already has. A name added for the first time needs all three; see Declare the hostnames a data plane serves.

Step 3: confirm the crossover​

  1. Read the declaration the upgrade wrote to the data plane. The output must match the list from Step 2 exactly:

    kubectl -n tars-system get configmap tars-gateway-declaration -o jsonpath='{.data.gateway-hostnames}'
  2. The data plane reports its hostnames to the management plane on its configuration polls, so the Admin Console reflects the change after the next poll. Reload the data plane's page until the rows that carried the from the management plane tag in Step 1 no longer carry it.

  3. Check the readiness probe on each hostname in the list. On a data plane on 0.7.0 or later the probe answers for the whole data plane. The data plane's periodic reconcile, the drift sweep, runs every five minutes, and until it runs the probe still reflects the previous list. A hostname from the new list can therefore answer 404 for up to five minutes after the upgrade without anything being wrong:

    curl -s https://ai.acme.example.com/healthz/membership
    {"status":"ready","dataplane":"dp-prod-use1"}

Step 4: clear certificate warnings​

Run the doctor check against the data plane:

tetrate agentrouter doctor /path/to/data-plane-credentials.json --only gateway

It warns for each declared hostname that the data plane's certificate does not cover. The Admin Console shows the same reading beside each hostname on the data plane's page. A warning clears when the certificate that terminates TLS for that hostname covers it; see Step 4: check the certificate the data plane reads.

Video walkthrough​

A recorded walkthrough of the steps above. The steps remain the reference.

Serve every project on every hostname​

Declaring hostnames does not change which projects a hostname serves. While the data plane carries two or more project gateways, each request is still matched on its hostname:

  • Each gateway's hostname serves only that gateway's project.
  • A declared name that no gateway carries serves no project. Every request on it is refused with 403.

A data plane serves its projects on every hostname only once a single project gateway remains on it. That gateway then serves every project labelled on the data plane, whatever hostname a request arrives on.

Keep the per-project gateways for now

In 0.7.0 and 0.8.0, two known issues affect a data plane that is reduced to one project gateway. The remaining gateway's hostname, usually the one callers already use, answers 404 with model_not_routed for most API keys and does not recover on its own. And once that gateway serves several projects, an API key can call models that only another project on the data plane enables. Until both are fixed, keep the per-project gateways and do not move several projects onto one gateway. See the known issues in the 0.8.0 release notes.

Getting there means labelling the other gateways' projects on the data plane, as in Manage a data plane's addresses and projects, and then deleting the other project gateways.

Draining a data plane​

Draining takes a data plane out of health-checked rotation by declaring that it serves no hostnames. It is used to retire a data plane on purpose, for example before decommissioning it, or to make every health checker stop sending it traffic.

Draining stops the readiness probe, not inference

The data plane keeps serving every caller that still reaches it, so two paths need separate handling:

  • A health checker or load balancer that polls /healthz/membership takes the data plane out of rotation once the probe answers 404. Draining does this.
  • A caller whose DNS points straight at the data plane is not affected and keeps sending it traffic. That needs a separate DNS change, made before the drain.

Declare an empty list. With the tetrate CLI:

tetrate agentrouter dataplane upgrade /path/to/data-plane-credentials.json --gateway-hostname=""

With Helm or GitOps, set an empty list in the values file and apply it with that file:

global:
gateway:
hostnames: []

What changes once the empty list is applied:

  • The management plane clears every hostname it recorded for the data plane.
  • The readiness probe answers 404 on every address. The probe changes at the data plane's next drift sweep, up to five minutes after the empty list is applied, so a health checker sees the old answer until then.
  • CreateProjectGateway, UpdateProjectGateway, and SetProjectGatewayURL accept a hostname the data plane does not declare again, as they did before 0.7.0.

To verify, wait out the drift sweep and request the probe on a hostname the data plane used to declare. It answers 404:

curl -s -o /dev/null -w '%{http_code}\n' https://ai.acme.example.com/healthz/membership

To undo a drain, apply the complete hostname list again, as in Step 2. The probe answers again at the next drift sweep.

An empty list is not the same as removing the key from the values file. The table under Rollback compares the two.

Rollback​

A missing or wrong name is fixed forward: correct the list and upgrade again, as in Step 2. That takes one upgrade and leaves the readiness probe answering. Roll back only to leave declared hostnames altogether.

Removing global.gateway.hostnames alone is not a rollback. A data plane whose chart carries no key reports nothing, and the management plane reads that as no change, so it keeps the names it last recorded. An empty list is the opposite half:

BehaviorKey removedEmpty list
Names the management plane recordsKept as last declaredCleared
Addresses the Admin Console listsThe declared namesThe stored address, tagged from the management plane
A gateway URL call with an undeclared hostnameStill refusedAccepted, as before 0.7.0
Readiness probe (/healthz/membership)Answers on the stored address and the project gateway hostnames, and 404 on any other declared nameAnswers 404 on every address

Neither change affects inference. A full rollback takes both, in this order:

  1. Empty the list: set hostnames: [] in the values file and apply it, or run tetrate agentrouter dataplane upgrade with --gateway-hostname="" alone, as in Draining a data plane.
  2. After the data plane's next configuration poll, reload its page in the Admin Console. The page lists the stored address again, tagged from the management plane.
  3. Remove the key from the values file and apply it with Helm. At the data plane's next drift sweep, up to five minutes later, the readiness probe answers on the stored address and the project gateway hostnames again, and the management plane keeps the cleared record.

From the first step until the last, the readiness probe answers 404 everywhere, so a health checker polling it takes the data plane out of rotation. Run the three steps back to back, or shift DNS away first.

After the last step, a declared name that is neither the stored address nor a project gateway's hostname keeps serving requests but goes on answering 404 on the readiness probe, because without a declaration the probe answers only on those two kinds of name. Every name added in Step 1 that was not already a gateway hostname is in this position. A health checker that polls such a name is pointed at the stored address before the rollback, or it takes the data plane out of rotation for good.

tetrate agentrouter dataplane upgrade cannot remove the key: without --gateway-hostname it carries the installed list forward, an empty one included. On a CLI install the last step is therefore a Helm upgrade of the tars release with its current values minus the key. Write the values to a file, delete hostnames under global.gateway, and apply the file without --reuse-values:

helm get values tars -n tars-system -o yaml > values.yaml
# delete the hostnames key under global.gateway in values.yaml
helm upgrade tars "oci://${PRIVATE_IMAGE_REGISTRY}/serve-helm" -n tars-system -f values.yaml
kubectl -n tars-system get configmap tars-gateway-declaration -o jsonpath='{.data.gateway-hostnames}'

The last command prints nothing once the key is gone. Do not use --reuse-values --set-json global.gateway.hostnames=null instead. On Helm 4.1 it removes the key from helm get values but still renders the last list, [], into tars-gateway-declaration, so the data plane stays drained and the readiness probe keeps answering 404 on every name.

Declaring hostnames never deletes the address stored through SetDataplaneURL, so nothing has to be re-entered.

Keep the list on every upgrade​

An upgrade that passes values without global.gateway.hostnames removes the key, with the effect described in the table under Rollback. The backup steps, and which upgrade commands carry the list forward, are in Back up before you upgrade.

Deprecation timeline​

SetDataplaneURL, its SetWorkspaceURL alias, SetProjectGatewayURL, and the url field of CreateProjectGateway and UpdateProjectGateway are deprecated in 0.7.0. They keep working until no supported data plane predates 0.7.0, because a data plane before 0.7.0 cannot declare hostnames. Keep using SetDataplaneURL for such a data plane until it is upgraded. The Admin Console on 0.7.0 no longer has a field that writes either URL. See Deprecations.

From 0.8.0, SetDataplaneURL and SetWorkspaceURL refuse to set or clear a URL with FailedPrecondition when another organization uses the same data plane ID, because the URL is stored under the ID alone and the write would also change the other organization's data plane. For the same reason, the inference API no longer returns a URL stored under a shared ID: GetDataplaneURL, ListDataplaneURLs, and Prompt reach such a data plane only through the hostnames it declares under global.gateway.hostnames. A data plane whose ID is shared therefore either declares its hostnames, which needs 0.7.0 or later, or is installed under an ID no other organization uses. The URL stored under the shared ID stays as it is.