Skip to main content

Gateway installation guide for EKS

This guide exposes the Agent Router data plane in an Amazon Elastic Kubernetes Service (EKS) cluster using the tare command-line utility. The install drives a Network Load Balancer through the egress EnvoyProxy, so the endpoint answers on a customer-facing hostname without an AWS Load Balancer Controller and without the aws CLI.

By the end of this guide the cluster has:

  • The egress EnvoyProxy (tars-egress-proxy in tars-system) patched to expose its service as a LoadBalancer, annotated for a Network Load Balancer.
  • The egress Service in tars-dataplane reporting an NLB hostname.
  • A customer-facing DNS CNAME record pointing at that NLB hostname.

Plan for 10--20 minutes end-to-end. The NLB itself usually publishes a hostname within a minute of the patch.

Architecture​

┌───────────────────────────────────────────┐
│  proxy.<your-domain>  ←  customer DNS     │
│          │              (CNAME)           │
│          ▼                                │
│  ┌────────────────┐                       │
│  │  Network Load  │  ← provisioned by the │
│  │    Balancer    │    in-tree EKS cloud  │
│  └────────┬───────┘    provider           │
│           │                               │
│  ┌────────▼───────┐                       │
│  │ egress Service │  ← reconciled by      │
│  │ (tars-dataplane)│   Envoy Gateway from │
│  └────────┬───────┘    the EnvoyProxy     │
│           │                               │
│  ┌────────▼───────┐                       │
│  │  Agent Router  │                       │
│  │   Dataplane    │  ← installed earlier  │
│  └────────────────┘    via tare CLI       │
└───────────────────────────────────────────┘

This is a layer 4 path. The tare utility patches one resource: the egress EnvoyProxy. It sets spec.provider.kubernetes.envoyService.type to LoadBalancer and adds service.beta.kubernetes.io/aws-load-balancer-type: nlb. Envoy Gateway then reconciles the egress Service, and the in-tree EKS cloud provider provisions the NLB from the annotation. The operator provides one customer-facing DNS record at the top.

Driving the load balancer through the EnvoyProxy is what makes the exposure durable. The egress Service is owned by Envoy Gateway, so a direct Service patch reverts to ClusterIP on the next reconcile; the EnvoyProxy setting is re-asserted across pod restarts, Envoy Gateway restarts, and tare upgrade.

info

Use this guide when:

  • An EKS cluster already exists and the data plane is installed (tare install ... has already run).
  • A layer 4 endpoint is sufficient and TLS is terminated upstream of the load balancer or by the client.
  • A reproducible, scripted exposure is preferred over hand-patching the egress Service or installing an ingress controller.

Use the ALB path in the AWS data plane installation guide instead when:

  • Layer 7 ingress is required, or TLS must terminate at the load balancer with an AWS Certificate Manager certificate. Steps 10 and 11 of that guide install the AWS Load Balancer Controller and create an ALB Ingress.
  • The data plane is not yet installed. Gateway install needs the egress EnvoyProxy to exist before it can patch it.

The two paths are alternatives, not a sequence. Pick one.

Prerequisites​

  • tare CLI v0.1.0-beta.2 or later. Install or upgrade with curl -fsSL https://tare.tetrate.ai/tools/install.sh | bash. The binary lands at ~/.tare/bin/tare; add it to PATH.
  • kubectl pointed at the EKS cluster (aws eks update-kubeconfig --region $REGION --name $CLUSTER); confirm with kubectl config current-context.
  • Data plane already installed, so the egress EnvoyProxy exists: kubectl get envoyproxy tars-egress-proxy -n tars-system returns a resource.
  • identity.json (the data plane credential and service-account file). The same file used for tare install.
  • Kubernetes RBAC allowing patch on envoyproxies.gateway.envoyproxy.io in tars-system. The pre-flight runs kubectl auth can-i and fails with a pointer to the required permission when it is denied.

No aws CLI, no AWS Load Balancer Controller, and no IAM policy work are required for this path. The --apply-prereqs and --ack-prereqs flags belong to the GCP and Azure paths and do not apply here.

note

Credential-less upstream authentication to Bedrock is a separate concern from ingress. See Bedrock workload identity for IRSA and EKS Pod Identity.

Conventions​

The commands below assume the following environment variables are exported in the current shell:

export SERVE_DOMAIN="proxy.acme.example.com"
export REGION="<eks-region>"
export CLUSTER="<eks-cluster-name>"

Each step assumes these variables remain set; export them again after opening a new shell.

Step 1: preview the patch​

tare gateway install identity.json --type aws --plan-only

The Plan view lists the EnvoyProxy that will be patched, the annotations that will be applied, and the Service that Envoy Gateway will reconcile into an NLB. Neither --plan-only nor --print-resources touches the cluster, so both are safe to run first.

To see the exact merge patch instead of the summary:

tare gateway install identity.json --type aws --print-resources

Unlike the GCP and Azure paths, --type aws takes no config file. It is configured entirely by flags.

Step 2: expose the egress as an NLB​

tare gateway install identity.json --type aws --wait

--wait polls the egress Service until the cloud provider publishes a hostname; NLBs surface a hostname rather than an IP address. Add --yes to skip the Proceed? prompt, which is required in non-TTY contexts such as CI. The wait timeout defaults to 10m and is set with --timeout.

For an internal NLB that is reachable only from inside the VPC:

tare gateway install identity.json --type aws --aws-internal --wait

This adds service.beta.kubernetes.io/aws-load-balancer-internal: true to the same patch.

Install sequence​

The install proceeds through the following stages:

  1. Plan and confirm - the plan prints before any cluster access, then the CLI prompts unless --yes is passed.
  2. Pre-flight kubectl and cluster - verifies kubectl is available and the cluster is reachable.
  3. Pre-flight EnvoyProxy - confirms tars-egress-proxy exists in tars-system and that the current context can patch it. A missing resource fails with an "is the data plane installed?" hint.
  4. Patch the EnvoyProxy - a JSON merge patch sets envoyService.type=LoadBalancer and the NLB annotations, preserving the rest of provider.kubernetes.
  5. --wait - polls status.loadBalancer.ingress[0].hostname on the egress Service until the NLB hostname appears.
  6. Completion output - prints the DNS CNAME action and a smoke-test command.

Flags for the AWS path​

FlagDefaultDescription
--aws-internalfalseProvision an internal NLB (adds the aws-load-balancer-internal annotation)
--aws-serviceegressEgress Service that Envoy Gateway exposes as an NLB
--aws-envoyproxytars-egress-proxyEgress EnvoyProxy resource to drive the NLB through

Step 3: wire DNS and verify​

Read the NLB hostname from the Service:

kubectl get service egress -n tars-dataplane \
  -o jsonpath='{.status.loadBalancer.ingress[0].hostname}'
# k8s-tarsdata-egress-xxxxxxxxxx-yyyyyyyyy.elb.us-east-1.amazonaws.com

Point the serve hostname at that value with a CNAME record at your DNS provider:

proxy.acme.example.com  CNAME  k8s-tarsdata-egress-xxxxxxxxxx-yyyyyyyyy.elb.us-east-1.amazonaws.com

Smoke-test the endpoint on the egress port:

curl -sk -o /dev/null -w 'HTTP %{http_code}\n' \
  http://<nlb-hostname>:10080/v1/chat/completions
# expected: HTTP 401

401 is the success signal: the request reached the tars-proxy filter, which is asking for an API key. 200 means the filter is not loaded, and 502 means the load balancer health check is failing.

Troubleshooting​

EnvoyProxy "tars-egress-proxy" not found​

The data plane is not installed in the current context, or the EnvoyProxy is named differently. Run tare install first, or pass the actual name:

tare gateway install identity.json --type aws --aws-envoyproxy <name> --wait

Egress service stays ClusterIP after patching​

Confirm the EnvoyProxy carries the setting:

kubectl get envoyproxy tars-egress-proxy -n tars-system \
  -o jsonpath='{.spec.provider.kubernetes.envoyService.type}'
# expected: LoadBalancer

When the EnvoyProxy reads LoadBalancer but the Service does not, Envoy Gateway has not reconciled it. Check the envoy-gateway controller logs.

NLB hostname never appears​

Inspect the Service events for provisioning errors:

kubectl describe service egress -n tars-dataplane

The in-tree cloud provider discovers subnets by tag. Missing or incorrect kubernetes.io/role/elb subnet tags are the usual cause.

Kubernetes RBAC denies the patch​

The pre-flight reports that the current context cannot patch envoyproxies.gateway.envoyproxy.io. Grant a role with patch on that resource in tars-system and rerun.

Notes​

  • The exposure is durable. Because Envoy Gateway owns the egress Service through the EnvoyProxy, the NLB is re-asserted across pod restarts, Envoy Gateway restarts, and tare upgrade.
  • The install is idempotent. Re-running applies the same merge patch, so it is safe to repeat after changing --aws-internal.
  • This path handles ingress only. Upstream provider credentials, including credential-less Bedrock access, are configured separately.

Where to go next

Bedrock workload identity

Give the data plane credential-less access to Bedrock with IRSA or EKS Pod Identity.

Console quickstart

Issue an API key and confirm a first routed request through the new endpoint.