{"record":{"id":"eeca98b6920ed01e","repo":"hashicorp/nomad","slug":"error-creating-bootstrap-configuration-for-connect","errorCode":null,"errorMessage":"error creating bootstrap configuration for Connect proxy sidecar","messagePattern":"error creating bootstrap configuration for Connect proxy sidecar","errorType":"exception","errorClass":"errEnvoyBootstrapError","httpStatus":null,"severity":"error","filePath":"client/allocrunner/taskrunner/envoy_bootstrap_hook.go","lineNumber":50,"sourceCode":"const envoyBootstrapHookName = \"envoy_bootstrap\"\n\nconst (\n\t// envoyBootstrapWaitTime is the amount of time this hook should wait on Consul\n\t// objects to be created before giving up.\n\tenvoyBootstrapWaitTime = 60 * time.Second\n\n\t// envoyBootstrapInitialGap is the initial amount of time the envoy bootstrap\n\t// retry loop will wait, exponentially increasing each iteration, not including\n\t// jitter.\n\tenvoyBootstrapInitialGap = 1 * time.Second\n\n\t// envoyBootstrapMaxJitter is the maximum amount of jitter applied to the\n\t// wait gap each iteration of the envoy bootstrap retry loop.\n\tenvoyBootstrapMaxJitter = 500 * time.Millisecond\n)\n\nvar (\n\terrEnvoyBootstrapError = errors.New(\"error creating bootstrap configuration for Connect proxy sidecar\")\n)\n\ntype consulTransportConfig struct {\n\tHTTPAddr   string // required\n\tAuth       string // optional, env CONSUL_HTTP_AUTH\n\tSSL        string // optional, env CONSUL_HTTP_SSL\n\tVerifySSL  string // optional, env CONSUL_HTTP_SSL_VERIFY\n\tGRPCCAFile string // optional, arg -grpc-ca-file\n\tCAFile     string // optional, arg -ca-file\n\tCertFile   string // optional, arg -client-cert\n\tKeyFile    string // optional, arg -client-key\n\tNamespace  string // optional, only consul Enterprise, env CONSUL_NAMESPACE\n\t// CAPath (dir) not supported by Nomad's config object\n}\n\nfunc newConsulTransportConfig(cc *config.ConsulConfig) consulTransportConfig {\n\treturn consulTransportConfig{\n\t\tHTTPAddr:   cc.Addr,","sourceCodeStart":32,"sourceCodeEnd":68,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/client/allocrunner/taskrunner/envoy_bootstrap_hook.go#L32-L68","documentation":"errEnvoyBootstrapError is the sentinel for failures while building the Envoy bootstrap configuration for a Consul Connect proxy/gateway task. It is wrapped (fmt.Errorf(\"%w: %v; see: <https://developer.hashicorp.com/nomad/s/envoy-bootstrap-error>\")) inside a recoverable error both when the `consul connect envoy -bootstrap` command fails (line 414) and when the Consul API returns an error (line 681). Because it's recoverable, Nomad retries the Prestart hook with backoff/jitter (envoyBootstrapMaxJitter) before failing the task.","triggerScenarios":"Prestart of the envoy bootstrap hook runs `consul connect envoy -bootstrap` (or calls the Consul xDS API) and the command exits non-zero or the API errors: Consul agent unreachable/ACL denied, Consul HTTP addr mismatch, TLS/auth problems, unsupported envoy/consul version combination, or service kind not registered yet in Consul.","commonSituations":"CONSUL_HTTP_ADDR unset or wrong on the node; Consul ACL token missing envoy-service permission (service:write, node:read, etc.); mTLS misconfigured between Nomad and Consul; envoy binary version not supported by the installed Consul version; Consul agent down or restarted during job deploy.","solutions":["Read the wrapped sub-error in the task log (it includes the underlying command/API message and the fix guide URL); address that root cause directly.","Verify Consul connectivity from the client: CONSUL_HTTP_ADDR, TLS (ca/cert/key) and ACL token settings in the nomad agent consul block; test with `consul members` and `consul connect envoy -bootstrap` manually.","Check ACL permissions: the token needs service:write for the connect service and node:read; update the policy.","Confirm envoy version compatibility (consul version ↔ envoy version matrix) and reinstall the matching envoy binary.","If transient (Consul restart), let the recoverable retry succeed; otherwise nomad alloc stop and reschedule after fixing Consul."],"exampleFix":"// before: nomad agent lacking consul ACL config\nconsul {\n  address = \"127.0.0.1:8500\"\n}\n// after:\nconsul {\n  address = \"127.0.0.1:8500\"\n  token   = \"<token-with-service:write-and-node:read>\"\n  grpc_address = \"127.0.0.1:8502\"\n  tls {\n    ca_file   = \"/etc/consul/tls/ca.pem\"\n    cert_file = \"/etc/consul/tls/cli.pem\"\n    key_file  = \"/etc/consul/tls/cli-key.pem\"\n  }\n}","handlingStrategy":"retry","validationCode":"// node-level preflight before running Connect workloads\n// CONSUL_HTTP_ADDR must be reachable, and consul version pairs with envoy version\nconsul members >/dev/null && echo consul-ok\nconsul version && envoy --version","typeGuard":null,"tryCatchPattern":"// hook-level: the error is recoverable; emulate its retry in tooling\nfor i := 0; i < 5; i++ {\n  err := bootstrapEnvoy()\n  if err == nil || errors.Is(err, errEnvoyBootstrapError) {\n    time.Sleep(time.Duration(rand.Intn(500))*time.Millisecond)\n    continue\n  }\n  break\n}","preventionTips":["Set consul block (address, grpc, token, TLS) fully on every nomad client.","Grant ACL tokens service:write on connect services plus node:read, and agent/service mesh permissions.","Keep envoy and consul versions on a tested compatibility matrix.","Alert on repeated recoverable envoy bootstrap failures — they indicate persistent Consul misconfig, not flakiness."],"tags":["consul","connect","envoy","nomad","recoverable"],"backgroundTag":"envoy-bootstrap-failed","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}