error retrieving supported Envoy versions from Consul: %w

Error message

error retrieving supported Envoy versions from Consul: %w

What it means

The envoy_version_hook Prestart queries Consul's local agent API (via proxiesClientFunc) to get the list of Envoy versions the Consul version supports. If that Proxies() call fails, the hook cannot determine which Envoy image to use and wraps the error. The caller may treat Consul API transport/HTTP errors as recoverable and retry.

Source

Thrown at client/allocrunner/taskrunner/envoy_version_hook.go:89

	// but could be a no-op or some other value if so configured.
	h.interpolateImage(request.Task, request.TaskEnv)

	// Detect whether this hook needs to run and return early if not. Only run if:
	// - task uses docker driver
	// - task is a connect sidecar or gateway
	// - task image needs ${NOMAD_envoy_version} resolved
	if h.skip(request) {
		return nil
	}

	// We either need to acquire Consul's preferred Envoy version or fallback
	// to the legacy default. Query Consul and use the (possibly empty) result.
	//
	// TODO(tgross): how do we select the right cluster here if we have multiple
	// services which could have their own cluster field value?
	proxies, err := h.proxiesClientFunc(structs.ConsulDefaultCluster).Proxies()
	if err != nil {
		return fmt.Errorf("error retrieving supported Envoy versions from Consul: %w", err)
	}

	// Second [pseudo] interpolation of task image. This determines the concrete
	// Envoy image identifier by applying version string substitution of
	// ${NOMAD_envoy_version} acquired from Consul.
	image, err := h.tweakImage(h.taskImage(request.Task.Config), proxies)
	if err != nil {
		return fmt.Errorf("error interpreting desired Envoy version from Consul: %w", err)
	}

	// Set the resulting image.
	h.logger.Trace("setting task envoy image", "image", image)
	request.Task.Config["image"] = image
	return nil
}

// interpolateImage applies the first pass of interpolation on the task's
// config.image value. This is where ${meta.connect.sidecar_image} or

View on GitHub (pinned to 482b49bf1a)

Solutions

Verify the Consul agent is healthy and reachable from the Nomad client (consul members, curl the agent API).
Upgrade Consul to a version that supports the local agent proxies endpoint if you see 404.
Check the Consul ACL token's agent/service read permissions.
If the error is recoverable, rely on Nomad's retry/restart; otherwise fix connectivity and reschedule the task.

Defensive patterns

Strategy: retry

Validate before calling

// verify Consul agent API reachability before Prestart
resp, err := http.Get("http://" + consulHTTPAddr + "/v1/agent/self")
if err != nil { return err }
if resp.StatusCode != 200 { return fmt.Errorf("consul agent API returned %d", resp.StatusCode) }

Try / catch

err := hook.Prestart(ctx, req)
var rec *structs.RecoverableError
if err != nil && errors.As(err, &rec) && rec.IsRecoverable() {
    // transient Consul/network issue: retry with backoff
} else if err != nil {
    // check Consul version/ACLs; the proxies endpoint may be unsupported
}

Prevention

Run a Consul version that supports the supported-proxies endpoint used by Nomad.
Monitor Consul agent health from all Nomad clients.
Grant the Consul token agent-read ACL permissions.
Allow Nomad's recoverable-error retry to absorb transient Consul flaps.

When it happens

Trigger: h.proxiesClientFunc(structs.ConsulDefaultCluster).Proxies() returns an error in Prestart — Consul agent unreachable, API endpoint unsupported (older Consul without /agent/proxies), ACL denied, or HTTP 5xx.

Common situations: Consul agent not running or network blocked; Consul version older than the supported-proxies endpoint; Consul ACL token lacking agent read permissions; transient network flaps during task start.

Related errors

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/8560c8b68bad81fd. Report an issue: GitHub.