{"record":{"id":"29cefc587db9a99f","repo":"hashicorp/nomad","slug":"failed-to-retrieve-namespaces-from-consul-w","errorCode":null,"errorMessage":"failed to retrieve namespaces from consul: %w","messagePattern":"failed to retrieve namespaces from consul: %w","errorType":"console","errorClass":null,"httpStatus":null,"severity":"error","filePath":"command/agent/consul/service_client.go","lineNumber":1778,"sourceCode":"// AllocRegistrations returns the registrations for the given allocation. If the\n// allocation has no registrations, the response is a nil object.\nfunc (c *ServiceClient) AllocRegistrations(allocID string) (*serviceregistration.AllocRegistration, error) {\n\t// Get the internal struct using the lock\n\tc.allocRegistrationsLock.RLock()\n\tregInternal, ok := c.allocRegistrations[allocID]\n\tif !ok {\n\t\tc.allocRegistrationsLock.RUnlock()\n\t\treturn nil, nil\n\t}\n\n\t// Copy so we don't expose internal structs\n\treg := regInternal.Copy()\n\tc.allocRegistrationsLock.RUnlock()\n\n\t// Get the list of all namespaces created so we can iterate them.\n\tnamespaces, err := c.namespacesClient.List()\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"failed to retrieve namespaces from consul: %w\", err)\n\t}\n\n\tservices := make(map[string]*api.AgentService)\n\tchecks := make(map[string]*api.AgentCheck)\n\n\t// Query the services and checks to populate the allocation registrations.\n\t// Note: these queries have to use the Nomad agent's own Consul token\n\tfor _, namespace := range namespaces {\n\t\tqo := &api.QueryOptions{\n\t\t\tNamespace: normalizeNamespace(namespace),\n\t\t}\n\n\t\tnsServices, err := c.agentAPI.ServicesWithFilterOpts(\"\", qo)\n\t\tif err != nil {\n\t\t\treturn nil, fmt.Errorf(\"failed to retrieve services from consul: %w\", err)\n\t\t}\n\t\tmaps.Copy(services, nsServices)\n","sourceCodeStart":1760,"sourceCodeEnd":1796,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/command/agent/consul/service_client.go#L1760-L1796","documentation":"When collecting all alloc registrations for self-healing (serviceClient.AllocRegistrations), Nomad lists Consul namespaces via the namespaces client. If the Consul API returns an error (Enterprise feature, permissions, connectivity), this wrapper aborts the listing, so allocation service/check state cannot be reconciled.","triggerScenarios":"AllocRegistrations calls c.namespacesClient.List() which hits the Consul /v1/namespaces endpoint; any API error — 403 ACL lacking 'namespace:list', Consul OSS lacking namespaces endpoint (404), or network failure — is wrapped here.","commonSituations":"Nomad configured with consul.namespace on Consul OSS (namespaces unsupported); Nomad ACL token missing the namespace:read/list permissions; Consul agent unreachable; enterprise Consul older than 1.7.","solutions":["Use Consul Enterprise (>=1.7) or remove consul.namespace config if running Consul OSS","Grant the Nomad Consul ACL token namespace list/read permissions (operator namespace list, service:write per namespace)","Verify connectivity from the Nomad client to the Consul agent HTTP endpoint","Check the wrapped %w error for 403 vs 404 to distinguish ACL vs missing-feature"],"exampleFix":"# before (consul stanza)\nconsul {\n  namespace = \"nomad\" # OSS Consul has no namespaces\n}\n# after\nconsul {\n  # namespace removed for Consul OSS\n}","handlingStrategy":"retry","validationCode":"// Pre-flight: confirm the Consul deployment supports namespaces and the token can list them\nns, _, err := consulClient.Namespaces().List(nil)\nif err != nil {\n  log.Printf(\"namespaces unavailable (OSS Consul or ACL gap): %v — unset consul.namespace\", err)\n}","typeGuard":null,"tryCatchPattern":"regs, err := client.AllocRegistrations(allocID)\nif err != nil && strings.Contains(err.Error(), \"failed to retrieve namespaces\") {\n  // reconcile later; check ACL token / Consul edition\n} else if err != nil {\n  log.Fatal(err)\n}","preventionTips":["Only set consul.namespace on Consul Enterprise >= 1.7","Grant the Nomad token namespace list/read ACLs","Monitor Consul agent reachability from Nomad clients","Distinguish 403 (ACL) from 404 (no namespace support) in the wrapped error"],"tags":["consul","nomad","namespaces","acl"],"backgroundTag":"consul-api-request-failed","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}