{"record":{"id":"15d345b3c363a790","repo":"hashicorp/nomad","slug":"unexpectedly-did-not-forward-variables-list-to-reg","errorCode":null,"errorMessage":"unexpectedly did not forward Variables.List to region %q","messagePattern":"unexpectedly did not forward Variables\\.List to region %q","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"nomad/namespace_endpoint.go","lineNumber":415,"sourceCode":"\treturn true, nil\n}\n\n// namespaceNoAssociatedVarsInRegion returns true if there are no variables\n// associated with the namespace in the given region.\nfunc (n *Namespace) namespaceNoAssociatedVarsInRegion(authToken, namespace, region string) (bool, error) {\n\tvarReq := &structs.VariablesListRequest{\n\t\tQueryOptions: structs.QueryOptions{\n\t\t\tRegion:     region,\n\t\t\tNamespace:  namespace,\n\t\t\tAllowStale: false,\n\t\t\tAuthToken:  authToken,\n\t\t},\n\t}\n\n\tvar varResp structs.VariablesListResponse\n\tdone, err := n.srv.forward(\"Variables.List\", varReq, varReq, &varResp)\n\tif !done {\n\t\treturn false, fmt.Errorf(\"unexpectedly did not forward Variables.List to region %q\", region)\n\t} else if err != nil {\n\t\treturn false, err\n\t}\n\n\tfor _, v := range varResp.Data {\n\t\tif v.Namespace == namespace {\n\t\t\treturn false, nil\n\t\t}\n\t}\n\treturn true, nil\n}\n\n// ListNamespaces is used to list the namespaces\nfunc (n *Namespace) ListNamespaces(args *structs.NamespaceListRequest, reply *structs.NamespaceListResponse) error {\n\n\tauthErr := n.srv.Authenticate(n.ctx, args)\n\tif done, err := n.srv.forward(\"Namespace.ListNamespaces\", args, args, reply); done {\n\t\treturn err","sourceCodeStart":397,"sourceCodeEnd":433,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/nomad/namespace_endpoint.go#L397-L433","documentation":"namespaceNoAssociatedVarsInRegion forwards a Variables.List RPC to a remote region to verify no variables remain in the namespace before deletion. If the forward helper returns done=false — the RPC was not actually sent to the remote region's server — this internal error is returned naming the region. It is an invariant check on the forwarding layer, not a user data problem.","triggerScenarios":"Namespace delete triggering the variables check across regions; forwarding to a peer region fails to dispatch because the region is down, not federated, or the region name is wrong.","commonSituations":"Multi-region Nomad with variables in use; a region that lost gossip connectivity; tooling using region names that do not match server configuration.","solutions":["Restore federation/connectivity to the affected region (verify nomad server members and RPC logs).","Correct the region name used by clients/servers if misspelled.","Retry the namespace deletion once the region is healthy."],"exampleFix":null,"handlingStrategy":"retry","validationCode":"for _, region := range regions {\n    if _, _, err := client.Agent().MembersOpts(&api.QueryOptions{Region: region}); err != nil {\n        return fmt.Errorf(\"region %s unreachable before variables check: %w\", region, err)\n    }\n}","typeGuard":null,"tryCatchPattern":"if err != nil && strings.Contains(err.Error(), \"did not forward Variables.List\") {\n    // check region health/federation, restore, retry\n}","preventionTips":["Verify federation health before cross-region namespace deletes.","Keep server region configuration consistent across all servers.","Retry with backoff after connectivity is restored."],"tags":["nomad","rpc-forwarding","variables","multi-region"],"backgroundTag":"rpc-forward-failed","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}