{"record":{"id":"4a60a26d868ee6fd","repo":"hashicorp/nomad","slug":"unexpectedly-did-not-forward-csivolume-list-to-reg","errorCode":null,"errorMessage":"unexpectedly did not forward CSIVolume.List to region %q","messagePattern":"unexpectedly did not forward CSIVolume\\.List to region %q","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"nomad/namespace_endpoint.go","lineNumber":387,"sourceCode":"\treturn true, nil\n}\n\n// namespaceNoAssociatedVolumesInRegion returns true if there are no volumes\n// associated with the namespace in the given region.\nfunc (n *Namespace) namespaceNoAssociatedVolumesInRegion(authToken, namespace, region string) (bool, error) {\n\tvolumesReq := &structs.CSIVolumeListRequest{\n\t\tQueryOptions: structs.QueryOptions{\n\t\t\tRegion:     region,\n\t\t\tNamespace:  namespace,\n\t\t\tAllowStale: false,\n\t\t\tAuthToken:  authToken,\n\t\t},\n\t}\n\n\tvar volumesResp structs.CSIVolumeListResponse\n\tdone, err := n.srv.forward(\"CSIVolume.List\", volumesReq, volumesReq, &volumesResp)\n\tif !done {\n\t\treturn false, fmt.Errorf(\"unexpectedly did not forward CSIVolume.List to region %q\", region)\n\t} else if err != nil {\n\t\treturn false, err\n\t}\n\n\tfor _, volume := range volumesResp.Volumes {\n\t\tif volume.Namespace == namespace {\n\t\t\treturn false, nil\n\t\t}\n\t}\n\treturn true, nil\n}\n\n// namespaceNoAssociatedVarsInRegion returns true if there are no variables\n// associated with the namespace in the given region.\nfunc (n *Namespace) namespaceNoAssociatedVarsInRegion(authToken, namespace, region string) (bool, error) {\n\tvarReq := &structs.VariablesListRequest{\n\t\tQueryOptions: structs.QueryOptions{\n\t\t\tRegion:     region,","sourceCodeStart":369,"sourceCodeEnd":405,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/nomad/namespace_endpoint.go#L369-L405","documentation":"namespaceNoAssociatedVolumesInRegion forwards CSIVolume.List to a remote region to ensure no CSI volumes remain in the namespace before deletion. When the forward helper returns done=false (request not dispatched to the remote server), this internal error is returned with the region name and the namespace delete aborts.","triggerScenarios":"Deleting a namespace whose CSI volumes must be checked in another region where forwarding fails; peer region unreachable; routing failure producing done=false.","commonSituations":"Multi-region clusters using CSI storage plugins with an offline region; federation misconfiguration; stale region entries after decommissioning a datacenter.","solutions":["Check connectivity/federation health to the target region before deleting.","Inspect CSI volume state per region (nomad volume status) once reachable.","Fix server region configuration and retry the deletion."],"exampleFix":null,"handlingStrategy":"retry","validationCode":"for _, region := range regions {\n    if _, _, err := client.Agent().MembersOpts(&api.QueryOptions{Region: region}); err != nil {\n        return fmt.Errorf(\"region %s unreachable before CSI volume check: %w\", region, err)\n    }\n}","typeGuard":null,"tryCatchPattern":"if err != nil && strings.Contains(err.Error(), \"did not forward CSIVolume.List\") {\n    // restore region federation/connectivity, verify volume state, retry\n}","preventionTips":["Keep CSI plugins and regions healthy before namespace cleanup.","Monitor federation health across datacenters.","Remove stale region entries from tooling after decommissioning a DC."],"tags":["nomad","rpc-forwarding","csi","multi-region"],"backgroundTag":"rpc-forward-failed","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}