{"record":{"id":"b85399e7b05d7b12","repo":"hashicorp/nomad","slug":"w-s","errorCode":null,"errorMessage":"%w %s","messagePattern":"%w %s","errorType":"exception","errorClass":"ErrUnknownNode","httpStatus":null,"severity":"error","filePath":"nomad/csi_endpoint.go","lineNumber":561,"sourceCode":"\t}\n\tif alloc == nil {\n\t\treturn fmt.Errorf(\"%s: %s\", structs.ErrUnknownAllocationPrefix, req.AllocationID)\n\t}\n\n\t// Some plugins support controllers for create/snapshot but not attach. So\n\t// if there's no plugin or the plugin doesn't attach volumes, then we can\n\t// skip the controller publish workflow and return nil.\n\tif plug == nil || !plug.HasControllerCapability(structs.CSIControllerSupportsAttachDetach) {\n\t\treturn nil\n\t}\n\n\t// get Nomad's ID for the client node (not the storage provider's ID)\n\ttargetNode, err := state.NodeByID(ws, alloc.NodeID)\n\tif err != nil {\n\t\treturn err\n\t}\n\tif targetNode == nil {\n\t\treturn fmt.Errorf(\"%w %s\", structs.ErrUnknownNode, alloc.NodeID)\n\t}\n\n\t// if the RPC is sent by a client node, it may not know the claim's\n\t// external node ID.\n\tif req.ExternalNodeID == \"\" {\n\t\texternalNodeID, err := v.lookupExternalNodeID(vol, req.ToClaim())\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"missing external node ID: %v\", err)\n\t\t}\n\t\treq.ExternalNodeID = externalNodeID\n\t}\n\n\tmethod := \"ClientCSI.ControllerAttachVolume\"\n\tcReq := &cstructs.ClientCSIControllerAttachVolumeRequest{\n\t\tVolumeID:        vol.RemoteID(),\n\t\tClientCSINodeID: req.ExternalNodeID,\n\t\tAttachmentMode:  req.AttachmentMode,\n\t\tAccessMode:      req.AccessMode,","sourceCodeStart":543,"sourceCodeEnd":579,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/nomad/csi_endpoint.go#L543-L579","documentation":"controllerPublishVolume wraps structs.ErrUnknownNode when the allocation's node (alloc.NodeID) cannot be found in the Nomad state store. The error carries the node ID so operators can identify which client is missing. Nomad must resolve the Nomad node before it can map it to the storage provider's external node ID for the CSI ControllerAttachVolume RPC.","triggerScenarios":"Raised in controllerPublishVolume (via CSIVolume.Claim) when state.NodeByID(ws, alloc.NodeID) returns nil: the allocation exists but the node it was placed on has been deregistered/garbage-collected from the node registry. Typically the node failed its heartbeat and was GC'd while an alloc on it still claims a volume.","commonSituations":"A client node lost connectivity long enough for its heartbeat to expire and the server garbage-collected it; its allocations remain and a volume claim retry hits the missing node. Also occurs in disconnected-client scenarios, after manual `nomad node eligibility`/purge operations, or when state was partially restored.","solutions":["Check `nomad node status <node-id>`; if the node is down, bring the Nomad client back online so it re-registers (node IDs are stable via the client's data dir).","If the node is permanently gone, stop/purge the allocations on it (`nomad node drain` then `nomad job stop`) so stale claims are released.","Reschedule the workload onto a live node to generate a fresh claim referencing a registered node.","If the node was erroneously GC'd, check server gc thresholds (node_gc_threshold) and client heartbeat TTL settings."],"exampleFix":"// before: retrying claim against a deregistered node's alloc\nclient.CSIVolumes().Claim(volID, namespace, writeOpts, claimReq)\n// after: verify the node is registered first\nnode, _, _ := client.Nodes().Info(alloc.NodeID, nil)\nif node != nil && node.Status == \"ready\" {\n    client.CSIVolumes().Claim(volID, namespace, writeOpts, claimReq)\n}","handlingStrategy":"validation","validationCode":"node, _, err := client.Nodes().Info(alloc.NodeID, nil)\nif err != nil || node == nil {\n    return fmt.Errorf(\"node %s is not registered; node may have been GC'd\", alloc.NodeID)\n}\nif node.Status != \"ready\" {\n    return fmt.Errorf(\"node %s is %s; not eligible for volume claims\", node.ID, node.Status)\n}","typeGuard":null,"tryCatchPattern":"if err != nil && strings.Contains(err.Error(), structs.ErrUnknownNode.Error()) {\n    // node deregistered: drain/re-reschedule the workload\n}","preventionTips":["Keep clients connected so heartbeats don't expire and nodes don't get GC'd.","Drain and stop jobs on decommissioned nodes before removing them.","Monitor `nomad node status` for `down` nodes holding volume claims."],"tags":["nomad","csi","node","heartbeat","garbage-collection"],"backgroundTag":"unknown-node-id","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}