{"record":{"id":"a59d0244bb922a11","repo":"hashicorp/nomad","slug":"node-not-found-a59d02","errorCode":null,"errorMessage":"node not found","messagePattern":"node not found","errorType":"error_code","errorClass":null,"httpStatus":null,"severity":"error","filePath":"nomad/state/state_store.go","lineNumber":1131,"sourceCode":"\ttxn := s.db.WriteTxnMsgT(msgType, index)\n\tdefer txn.Abort()\n\n\tif err := s.updateNodeStatusTxn(txn, req); err != nil {\n\t\treturn err\n\t}\n\n\treturn txn.Commit()\n}\n\nfunc (s *StateStore) updateNodeStatusTxn(txn *txn, req *structs.NodeUpdateStatusRequest) error {\n\n\t// Lookup the node\n\texisting, err := txn.First(TableNodes, indexID, req.NodeID)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"node lookup failed: %v\", err)\n\t}\n\tif existing == nil {\n\t\treturn fmt.Errorf(\"node not found\")\n\t}\n\n\t// Copy the existing node\n\texistingNode := existing.(*structs.Node)\n\tcopyNode := existingNode.Copy()\n\tcopyNode.StatusUpdatedAt = req.UpdatedAt\n\n\t// If the request has a signing key ID, we should update the node reference\n\t// to this. We need to check for the empty string, as a new identity won't\n\t// always be generated, and we don't want to overwrite the exiting entry\n\t// with an empty string.\n\tif req.IdentitySigningKeyID != \"\" {\n\t\tcopyNode.IdentitySigningKeyID = req.IdentitySigningKeyID\n\t}\n\n\t// Add the event if given\n\tif req.NodeEvent != nil {\n\t\tappendNodeEvents(txn.Index, copyNode, []*structs.NodeEvent{req.NodeEvent})","sourceCodeStart":1113,"sourceCodeEnd":1149,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/nomad/state/state_store.go#L1113-L1149","documentation":"updateNodeStatusTxn returns 'node not found' when UpdateNodeStatus is called for a node ID with no row in the 'nodes' table. The status update is rejected because the store requires the existing node record to copy and mutate. Surfaced through StateStore.UpdateNodeStatus.","triggerScenarios":"Calling UpdateNodeStatus with req.NodeID that is unregistered or already deregistered/garbage-collected; heartbeats arriving from a node whose registration was removed; a stale node ID in a restore snapshot.","commonSituations":"Client heartbeats racing with operator deregistration; node was purged between event emit and status write; wrong/typo'd node UUID in a tooling script.","solutions":["Confirm the node is registered (NodeByID) before updating its status.","Ignore/handle this as a benign race when the heartbeat races deregistration.","Re-register the node (UpsertNode) if it should exist, then retry the status update.","Validate the node UUID for typos in scripts or configs."],"exampleFix":"// before\nerr := store.UpdateNodeStatus(idx, nodeID, \"ready\", now)\n// after\nnode, _ := store.NodeByID(nil, nodeID)\nif node != nil {\n    err := store.UpdateNodeStatus(idx, nodeID, \"ready\", now)\n}","handlingStrategy":"try-catch","validationCode":"node, err := store.NodeByID(nil, req.NodeID)\nif err != nil {\n    return err\n}\nif node == nil {\n    return fmt.Errorf(\"cannot update status: node %s not registered\", req.NodeID)\n}","typeGuard":"func nodeExists(s *state.StateStore, id string) bool {\n    n, err := s.NodeByID(nil, id)\n    return err == nil && n != nil\n}","tryCatchPattern":"if err := store.UpdateNodeStatus(idx, nodeID, status, now); err != nil {\n    if err.Error() == \"node not found\" {\n        logger.Warn(\"status update skipped; node gone\", \"node\", nodeID)\n        return nil\n    }\n    return err\n}","preventionTips":["Ignore heartbeat failures for nodes that were concurrently deregistered.","Re-register the node if it should exist before updating status.","Validate node UUIDs in tooling to catch typos.","Log and monitor not-found heartbeats as potential GC/registration races."],"tags":["nomad","state-store","node-status","missing-entity"],"backgroundTag":"resource-not-found","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}