{"record":{"id":"f69759bdec72358b","repo":"hashicorp/nomad","slug":"heap-doesn-t-contain-object-with-id-q-s","errorCode":null,"errorMessage":"heap doesn't contain object with ID %q (%s)","messagePattern":"heap doesn't contain object with ID %q \\((.+?)\\)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"lib/delayheap/delay_heap.go","lineNumber":150,"sourceCode":"\t}\n\t_, ok := p.index[tuple]\n\treturn ok\n}\n\nfunc (p *DelayHeap) Update(heapNode HeapNode, waitUntil time.Time) error {\n\ttuple := structs.NamespacedID{\n\t\tID:        heapNode.ID(),\n\t\tNamespace: heapNode.Namespace(),\n\t}\n\tif existingHeapNode, ok := p.index[tuple]; ok {\n\t\t// Need to update the job as well because its spec can change.\n\t\texistingHeapNode.Node = heapNode\n\t\texistingHeapNode.WaitUntil = waitUntil\n\t\theap.Fix(&p.heap, existingHeapNode.index)\n\t\treturn nil\n\t}\n\n\treturn fmt.Errorf(\"heap doesn't contain object with ID %q (%s)\", heapNode.ID(), heapNode.Namespace())\n}\n\nfunc (p *DelayHeap) Remove(heapNode HeapNode) error {\n\ttuple := structs.NamespacedID{\n\t\tID:        heapNode.ID(),\n\t\tNamespace: heapNode.Namespace(),\n\t}\n\tif node, ok := p.index[tuple]; ok {\n\t\theap.Remove(&p.heap, node.index)\n\t\tdelete(p.index, tuple)\n\t\treturn nil\n\t}\n\n\treturn fmt.Errorf(\"heap doesn't contain object with ID %q (%s)\", heapNode.ID(), heapNode.Namespace())\n}\n\nfunc (p *DelayHeap) Length() int {\n\treturn len(p.heap)","sourceCodeStart":132,"sourceCodeEnd":168,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/lib/delayheap/delay_heap.go#L132-L168","documentation":"DelayHeap.Update mutates an existing node's payload and WaitUntil time, but only if the node's NamespacedID is present in the index. If the tuple (ID, namespace) is absent, it returns this error instead of silently creating or missing an entry.","triggerScenarios":"Update is called with a HeapNode that was never Pushed, or was already removed via Remove/Pop — e.g. updating an evaluation whose timer already fired or was cancelled before the update arrives.","commonSituations":"Race between a node's deadline firing (Pop) and a late Update from the scheduler; calling Update after Remove during reconciliation; stale references to nodes across heap rebuilds/restarts; test code constructing nodes without pushing first.","solutions":["Verify the node is still in the heap (Push succeeded and Remove/Pop hasn't run) before calling Update","Treat this error as benign when it results from the timer already firing; remove instead of update in that path","If the node may not exist, check containment first (or add a Get/Lookup helper on DelayHeap) and choose Push vs Update accordingly","Check for out-of-order processing between deletion events and update events for the same ID"],"exampleFix":"// before\nif err := heap.Update(node, when); err != nil {\n\treturn err\n}\n// after\nif err := heap.Update(node, when); err != nil {\n\t// node may have already fired/was removed; enqueue fresh\n\treturn heap.Push(node, when)\n}","handlingStrategy":"try-catch","validationCode":"// Ensure the node was pushed and not yet removed before updating\nif !nodePushed[node.ID()] || nodeRemoved[node.ID()] {\n\treturn fmt.Errorf(\"cannot update node %q: not in heap\", node.ID())\n}","typeGuard":null,"tryCatchPattern":"if err := delayHeap.Update(node, when); err != nil {\n\tif strings.Contains(err.Error(), \"heap doesn't contain object\") {\n\t\t// node already fired or was removed; push it fresh instead\n\t\treturn delayHeap.Push(node, when)\n\t}\n\treturn err\n}","preventionTips":["Serialize Update/Remove/Pop operations per node ID to avoid firing-vs-update races","Drop stale update events for nodes whose timers already fired","Re-check heap membership after any Remove/Pop before issuing an Update","In tests, always Push nodes before calling Update"],"tags":["heap","missing-node","scheduler","race-condition"],"backgroundTag":"node-not-found-in-delay-heap","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}