hashicorp/nomad · error

node doesn't have a drain strategy set

Error message

node doesn't have a drain strategy set

What it means

drainingNode.IsDone checks whether a node's drain has completed, but the node must have a DrainStrategy set to make that determination. Nomad returns this error (marked 'should never happen') when the cached node has lost its drain strategy, meaning the drain bookkeeping state is inconsistent.

Source

Thrown at nomad/drainer/draining_node.go:62

	// Should never happen
	if n.node == nil || n.node.DrainStrategy == nil {
		return false, time.Time{}
	}

	return n.node.DrainStrategy.DeadlineTime()
}

// IsDone returns if the node is done draining batch and service allocs. System
// allocs must be stopped before marking drain complete unless they're being
// ignored.
func (n *drainingNode) IsDone() (bool, error) {
	n.l.RLock()
	defer n.l.RUnlock()

	// Should never happen
	if n.node == nil || n.node.DrainStrategy == nil {
		return false, fmt.Errorf("node doesn't have a drain strategy set")
	}

	// Retrieve the allocs on the node
	allocs, err := n.state.AllocsByNode(nil, n.node.ID)
	if err != nil {
		return false, err
	}

	for _, alloc := range allocs {
		// System and plugin jobs are only stopped after a node is
		// done draining everything else, so ignore them here.
		if alloc.Job.Type == structs.JobTypeSystem || alloc.Job.IsPlugin() {
			continue
		}

		// If there is a non-terminal we aren't done
		if !alloc.ClientTerminalStatus() {
			return false, nil

View on GitHub (pinned to 482b49bf1a)

Solutions

1. Verify the node's drain state with `nomad node status <id>` and `nomad node drain -status <id>`
2. If the drain was intentionally stopped, this error is transient — the drainer will drop the node on next refresh
3. If a drain is desired, re-issue `nomad node drain -enable <id>`
4. Restart the leader to rebuild drainer state if it persists

Example fix

// before
drain, _ := node.DrainStrategy == nil, node
// guard in tooling before asserting drain done
if node.DrainStrategy != nil {
    done, err := drainTracker.IsDone(node)
}

node, _ := state.NodeByID(nil, nodeID)
if node == nil || node.DrainStrategy == nil {
    // not draining; skip IsDone check
}

func isDraining(n *structs.Node) bool { return n != nil && n.DrainStrategy != nil }

try {
    done, err := dn.IsDone()
} catch (e) {
    if (e.message.includes("drain strategy set")) {
        // drain was cancelled/removed; drop node from tracking
        return
    }
    throw e
}

When it happens

Trigger: assertDrainingNode calls IsDone on a drainingNode whose cached n.node is nil or n.node.DrainStrategy is nil — e.g. the drain was removed concurrently while the drainer still held the node object.

Common situations: Drain cancelled (nomad node drain -disable) at nearly the same moment the drainer evaluates completion; leader failover with stale cached node data; bugs in NodeDrain Raft updates clearing DrainStrategy while the node remains in the drainer's set.

Related errors

• drain for task group %q failed: %v
• failed to remove tracked job %q (%s): %v
• can not set node's scheduling eligibility to eligible while
• error parsing: root should be an object
• missing policy name

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/f8cc63b5def4d5fc. Report an issue: GitHub.