hashicorp/nomad · error

node pool %q has nodes in regions: %v

Error message

node pool %q has nodes in regions: %v

What it means

DeleteNodePools also blocks deletion while the node pool still contains nodes. nodePoolRegionsInUse detects nodes assigned to the pool and the endpoint returns this error naming the regions where those nodes live, as part of a multierror.

Source

Thrown at nomad/node_pool_endpoint.go:284

	}
	if slices.Contains(args.Names, "") {
		return structs.NewErrRPCCodedf(http.StatusBadRequest, "node pool name is empty")
	}

	// Verify that the node pools we're deleting do not have nodes or
	// non-terminal jobs in this region or in any federated region.
	var mErr multierror.Error
	for _, name := range args.Names {
		regionsWithNonTerminal, regionsWithNodes, err := n.nodePoolRegionsInUse(args.AuthToken, name)
		if err != nil {
			_ = multierror.Append(&mErr, err)
		}
		if len(regionsWithNonTerminal) != 0 {
			_ = multierror.Append(&mErr, fmt.Errorf(
				"node pool %q has non-terminal jobs in regions: %v", name, regionsWithNonTerminal))
		}
		if len(regionsWithNodes) != 0 {
			_ = multierror.Append(&mErr, fmt.Errorf(
				"node pool %q has nodes in regions: %v", name, regionsWithNodes))
		}
	}

	if err := mErr.ErrorOrNil(); err != nil {
		return err
	}

	// Delete via Raft.
	_, index, err := n.srv.raftApply(structs.NodePoolDeleteRequestType, args)
	if err != nil {
		return err
	}

	reply.Index = index
	return nil
}

View on GitHub (pinned to 482b49bf1a)

Solutions

  1. Drain and stop the Nomad agents on the nodes (`nomad node drain <id>`, then stop the agent) so they deregister
  2. Force-deregister stale nodes: `nomad node status` then `nomad node eligibility -disable`/agent shutdown, or purge via the API
  3. Re-attempt the pool deletion once `nomad node status -filter 'NodePool=="<pool>"'` returns no nodes
  4. Use force-delete only after confirming nodes are truly gone

Example fix

// before
$ nomad node pool delete gpu
Error: node pool "gpu" has nodes in regions: [us-west]
// after
$ nomad node drain -disable -detach <node-id> && systemctl stop nomad
$ nomad node pool delete gpu
Defensive patterns

Strategy: validation

Validate before calling

nodes, _ := client.Nodes().List(nil)
for _, n := range nodes {
    if n.NodePool == poolName {
        return fmt.Errorf("node %s still in pool %s", n.ID, poolName)
    }
}

Try / catch

err := client.NodePools().Delete([]string{poolName}, wr)
if err != nil && strings.Contains(err.Error(), "has nodes in regions") {
    // drain/deregister nodes, then retry
}

Prevention

When it happens

Trigger: Deleting a node pool that has registered (non-archived) nodes in any region — e.g. `nomad node pool delete gpu` while agents still report in from that pool.

Common situations: Decommissioning a pool of clients without draining/deregistering them first; stale node registrations from hosts that were wiped but whose agents are still registered.

Related errors


AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/5ba0af407a54fb08. Report an issue: GitHub.