neondatabase/neon · error

There are no nodes to migrate to

Error message

There are no nodes to migrate to

What it means

After partitioning nodes into drain and fill sets, storcon_cli retains only fill nodes whose availability is Active and whose scheduling policy is Active or Filling — i.e. nodes the scheduler may place shards onto. If that filtered set is empty, there is nowhere to migrate shards to and the command bails before setting any node to Draining.

Source

Thrown at control_plane/storcon_cli/src/main.rs:1095

                } else {
                    node_to_fill_descs.push(desc);
                }
            }

            if nodes.len() != node_to_drain_descs.len() {
                anyhow::bail!("Bulk migration requested away from node which doesn't exist.")
            }

            node_to_fill_descs.retain(|desc| {
                matches!(desc.availability, NodeAvailabilityWrapper::Active)
                    && matches!(
                        desc.scheduling,
                        NodeSchedulingPolicy::Active | NodeSchedulingPolicy::Filling
                    )
            });

            if node_to_fill_descs.is_empty() {
                anyhow::bail!("There are no nodes to migrate to")
            }

            // Set the node scheduling policy to draining for the nodes which
            // we plan to drain.
            for node_desc in node_to_drain_descs.iter() {
                let req = NodeConfigureRequest {
                    node_id: node_desc.id,
                    availability: None,
                    scheduling: Some(NodeSchedulingPolicy::Draining),
                };

                storcon_client
                    .dispatch::<_, ()>(
                        Method::PUT,
                        format!("control/v1/node/{}/config", node_desc.id),
                        Some(req),
                    )
                    .await?;

View on GitHub (pinned to 8f60b04da4)

Solutions

1. List nodes and their availability/scheduling; set at least one non-drain node to availability active and scheduling active (or filling)
2. Unpause/unstop a candidate fill node (its scheduling must be Active or Filling) and retry
3. Add or restore a healthy node before draining, or reduce the drain list so a fill node remains

Example fix

# before: all other nodes are paused
storcon_cli tenant migrate-bulk --drain-nodes 1,2
# after: make node 3 fillable first
storcon_cli node configure --node-id 3 --scheduling active
storcon_cli tenant migrate-bulk --drain-nodes 1,2

// Reproduce the CLI's fill-node filter client-side and fail early with context
let fill_nodes: Vec<_> = nodes_list.into_iter()
    .filter(|n| !drain_set.contains(&n.id))
    .filter(|n| matches!(n.availability, NodeAvailabilityWrapper::Active)
        && matches!(n.scheduling, NodeSchedulingPolicy::Active | NodeSchedulingPolicy::Filling))
    .collect();
anyhow::ensure!(!fill_nodes.is_empty(),
    "no fillable nodes: set a node to availability=active, scheduling=active before draining");

fn is_fill_candidate(desc: &NodeDescribeResponse) -> bool {
    matches!(desc.availability, NodeAvailabilityWrapper::Active)
        && matches!(desc.scheduling, NodeSchedulingPolicy::Active | NodeSchedulingPolicy::Filling)
}

When it happens

Trigger: Running bulk migration when every non-drained node is either Offline, or has scheduling policy Pause/Stop/Essential/Draining (not Active or Filling). Can also happen when all existing nodes are in the drain set, leaving zero fill candidates.

Common situations: Draining nodes in a small cluster where the remaining nodes were paused for maintenance; node availability stuck Offline after an outage; scheduling policies left in Pause/Stop from earlier operations; attempting to drain every node in the cluster.

Related errors

• Migration to {node} rejected, may require `--force` ({})
• Bulk migration requested away from node which doesn't exist.
• Unknown scheduling policy '{s}', try active,essential,pause,
• Unknown availability state '{s}'
• AZ {} not found on any node: known AZs are: {:?}

AI-assisted analysis of neondatabase/neon@8f60b04da4 (2026-08-16). Data as JSON: /api/errors/911abb94977cdab9. Report an issue: GitHub.