{"record":{"id":"15860ce2488c8bbb","repo":"neondatabase/neon","slug":"bulk-migration-requested-away-from-node-which-does","errorCode":null,"errorMessage":"Bulk migration requested away from node which doesn't exist.","messagePattern":"Bulk migration requested away from node which doesn't exist\\.","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"control_plane/storcon_cli/src/main.rs","lineNumber":1083,"sourceCode":"                    \"control/v1/node\".to_string(),\n                    None,\n                )\n                .await?;\n\n            let mut node_to_drain_descs = Vec::new();\n            let mut node_to_fill_descs = Vec::new();\n\n            for desc in node_descs {\n                let to_drain = nodes.contains(&desc.id);\n                if to_drain {\n                    node_to_drain_descs.push(desc);\n                } else {\n                    node_to_fill_descs.push(desc);\n                }\n            }\n\n            if nodes.len() != node_to_drain_descs.len() {\n                anyhow::bail!(\"Bulk migration requested away from node which doesn't exist.\")\n            }\n\n            node_to_fill_descs.retain(|desc| {\n                matches!(desc.availability, NodeAvailabilityWrapper::Active)\n                    && matches!(\n                        desc.scheduling,\n                        NodeSchedulingPolicy::Active | NodeSchedulingPolicy::Filling\n                    )\n            });\n\n            if node_to_fill_descs.is_empty() {\n                anyhow::bail!(\"There are no nodes to migrate to\")\n            }\n\n            // Set the node scheduling policy to draining for the nodes which\n            // we plan to drain.\n            for node_desc in node_to_drain_descs.iter() {\n                let req = NodeConfigureRequest {","sourceCodeStart":1065,"sourceCodeEnd":1101,"githubUrl":"https://github.com/neondatabase/neon/blob/8f60b04da47ffefe0e52bda2440134b42874eb75/control_plane/storcon_cli/src/main.rs#L1065-L1101","documentation":"In the bulk-migration (drain) flow, storcon_cli splits the node list into drain nodes and fill nodes by matching the requested node ids against control-plane node descriptors. If the number of requested ids does not equal the number of matched descriptors, some requested nodes do not exist, and the command bails before changing any scheduling state.","triggerScenarios":"Invoking the bulk migration command with a --drain-nodes-style list (or individual node args collected into `nodes`) where at least one node id is not present in GET control/v1/node results: a typo, a node that was already dropped, or a node id from a different environment.","commonSituations":"Copy-paste of stale node ids after nodes were replaced; scripts built from an old node inventory; typing node ids by hand; confusing node id with pageserver id or hostname.","solutions":["List current nodes (storcon_cli node list / GET control/v1/node) and correct the node ids in the command","Remove ids of nodes that were already dropped or do not exist and re-run","Double-check for typos and for ids from a different storage-controller environment"],"exampleFix":"# before\nstorcon_cli tenant migrate-bulk --drain-nodes 1,2,99\n# after (node 99 does not exist)\nstorcon_cli tenant migrate-bulk --drain-nodes 1,2","handlingStrategy":"validation","validationCode":"// Verify requested drain node ids exist before running bulk migration\nlet nodes_list: Vec<NodeDescribeResponse> = client\n    .dispatch(Method::GET, \"control/v1/node\".to_string(), None).await?;\nlet existing: HashSet<NodeId> = nodes_list.iter().map(|n| n.id).collect();\nlet unknown: Vec<NodeId> = requested_drain_nodes.iter()\n    .filter(|id| !existing.contains(*id)).cloned().collect();\nanyhow::ensure!(unknown.is_empty(), \"unknown node ids: {unknown:?}\");","typeGuard":"fn all_nodes_exist(requested: &[NodeId], existing: &HashSet<NodeId>) -> bool {\n    requested.iter().all(|id| existing.contains(id))\n}","tryCatchPattern":null,"preventionTips":["Refresh the node inventory immediately before issuing drain commands","Fail the whole batch when any id is unknown instead of partially draining","Source node ids from control/v1 node listings, never from memory or other environments"],"tags":["storcon","migration","node-id","validation","cli"],"backgroundTag":"unknown-node-id","analyzedSha":"8f60b04da47ffefe0e52bda2440134b42874eb75","analyzedAt":"2026-08-16T23:39:28.135Z","schemaVersion":2},"datasetVersion":"2026-08-17T04:17:16.089Z"}