{"record":{"id":"5c7a06bc1e0c90b5","repo":"neondatabase/neon","slug":"migration-to-node-rejected-may-require-force","errorCode":null,"errorMessage":"Migration to {node} rejected, may require `--force` ({}) ","messagePattern":"Migration to (.+?) rejected, may require `--force` \\((.+?)\\) ","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"control_plane/storcon_cli/src/main.rs","lineNumber":731,"sourceCode":"                ..Default::default()\n            };\n\n            let req = TenantShardMigrateRequest {\n                node_id: node,\n                origin_node_id: None,\n                migration_config,\n            };\n\n            match storcon_client\n                .dispatch::<TenantShardMigrateRequest, TenantShardMigrateResponse>(\n                    Method::PUT,\n                    format!(\"control/v1/tenant/{tenant_shard_id}/migrate\"),\n                    Some(req),\n                )\n                .await\n            {\n                Err(mgmt_api::Error::ApiError(StatusCode::PRECONDITION_FAILED, msg)) => {\n                    anyhow::bail!(\n                        \"Migration to {node} rejected, may require `--force` ({}) \",\n                        msg\n                    );\n                }\n                Err(e) => return Err(e.into()),\n                Ok(_) => {}\n            }\n\n            watch_tenant_shard(storcon_client, tenant_shard_id, Some(node)).await?;\n        }\n        Command::TenantShardWatch { tenant_shard_id } => {\n            watch_tenant_shard(storcon_client, tenant_shard_id, None).await?;\n        }\n        Command::TenantShardMigrateSecondary {\n            tenant_shard_id,\n            node,\n        } => {\n            let req = TenantShardMigrateRequest {","sourceCodeStart":713,"sourceCodeEnd":749,"githubUrl":"https://github.com/neondatabase/neon/blob/8f60b04da47ffefe0e52bda2440134b42874eb75/control_plane/storcon_cli/src/main.rs#L713-L749","documentation":"storcon_cli sent PUT control/v1/tenant/{tenant_shard_id}/migrate to the storage controller and got an HTTP 412 Precondition Failed ApiError back. The controller refuses the migration because the requested placement violates its preconditions (for example a node that is not a valid destination for this shard). The CLI rewraps the server message and hints at --force, which sets a flag in TenantShardMigrateRequest that bypasses those checks.","triggerScenarios":"Calling storcon_cli tenant-shard-migrate without --force where the destination violates controller preconditions: migrating a shard to a node in the wrong role (e.g. the node is already the attached pageserver), a node excluded by the tenant's preferred AZ, an unavailable node, or otherwise rejected placement. The 412 response is matched explicitly in the dispatch error handler.","commonSituations":" scripting migrations without first describing the tenant's current attachment; forcing shards into an AZ that conflicts with the tenant's preferred AZ policy; node availability or scheduling changed between planning and executing the migration; operator intent really is to override the scheduler, so --force is the correct escape hatch.","solutions":["Add --force to the tenant-shard-migrate command if you intentionally want to override the controller's placement checks","Describe the tenant (control/v1/tenant/{tenant_id}) and check which node the shard is currently attached to and its secondaries, then request a placement that is actually a change","Verify the target node id exists, is Active, and does not conflict with the tenant's preferred AZ before retrying without --force"],"exampleFix":"# before\nstorcon_cli tenant-shard-migrate --tenant-shard-id <id> --node <node_id>\n# after (when overriding is intended)\nstorcon_cli tenant-shard-migrate --tenant-shard-id <id> --node <node_id> --force","handlingStrategy":"try-catch","validationCode":"// Describe the tenant and inspect current placement before migrating\nlet desc: TenantDescribeResponse = client\n    .dispatch(Method::GET, format!(\"control/v1/tenant/{tenant_id}\"), None)\n    .await?;\nlet shard = desc.shards.iter().find(|s| s.tenant_shard_id == *tenant_shard_id)\n    .ok_or_else(|| anyhow::anyhow!(\"shard not in tenant\"))?;\nlet target_change = shard.node_attached != Some(target_node)\n    && !shard.node_secondary.contains(&target_node);\nif !target_change && !force {\n    anyhow::bail!(\"migration would be a no-op or violate preconditions; pass force\");\n}","typeGuard":null,"tryCatchPattern":"match storcon_client.dispatch::<TenantShardMigrateRequest, TenantShardMigrateResponse>(\n    Method::PUT,\n    format!(\"control/v1/tenant/{tenant_shard_id}/migrate\"),\n    Some(req),\n).await {\n    Err(mgmt_api::Error::ApiError(StatusCode::PRECONDITION_FAILED, msg)) => {\n        // decide: retry without change, or re-dispatch with force=true\n        if cli.force {\n            let forced = TenantShardMigrateRequest { migration_config: migration_config.force(), ..req };\n            storcon_client.dispatch(Method::PUT, url, Some(forced)).await?;\n        } else {\n            return Err(anyhow::anyhow!(\"migration rejected by controller: {msg}\"));\n        }\n    }\n    Err(e) => return Err(e.into()),\n    Ok(resp) => resp,\n}","preventionTips":["Always describe the tenant and compare attachment/secondaries against the target before migrating","Make --force an explicit, logged decision in automation instead of a default","Check the target node's availability and the tenant's preferred AZ before issuing the request"],"tags":["storcon","migration","http-412","precondition-failed","cli"],"backgroundTag":"precondition-failed","analyzedSha":"8f60b04da47ffefe0e52bda2440134b42874eb75","analyzedAt":"2026-08-16T23:39:28.135Z","schemaVersion":2},"datasetVersion":"2026-08-17T04:17:16.089Z"}