{"record":{"id":"6b6abc18dad40c57","repo":"nautechsystems/nautilus_trader","slug":"execution-payload-storage-is-in-maintenance","errorCode":null,"errorMessage":"Execution payload storage is in {} maintenance","messagePattern":"Execution payload storage is in (.+?) maintenance","errorType":"validation","errorClass":"anyhow::Error","httpStatus":null,"severity":"error","filePath":"crates/adapters/blockchain/src/cache/database.rs","lineNumber":4715,"sourceCode":"        let version = marker.ok_or_else(|| {\n            anyhow::anyhow!(\"Postgres execution requires protected payload storage\")\n        })?;\n        anyhow::ensure!(\n            version == EXECUTION_PAYLOAD_PROTOCOL_VERSION,\n            \"Unsupported execution payload protection version {version}\"\n        );\n        let row = sqlx::query(\n            \"SELECT deployment_id, protocol_version, operation, active_key_id \\\n             FROM execution_payload_state WHERE component = 'signed_transactions' \\\n             FOR SHARE\",\n        )\n        .fetch_optional(&mut *transaction)\n        .await\n        .context(\"failed to lock execution payload state\")?\n        .ok_or_else(|| anyhow::anyhow!(\"Execution payload marker exists without state\"))?;\n        let state = execution_payload_state_from_row(&row)?;\n        validate_execution_payload_state(&state, keys)?;\n        anyhow::ensure!(\n            state.operation == \"ready\",\n            \"Execution payload storage is in {} maintenance\",\n            state.operation\n        );\n\n        Ok(ExecutionPayloadLease {\n            _transaction: transaction,\n        })\n    }\n\n    /// Reserves one nonce use under the active payload key.\n    pub(crate) async fn reserve_execution_payload_seal(\n        &self,\n        keys: &PayloadKeySet,\n    ) -> anyhow::Result<()> {\n        let mut transaction = self\n            .pool\n            .begin()","sourceCodeStart":4697,"sourceCodeEnd":4733,"githubUrl":"https://github.com/nautechsystems/nautilus_trader/blob/18893faf8b356be3320add8de2f861b0b647cf06/crates/adapters/blockchain/src/cache/database.rs#L4697-L4733","documentation":"Execution payload storage has a lifecycle operation column; leases are only granted when operation == 'ready'. While any other maintenance operation (e.g. activate, migrate, seal, rotate) is recorded, acquire_execution_payload_lease refuses to proceed so readers/writers never observe a half-migrated store.","triggerScenarios":"Calling acquire_execution_payload_lease while execution_payload_state.operation for 'signed_transactions' is anything other than 'ready' — typically during an in-flight migration, key rotation, or activation initiated elsewhere.","commonSituations":"A second node/process attempts signing while a migration is running; a previous migration crashed leaving operation stuck at a non-ready value; key rotation scheduled concurrently with trading.","solutions":["Wait for the in-flight maintenance operation to complete, then retry the lease acquisition","If a migration is stuck (crashed mid-run), resume or finalize it via migrate_execution_payload_batch until operation returns to 'ready'","Inspect SELECT operation FROM execution_payload_state WHERE component='signed_transactions' to identify the current operation","Coordinate maintenance windows so signing/broadcast does not run concurrently with migrations"],"exampleFix":"// before: leasing during migration\nlet lease = db.acquire_execution_payload_lease(&keys).await?;\n// after: check operation first\nif db.execution_payload_state().await?.map(|s| s.operation) != Some(\"ready\".into()) {\n    // defer signing until maintenance completes\n}","handlingStrategy":"retry","validationCode":"let op: Option<String> = sqlx::query_scalar(\n    \"SELECT operation FROM execution_payload_state WHERE component = 'signed_transactions'\",\n).fetch_optional(&pool).await?;\nlet ready = op.as_deref() == Some(\"ready\");","typeGuard":null,"tryCatchPattern":"loop {\n    match db.acquire_execution_payload_lease(&keys).await {\n        Ok(lease) => break lease,\n        Err(e) if e.to_string().contains(\"maintenance\") => {\n            tokio::time::sleep(Duration::from_secs(5)).await; // wait out maintenance\n        }\n        Err(e) => return Err(e),\n    }\n}","preventionTips":["Schedule migrations/rotations outside trading windows","Check storage operation status before starting signing workers","Always finalize migrations (drive batches until they report completion)","Alert when operation stays non-ready beyond an expected duration"],"tags":["database","maintenance","state"],"backgroundTag":"invalid-state-transition","analyzedSha":"18893faf8b356be3320add8de2f861b0b647cf06","analyzedAt":"2026-09-08T20:49:34.690Z","contentChangedAt":"2026-09-08T20:49:34.690Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}