{"record":{"id":"d171f66d2a3028e2","repo":"rustfs/rustfs","slug":"interrupted","errorCode":"Interrupted","errorMessage":"tier free-version recovery cancelled","messagePattern":"tier free-version recovery cancelled","errorType":"exception","errorClass":"std::io::Error","httpStatus":null,"severity":"info","filePath":"crates/ecstore/src/bucket/lifecycle/tier_free_version_recovery.rs","lineNumber":216,"sourceCode":"    let mut retry_cursor = RetryCursor::new(bucket_marker, object_marker);\n    for oi in page.items {\n        if cancel_token.is_cancelled() {\n            return Err(tier_free_version_recovery_cancelled());\n        }\n        retry_cursor.visit(&oi);\n        if !record_recovered_free_version_enqueue(&mut stats, enqueue_recovered_free_version(&api, oi).await) {\n            let (bucket_marker, object_marker) = retry_cursor.retry_markers();\n            stats.truncated = true;\n            stats.next_bucket_marker = bucket_marker;\n            stats.next_object_marker = object_marker;\n            break;\n        }\n    }\n\n    Ok(stats)\n}\n\nfn tier_free_version_recovery_cancelled() -> crate::error::Error {\n    std::io::Error::new(std::io::ErrorKind::Interrupted, \"tier free-version recovery cancelled\").into()\n}\n\nfn tier_free_version_recovery_walk_shutdown_timed_out() -> crate::error::Error {\n    std::io::Error::new(\n        std::io::ErrorKind::TimedOut,\n        \"tier free-version recovery walk did not stop after cancellation\",\n    )\n    .into()\n}\n\nfn record_recovered_free_version_enqueue(stats: &mut FreeVersionRecoveryStats, queued: bool) -> bool {\n    stats.scanned += 1;\n    if queued {\n        stats.enqueued += 1;\n        true\n    } else {\n        stats.failed += 1;","sourceCodeStart":198,"sourceCodeEnd":234,"githubUrl":"https://github.com/rustfs/rustfs/blob/5dca076efed96e7b842de07c4c2111035ae7c7a2/crates/ecstore/src/bucket/lifecycle/tier_free_version_recovery.rs#L198-L234","documentation":"Free-version recovery walks tier free-version records with a cancellation token; when the token fires (node shutdown, task cancellation, operator abort) mid-walk, the scan stops and returns this Interrupted error instead of partial stats. Stats accumulated so far are discarded in favor of an explicit signal, and the retry markers (retry_cursor / next_bucket_marker / next_object_marker) exist so the walk can resume where it stopped. This is an expected, benign shutdown signal — not a fault.","triggerScenarios":"recover/free-version scan is running when cancel_token.cancelled() resolves (select! at line 310), the loop checks is_cancelled() between items (line 189/319), or the walker channel closes due to cancellation (line 422/438). Any graceful shutdown or cancel during an active recovery produces it.","commonSituations":"Rolling restarts while the lifecycle free-version scanner is mid-bucket; cancellation triggered by reconfiguration of lifecycle/tier settings; CI timeouts cancelling the recovery task.","solutions":["Nothing to repair — restart or re-invoke recovery and pass the previous next_bucket_marker/next_object_marker to resume","If shutdowns repeatedly interrupt long walks, raise the per-run limit or schedule recovery during quiet windows","Distinguish from TimedOut sibling error: this one means cancellation worked correctly"],"exampleFix":null,"handlingStrategy":"retry","validationCode":"// Pass through the cancellation token and check it before starting\nif cancel_token.is_cancelled() {\n    return Ok(default_stats()); // skip the run instead of eating an Interrupted\n}\nlet stats = recover_tier_free_versions(api, limit, marker, &cancel_token).await?;","typeGuard":"fn is_cancellation(e: &crate::error::Error) -> bool {\n    matches!(e.io_error_kind(), Some(std::io::ErrorKind::Interrupted))\n        && e.to_string().contains(\"recovery cancelled\")\n}","tryCatchPattern":"match recover_tier_free_versions(api, limit, marker, &cancel).await {\n    Err(e) if is_cancellation(&e) => {\n        // benign shutdown: persist next_bucket_marker/next_object_marker and resume later\n        Ok(partial)\n    }\n    other => other,\n}","preventionTips":["Persist the retry markers so interrupted walks resume instead of restarting","Do not alert on Interrupted during planned restarts; alert on the TimedOut sibling instead","Size the per-run limit so walks finish within maintenance windows"],"tags":["tiering","lifecycle","recovery","cancellation","shutdown"],"backgroundTag":"operation-cancelled","analyzedSha":"5dca076efed96e7b842de07c4c2111035ae7c7a2","analyzedAt":"2026-08-20T21:57:04.799Z","contentChangedAt":"2026-08-20T21:57:04.799Z","schemaVersion":2},"datasetVersion":"2026-09-14T00:17:10.932Z"}