{"record":{"id":"3fefab177d729e99","repo":"rustfs/rustfs","slug":"remote-tier-cannot-discover-the-outcome-of-a-probe","errorCode":null,"errorMessage":"remote tier cannot discover the outcome of a probe PUT","messagePattern":"remote tier cannot discover the outcome of a probe PUT","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"crates/ecstore/src/services/tier/warm_backend.rs","lineNumber":560,"sourceCode":"async fn remove_discovered_probe_candidate(\n    w: &WarmBackendImpl,\n    probe_object: &str,\n    candidate: TransitionCandidateProbe,\n) -> Result<bool, std::io::Error> {\n    match candidate {\n        TransitionCandidateProbe::Missing => Ok(false),\n        TransitionCandidateProbe::VersionedPresent(remote_version_id) => {\n            w.remove_exact(probe_object, &remote_version_id).await?;\n            Ok(true)\n        }\n        TransitionCandidateProbe::UnversionedPresent => {\n            w.remove(probe_object, \"\").await?;\n            Ok(true)\n        }\n        TransitionCandidateProbe::Ambiguous => {\n            Err(std::io::Error::other(\"remote tier probe PUT produced multiple possible versions\"))\n        }\n        TransitionCandidateProbe::Unsupported => Err(std::io::Error::new(\n            std::io::ErrorKind::Unsupported,\n            \"remote tier cannot discover the outcome of a probe PUT\",\n        )),\n    }\n}\n\nasync fn compensate_uncertain_probe_put(\n    w: &WarmBackendImpl,\n    probe_object: &str,\n    settle_deadline: tokio::time::Instant,\n) -> Result<(), std::io::Error> {\n    let final_deadline = settle_deadline + WARM_BACKEND_PROBE_FINAL_RECONCILE_TIMEOUT;\n    let mut removed_any = false;\n    while tokio::time::Instant::now() < settle_deadline {\n        let candidate = match tokio::time::timeout_at(settle_deadline, w.probe_transition_candidate(probe_object)).await {\n            Ok(candidate) => candidate?,\n            Err(_) => break,\n        };","sourceCodeStart":542,"sourceCodeEnd":578,"githubUrl":"https://github.com/rustfs/rustfs/blob/5dca076efed96e7b842de07c4c2111035ae7c7a2/crates/ecstore/src/services/tier/warm_backend.rs#L542-L578","documentation":"When a probe PUT's candidate listing on the remote tier yields Unsupported, remove_discovered_probe_candidate cannot discover which remote object version the probe created and therefore cannot clean it up. The backend signals that it cannot determine the outcome of the probe PUT, so the warm-backend check fails rather than guessing.","triggerScenarios":"Calling check_warm_backend_with_deadlines (which compensates uncertain probe PUTs) against a remote backend whose candidate listing/probe API reports TransitionCandidateProbe::Unsupported — e.g. a warm backend that cannot enumerate object versions for the probe object.","commonSituations":"Remote tier implementations lacking version-aware listing (unversioned buckets or backends without generation support), misconfigured tier target, or a backend type that does not implement candidate discovery.","solutions":["Use a versioned remote tier backend that supports candidate discovery (S3/GCS with versioning or generation listing enabled).","Check the tier target configuration and re-create the tier pointing at a supported backend.","If Unsupported is expected for this backend, avoid probe-based warm-backend checks for it and use a simpler reachability test."],"exampleFix":null,"handlingStrategy":"fallback","validationCode":"let supports_probe = matches!(backend_kind, BackendKind::S3 | BackendKind::GCS);\nif !supports_probe { use basic warm-backend reachability check; }","typeGuard":null,"tryCatchPattern":"match check_warm_backend_with_deadlines(&mut backend).await {\n    Err(e) if e.kind() == std::io::ErrorKind::Unsupported => {\n        // backend cannot support probe PUT discovery; fall back to reachability-only check\n    }\n    other => other?,\n}","preventionTips":["Configure tier targets only on versioned remote buckets (S3 versioning or GCS generations).","Document which warm-backend implementations support probe-based checks.","Gate probe checks behind a backend capability flag instead of assuming support."],"tags":["tiering","warm-backend","probe","unsupported"],"backgroundTag":"unsupported-operation","analyzedSha":"5dca076efed96e7b842de07c4c2111035ae7c7a2","analyzedAt":"2026-09-06T05:54:05.891Z","contentChangedAt":"2026-09-06T05:54:05.891Z","schemaVersion":2},"datasetVersion":"2026-09-14T00:17:10.932Z"}