{"record":{"id":"038623e20a78fc91","repo":"block/buzz","slug":"git-conformance-probe-failed-e","errorCode":null,"errorMessage":"git conformance probe failed: {e}","messagePattern":"git conformance probe failed: (.+?)","errorType":"exception","errorClass":"anyhow::Error","httpStatus":null,"severity":"critical","filePath":"crates/buzz-relay/src/main.rs","lineNumber":589,"sourceCode":"            .unwrap_or(32);\n        let race_rounds = std::env::var(\"BUZZ_GIT_PROBE_ROUNDS\")\n            .ok()\n            .and_then(|v| v.parse().ok())\n            .unwrap_or(3);\n        let cfg = buzz_relay::api::git::store::ProbeConfig {\n            race_width,\n            race_rounds,\n        };\n        tracing::info!(\n            race_width,\n            race_rounds,\n            \"running git object-store conformance probe (A3 gate)\"\n        );\n        let report = state\n            .git_store\n            .run_conformance_probe(cfg)\n            .await\n            .map_err(|e| anyhow::anyhow!(\"git conformance probe failed: {e}\"))?;\n        tracing::info!(\n            race_width = report.race_width,\n            race_rounds = report.race_rounds,\n            transport_drops = report.transport_drops,\n            \"git object-store backend admitted: A3 conformance probe passed\"\n        );\n    }\n\n    match state.db.verify_channel_roster_fence().await {\n        Ok(()) => {\n            info!(\"Channel roster fence verified\");\n        }\n        Err(error) => {\n            error!(%error, \"Channel roster fence validation failed\");\n            return Err(anyhow::anyhow!(\n                \"Channel roster fence is unsafe; apply or repair migration 0032 before starting this relay: {error}\"\n            ));\n        }","sourceCodeStart":571,"sourceCodeEnd":607,"githubUrl":"https://github.com/block/buzz/blob/ef2aa1ae38fadcc0bc22b8bf6ed96b35933146be/crates/buzz-relay/src/main.rs#L571-L607","documentation":"Fatal startup error: the git object-store conformance probe (A3 gate) failed. Before serving git traffic, the relay runs concurrent conditional-write races against the configured S3/MinIO backend to prove it satisfies the linearizable conditional-write axiom the manifest-pointer protocol depends on. A backend that cannot pass CAS is rejected outright — this is a deployment gate, deliberately fatal. The probe runs by default; BUZZ_GIT_CONFORMANCE_PROBE=false disables it, and BUZZ_GIT_PROBE_WRITERS / BUZZ_GIT_PROBE_ROUNDS scale its width (default 32 writers, 3 rounds).","triggerScenarios":"The configured object store does not implement (or misimplements) conditional writes — e.g. an S3-compatible appliance without If-None-Match support, or eventual-consistency behavior breaking the race; the probe also fails on transport errors reaching the endpoint (down MinIO, wrong credentials) since it performs real writes.","commonSituations":"Deploying git-over-object-storage against a niche S3 clone that lacks conditional-put support; MinIO version too old for conditional writes; S3 endpoint/credential misconfig that only surfaces when writes are attempted; heavy throttling during boot causing probe writers to fail.","solutions":["Read the {e} detail: transport/auth errors mean fix endpoint/credentials first; CAS/race failures mean the backend genuinely lacks linearizable conditional writes","Upgrade to a backend with conditional-write support (recent MinIO, real AWS S3)","If throttling is implicated, lower probe pressure via BUZZ_GIT_PROBE_WRITERS / BUZZ_GIT_PROBE_ROUNDS and retry","Only as a last resort, set BUZZ_GIT_CONFORMANCE_PROBE=false — and only after independently verifying the backend, since you are disabling the safety gate for the manifest-pointer protocol"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"# Pre-deploy: run the same probe the gate runs, in staging, against the\n# production-shaped S3/MinIO:\n#   BUZZ_GIT_CONFORMANCE_PROBE=true ./buzz-relay   (staging only)\n# Passing there before deploy prevents a fatal boot in production.","typeGuard":null,"tryCatchPattern":"// Distinguish transport errors (fix endpoint/credentials, retry boot) from\n// CAS/race failures (replace or upgrade the object-store backend — retries\n// cannot help a backend that lacks linearizable conditional writes).","preventionTips":["Choose an S3-compatible backend that implements conditional writes (recent MinIO, AWS S3)","Run the conformance probe in staging against production-shaped storage before each deploy","Do not set BUZZ_GIT_CONFORMANCE_PROBE=false to make boot pass — it disables the safety gate the manifest-pointer protocol relies on","Keep probe pressure (BUZZ_GIT_PROBE_WRITERS/ROUNDS) within the backend's throttle budget"],"tags":["git","s3","minio","concurrency","startup","buzz-relay"],"backgroundTag":"storage-conformance-check-failed","analyzedSha":"ef2aa1ae38fadcc0bc22b8bf6ed96b35933146be","analyzedAt":"2026-08-20T04:38:24.874Z","contentChangedAt":"2026-08-20T04:38:24.874Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}