{"record":{"id":"a937a169e15458e6","repo":"block/buzz","slug":"community-deletion-serving-fence-is-unsafe-e","errorCode":null,"errorMessage":"Community deletion serving fence is unsafe: {e}","messagePattern":"Community deletion serving fence is unsafe: (.+?)","errorType":"exception","errorClass":"anyhow::Error","httpStatus":null,"severity":"critical","filePath":"crates/buzz-relay/src/main.rs","lineNumber":206,"sourceCode":"    let auto_migrate =\n        buzz_auto_migrate_enabled(std::env::var(\"BUZZ_AUTO_MIGRATE\").ok().as_deref());\n    if auto_migrate {\n        db.migrate().await.map_err(|e| {\n            error!(\"Failed to run database migrations: {e}\");\n            anyhow::anyhow!(\"Database migration failed: {e}\")\n        })?;\n        info!(\"Database migrations complete\");\n    } else {\n        info!(\"Skipping database migrations because BUZZ_AUTO_MIGRATE is not enabled\");\n    }\n\n    if let Err(e) = db.ensure_future_partitions(3).await {\n        error!(\"Failed to ensure partitions: {e}\");\n    }\n\n    db.validate_deletion_serving_catalog().await.map_err(|e| {\n        error!(\"Community deletion serving-fence validation failed: {e}\");\n        anyhow::anyhow!(\"Community deletion serving fence is unsafe: {e}\")\n    })?;\n    info!(\"Community deletion serving fences verified\");\n\n    // Freshness fence probe: cursor pages route to the replica only for\n    // history the probe has verified as fully replayed. Deliberately AFTER\n    // the migration decision: spawn_fence_probe first verifies the\n    // commit-time floor guard (catalog shape + observed behavior through the\n    // armed pool) against the live schema, so a relay running with\n    // BUZZ_AUTO_MIGRATE off and migration 0021 unapplied can never open the\n    // fence over an unenforced floor. Verification failure is loud but\n    // non-fatal: the fence stays closed and every cursor page routes to the\n    // writer.\n    match db.spawn_fence_probe().await {\n        Ok(true) => info!(\"Replica fence probe started (floor guard verified)\"),\n        Ok(false) => {}\n        Err(e) => {\n            error!(\n                \"Replica fence disabled — floor guard verification failed: {e}. \\","sourceCodeStart":188,"sourceCodeEnd":224,"githubUrl":"https://github.com/block/buzz/blob/f956e6fe06a76e50cbd8fba1a162482e752e7f1a/crates/buzz-relay/src/main.rs#L188-L224","documentation":"Fatal startup error from db.validate_deletion_serving_catalog(), which verifies the catalog backing the community-deletion serving fence — the mechanism that keeps a half-deleted community from serving stale events. If catalog shape or contents cannot be proven safe, the relay refuses to boot rather than serve potentially unfenced data. The source notes migration 0021 as the floor-guard schema this depends on.","triggerScenarios":"Running with BUZZ_AUTO_MIGRATE off while fence-related migrations (0021 era) are unapplied; fence catalog tables missing or malformed; manually edited catalog rows.","commonSituations":"New binary against a database whose migrations are pinned/managed externally and one was skipped; partial database restore; environments where DBAs cherry-pick migrations.","solutions":["Apply pending migrations (enable BUZZ_AUTO_MIGRATE once, or run them manually), then restart","Inspect {e} for the specific catalog/table mismatch it names","Compare the live schema against migrations/ up to HEAD, focusing on the deletion-serving fence migration","If the catalog was hand-edited, restore from a known-good backup rather than patching rows"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"# Fence depends on current schema: assert applied migrations match migrations/ HEAD.\nlatest=$(ls migrations | grep -oE '^[0-9]+' | sort -n | tail -1)\napplied=$(psql \"$DATABASE_URL\" -tAc \"select max(version) from _sqlx_migrations\")\n[ \"$latest\" = \"$(echo $applied | tr -d ' ')\" ] || { echo 'fence catalog may be stale: apply migrations'; exit 1; }","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Never hand-edit fence/catalog tables; change them only through migrations","Keep BUZZ_AUTO_MIGRATE on, or make external migration application a deploy gate","After restoring from backup, re-run the fence validation before returning the relay to service"],"tags":["postgres","data-integrity","startup","fence","migrations"],"backgroundTag":"schema-invariant-check-failed","analyzedSha":"f956e6fe06a76e50cbd8fba1a162482e752e7f1a","analyzedAt":"2026-08-16T22:11:40.750Z","schemaVersion":2},"datasetVersion":"2026-08-16T23:17:17.608Z"}