block/buzz · critical · anyhow::Error

Community deletion serving fence is unsafe

Error message

Community deletion serving fence is unsafe: {e}

What it means

Fatal startup error: validate_deletion_serving_catalog() checked that the Postgres catalog backing community-deletion serving fences actually enforces what the fence protocol assumes (commit-time floor etc.). If the live schema does not match, the relay refuses to start rather than risk serving events from deleted communities through the fence.

Solutions

  1. Apply all pending migrations: start with BUZZ_AUTO_MIGRATE=true (or run the migration tooling) so the fence catalog matches the relay version
  2. Inspect the {e} detail for which catalog object/check failed and verify in psql
  3. Confirm the schema_migrations table shows the deletion-fence migrations (0021 and later) as applied
  4. Do not bypass this check — it guards deleted-community data from being served
Defensive patterns

Strategy: validation

Validate before calling

# Before deploying a new relay image with BUZZ_AUTO_MIGRATE off:
# confirm the fence migrations are recorded:
psql "$DATABASE_URL" -c "select version from schema_migrations order by version desc limit 5"

Prevention

When it happens

Trigger: The deletion-serving-fence catalog objects (from the migration 0021 lineage) are missing or shaped wrong on the live DB: migrations not applied while older data exists, a partially applied migration, or manual schema changes that dropped/altered the fence objects.

Common situations: Running a newer relay image against a database with BUZZ_AUTO_MIGRATE left off in a managed-migration deployment; restoring a DB dump that skipped catalog objects; environments where migrations are applied out of order.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

AI-assisted analysis of block/buzz@ef2aa1ae38 (2026-08-20). Data as JSON: /api/errors/a937a169e15458e6. Report an issue: GitHub.

Appendix: source

Thrown at crates/buzz-relay/src/main.rs:291

    let auto_migrate =
        buzz_auto_migrate_enabled(std::env::var("BUZZ_AUTO_MIGRATE").ok().as_deref());
    if auto_migrate {
        db.migrate().await.map_err(|e| {
            error!("Failed to run database migrations: {e}");
            anyhow::anyhow!("Database migration failed: {e}")
        })?;
        info!("Database migrations complete");
    } else {
        info!("Skipping database migrations because BUZZ_AUTO_MIGRATE is not enabled");
    }

    if let Err(e) = db.ensure_future_partitions(3).await {
        error!("Failed to ensure partitions: {e}");
    }

    db.validate_deletion_serving_catalog().await.map_err(|e| {
        error!("Community deletion serving-fence validation failed: {e}");
        anyhow::anyhow!("Community deletion serving fence is unsafe: {e}")
    })?;
    info!("Community deletion serving fences verified");

    // Freshness fence probe: cursor pages route to the replica only for
    // history the probe has verified as fully replayed. Deliberately AFTER
    // the migration decision: spawn_fence_probe first verifies the
    // commit-time floor guard (catalog shape + observed behavior through the
    // armed pool) against the live schema, so a relay running with
    // BUZZ_AUTO_MIGRATE off and migration 0021 unapplied can never open the
    // fence over an unenforced floor. Verification failure is loud but
    // non-fatal: the fence stays closed and every cursor page routes to the
    // writer.
    match db.spawn_fence_probe().await {
        Ok(true) => info!("Replica fence probe started (floor guard verified)"),
        Ok(false) => {}
        Err(e) => {
            error!(
                "Replica fence disabled — floor guard verification failed: {e}. \

View on GitHub (pinned to ef2aa1ae38)