risingwavelabs/risingwave · error

Iceberg compaction task failed before dispatch for sink {}

Error message

Iceberg compaction task failed before dispatch for sink {}

What it means

The scheduler entry's Drop implementation must never leak a registered manual-compaction waiter: if the task is dropped before being dispatched to a compactor, it sends a pre-formatted error through the waiter channel so trigger_manual_compaction fails fast instead of hanging forever.

Source

Thrown at src/meta/src/manager/iceberg_compaction/schedule.rs:572

                Some(track.revert_pre_dispatch_failure(Instant::now()))
            } else {
                None
            };
            if let Some(finish_action) = finish_action {
                let waiter = guard.manual_compaction_waiters.remove(&self.sink_id);
                IcebergCompactionManager::apply_track_finish_action(
                    &mut guard,
                    self.sink_id,
                    finish_action,
                );
                waiter
            } else {
                None
            }
        };

        if let Some(waiter) = waiter {
            let _ = waiter.send(Err(anyhow!(
                "Iceberg compaction task failed before dispatch for sink {}",
                self.sink_id
            )
            .into()));
        }
    }
}

#[derive(Debug, Clone)]
enum SinkUpdateKind {
    /// A normal sink commit. It increases the pending snapshot count.
    Commit {
        observed_snapshot: IcebergCommittedSnapshot,
    },
    /// A force signal from the sink update path. It triggers the configured
    /// compaction type and still follows the automatic-compaction config gate.
    ForceCompaction {
        observed_snapshot: IcebergCommittedSnapshot,

View on GitHub (pinned to 6469eb736d)

Solutions

1. Retry the manual compaction after the scheduler is healthy.
2. Check meta logs for scheduler shutdown/eviction around the failure time.
3. If this happens without shutdown/cancellation, it indicates a Drop-path bug; report with sink_id and logs.
4. Ensure compactor workers are online so tasks get dispatched instead of evicted.

match trigger_manual_compaction(sink_id).await {
    Err(e) if e.to_string().contains("failed before dispatch") => {
        // scheduler dropped the task early; retry while scheduler is healthy
        retry_with_backoff(3, || trigger_manual_compaction(sink_id)).await
    }
    other => other,
}

When it happens

Trigger: An IcebergCompactionScheduler task struct is dropped before dispatch (e.g. queue eviction, cancellation, scheduler shutdown, early error return) while a waiter Option is still Some.

Common situations: Meta node shutting down with pending manual compaction tasks; task cancelled/evicted before a compactor picked it up; internal bugs dropping the scheduler entry without completing the waiter.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

• Manual iceberg compaction task {} for sink {} failed: {}
• No iceberg compactor available
• Manual iceberg compaction waiter dropped unexpectedly for si
• manual iceberg compaction is already waiting for sink {}
• Iceberg branch {} disappeared after manifest rewrite for sin

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/d4536f41b936f3fa. Report an issue: GitHub.