risingwavelabs/risingwave · warning

iceberg compaction task is already running for sink {} (stat

Error message

iceberg compaction task is already running for sink {} (state=in_flight, task_id={}, pending_commit_count_at_start={}, pending_commit_count={}, report_timeout_after_sec={})

What it means

The compaction scheduler rejects a request because the sink's compaction task is InFlight: a task with the given task_id has been dispatched to a worker and has not yet reported completion. The error reports task_id, pending commit counts, and the seconds remaining before the report deadline so callers know when the state may self-clean via timeout.

Source

Thrown at src/meta/src/manager/iceberg_compaction/schedule.rs:942

            match &track.state {
                CompactionTrackState::PendingDispatch { attempt } => {
                    return Err(anyhow!(
                        "iceberg compaction task is already running for sink {} \
                         (state=pending_dispatch, pending_commit_count_at_start={}, \
                         pending_commit_count={})",
                        sink_id,
                        attempt.pending_commit_count_at_start,
                        track.pending_commit_count
                    )
                    .into());
                }
                CompactionTrackState::InFlight {
                    task_id,
                    attempt,
                    report_deadline,
                    ..
                } => {
                    return Err(anyhow!(
                        "iceberg compaction task is already running for sink {} \
                         (state=in_flight, task_id={}, pending_commit_count_at_start={}, \
                         pending_commit_count={}, report_timeout_after_sec={})",
                        sink_id,
                        task_id,
                        attempt.pending_commit_count_at_start,
                        track.pending_commit_count,
                        report_deadline.saturating_duration_since(now).as_secs()
                    )
                    .into());
                }
                CompactionTrackState::Idle { .. } => {}
            }
        }

        if self.apply_sink_update(&mut guard, prepared_update) {
            let (tx, rx) = oneshot::channel();
            guard.manual_compaction_waiters.insert(sink_id, tx);

View on GitHub (pinned to 6469eb736d)

Solutions

1. Wait until report_timeout_after_sec elapses or the task reports completion, then retry
2. Check meta logs and worker logs for task_id to see whether the worker is alive and progressing
3. If the worker died without reporting, the scheduler will time out the task and reset the track to Idle; retry after that
4. Investigate worker crash/network issues that prevent task reports

Example fix

// before
let rx = manager.start_manual_compaction(sink_id).await?; // Err: in_flight, timeout in Ns
// after
// Wait out the report deadline before retrying
tokio::time::sleep(Duration::from_secs(timeout_after_sec + 1)).await;
manager.start_manual_compaction(sink_id).await?;

match rx_or_err {
    Err(e) if e.to_string().contains("state=in_flight") => {
        // parse report_timeout_after_sec from message and wait it out
        let secs = parse_timeout_secs(&e.to_string());
        tokio::time::sleep(Duration::from_secs(secs + 1)).await;
        retry(sink_id)
    }
    other => other,
}

When it happens

Trigger: Calling start_manual_compaction (or applying a sink update) while guard.sink_schedules[sink].state is CompactionTrackState::InFlight { task_id, attempt, report_deadline, .. }.

Common situations: A worker is still executing the compaction task; a manual compaction attempt collides with it. If report_timeout_after_sec is 0 or small, the task is about to be timed out and can be retried shortly; a consistently hung task points to worker failure.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

• Timeouts: ETIMEDOUT, deadlines, and hung requests — what actually expires when a request times out.

Related errors

• iceberg compaction task is already running for sink {} (stat
• manual iceberg compaction is already waiting for sink {}
• manual Full compaction is rejected while an automatic round
• commit iceberg transaction
• reload iceberg table

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/f3739d8dfadc1751. Report an issue: GitHub.