risingwavelabs/risingwave · warning

iceberg compaction task is already running for sink {} (stat

Error message

iceberg compaction task is already running for sink {} (state=pending_dispatch, pending_commit_count_at_start={}, pending_commit_count={})

What it means

The compaction scheduler rejects a new sink update because the sink's compaction track is in PendingDispatch state: a compaction task has been prepared but not yet dispatched to a worker, so another task cannot be started for the same sink. The error includes the pending commit counts captured at attempt start and now, for diagnosis.

Source

Thrown at src/meta/src/manager/iceberg_compaction/schedule.rs:926

        if guard.manual_compaction_waiters.contains_key(&sink_id) {
            return Err(anyhow!(
                "manual iceberg compaction is already waiting for sink {}",
                sink_id
            )
            .into());
        }

        if let Some(track) = guard.sink_schedules.get(&sink_id) {
            if track.round_max_file_sequence_number.is_some() {
                return Err(anyhow!(
                    "manual Full compaction is rejected while an automatic round is active for sink {}",
                    sink_id
                )
                .into());
            }
            match &track.state {
                CompactionTrackState::PendingDispatch { attempt } => {
                    return Err(anyhow!(
                        "iceberg compaction task is already running for sink {} \
                         (state=pending_dispatch, pending_commit_count_at_start={}, \
                         pending_commit_count={})",
                        sink_id,
                        attempt.pending_commit_count_at_start,
                        track.pending_commit_count
                    )
                    .into());
                }
                CompactionTrackState::InFlight {
                    task_id,
                    attempt,
                    report_deadline,
                    ..
                } => {
                    return Err(anyhow!(
                        "iceberg compaction task is already running for sink {} \
                         (state=in_flight, task_id={}, pending_commit_count_at_start={}, \

View on GitHub (pinned to 6469eb736d)

Solutions

1. Wait for the pending task to be dispatched and complete, then retry
2. Check meta logs for dispatch failures (worker unavailable) and ensure stream workers are healthy
3. If the pending state is stuck (no dispatch progress), restart the meta node to rebuild the in-memory schedule state
4. Investigate why pending_commit_count is not draining (downstream commit failures for the sink)

Example fix

// before
let rx = manager.start_manual_compaction(sink_id).await?; // Err: pending_dispatch
// after
match manager.start_manual_compaction(sink_id).await {
    Ok(rx) => rx.await,
    Err(e) if e.to_string().contains("already running") => {
        // back off and retry after current task completes
        tokio::time::sleep(CHECK_INTERVAL).await;
        retry(sink_id)
    }
    Err(e) => Err(e),
}

// Retry only when the track is idle
fn track_ready_for_update(track: Option<&CompactionTrack>) -> bool {
    track.map(|t| matches!(t.state, CompactionTrackState::Idle { .. })).unwrap_or(true)
}

match start_manual_compaction(sink_id).await {
    Err(e) if e.to_string().contains("already running") && e.to_string().contains("pending_dispatch") => {
        tokio::time::sleep(RETRY_DELAY).await;
        retry(sink_id)
    }
    other => other,
}

When it happens

Trigger: Calling start_manual_compaction (or otherwise applying a sink update) while guard.sink_schedules[sink].state is CompactionTrackState::PendingDispatch { attempt }.

Common situations: A compaction task was scheduled but its dispatch is delayed (worker contention, backpressure); operators issue a manual compaction that collides with the pending task. Commit count imbalance in the message hints at in-flight data commits the task is waiting on.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

• iceberg compaction task is already running for sink {} (stat
• manual iceberg compaction is already waiting for sink {}
• manual Full compaction is rejected while an automatic round
• commit iceberg transaction
• reload iceberg table

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/4c3b6b308d14ea3b. Report an issue: GitHub.