risingwavelabs/risingwave · error

failed to trigger manual iceberg compaction for sink

Error message

failed to trigger manual iceberg compaction for sink {}

What it means

start_manual_compaction prepared a manual Full compaction update for the sink but apply_sink_update returned false, meaning the prepared update could not be applied to the manager state (e.g. the sink has no registered schedule or the state changed after preparation). The internal oneshot waiter is never registered and an error is returned to the caller.

Solutions

  1. Verify the sink ID is correct and the sink exists with Iceberg compaction enabled
  2. Check whether the sink has a compaction schedule (automatic compaction configured) before issuing manual compaction
  3. Retry: the mismatch may be a transient race between prepare and apply
  4. If the sink was dropped concurrently, re-create the sink or abandon the compaction request

Example fix

// before
let rx = manager.start_manual_compaction(sink_id.clone()).await?;
// after
if !manager.sink_exists(&sink_id) {
    return Err(anyhow!("sink {} not found; cannot compact", sink_id));
}
let rx = manager.start_manual_compaction(sink_id.clone()).await
    .map_err(|e| e.context("ensure sink has compaction schedule before manual trigger"))?;
Defensive patterns

Strategy: validation

Validate before calling

// Verify the sink exists and has a compaction schedule before triggering
fn manual_compaction_applicable(sink_exists: bool, has_schedule: bool) -> bool {
    sink_exists && has_schedule
}

Try / catch

match start_manual_compaction(sink_id).await {
    Err(e) if e.to_string().contains("failed to trigger") => {
        Err(anyhow!("sink {} missing or has no compaction schedule", sink_id).context(e))
    }
    other => other,
}

Prevention

When it happens

Trigger: Calling start_manual_compaction when the sink has no entry in sink_schedules or its track state does not permit applying the ManualForceCompaction update (apply_sink_update returns false).

Common situations: Manual compaction issued for a sink ID that does not exist / whose sink was just dropped, or the sink never had automatic compaction configured so no schedule track exists; race where the track is mutated between prepare and apply.

Understand the failure class

Background: "Not found" and "does not exist" errors: why "Task not found", "No such folder", and "Can't find" fire when a lookup comes back empty — this error's family across 14 libraries.

Related errors

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/56fa0ce432b485ba. Report an issue: GitHub.

Appendix: source

Thrown at src/meta/src/manager/iceberg_compaction/schedule.rs:963

                         pending_commit_count={}, report_timeout_after_sec={})",
                        sink_id,
                        task_id,
                        attempt.pending_commit_count_at_start,
                        track.pending_commit_count,
                        report_deadline.saturating_duration_since(now).as_secs()
                    )
                    .into());
                }
                CompactionTrackState::Idle { .. } => {}
            }
        }

        if self.apply_sink_update(&mut guard, prepared_update) {
            let (tx, rx) = oneshot::channel();
            guard.manual_compaction_waiters.insert(sink_id, tx);
            Ok(rx)
        } else {
            Err(anyhow!(
                "failed to trigger manual iceberg compaction for sink {}",
                sink_id
            )
            .into())
        }
    }

    pub(super) fn cancel_manual_compaction_waiter(&self, sink_id: SinkId) {
        self.inner
            .write()
            .manual_compaction_waiters
            .remove(&sink_id);
    }

    fn finish_timed_out_compaction_tasks(
        guard: &mut IcebergCompactionManagerInner,
        now: Instant,
    ) -> Vec<(SinkId, ManualCompactionWaiter)> {

View on GitHub (pinned to 6469eb736d)