risingwavelabs/risingwave · error

No iceberg compactor available

Error message

No iceberg compactor available

What it means

Manual Iceberg compaction requires at least one registered Iceberg compactor worker. trigger_manual_compaction fast-fails before scheduling when iceberg_compactor_manager.compactor_num() == 0, instead of letting the request hang on a waiter no compactor would ever complete.

Source

Thrown at src/meta/src/manager/iceberg_compaction/manual.rs:59

                    report.task_id,
                    report.sink_id,
                    message
                )
                .into())
            }
        };

        let _ = waiter.send(result);
    }

    pub async fn trigger_manual_compaction(
        &self,
        sink_id: SinkId,
    ) -> MetaResult<IcebergCompactionTaskId> {
        // Fast-fail before registering a waiter when no compactor can pull the
        // scheduler task.
        if self.iceberg_compactor_manager.compactor_num() == 0 {
            return Err(anyhow!("No iceberg compactor available").into());
        }

        let waiter = self.start_manual_compaction(sink_id).await?;
        let cleanup_guard = scopeguard::guard(sink_id, |sink_id| {
            self.cancel_manual_compaction_waiter(sink_id)
        });

        tracing::info!(
            "Manual compaction requested for sink {}, waiting for completion...",
            sink_id
        );

        match waiter.await {
            Ok(result) => {
                let _ = scopeguard::ScopeGuard::into_inner(cleanup_guard);
                result
            }
            Err(_) => {

View on GitHub (pinned to 6469eb736d)

Solutions

1. Start or restart the compactor component so it registers with the meta node.
2. Check compactor logs/health for registration failures (ports, advertise address).
3. Retry trigger_manual_compaction once compactor_num() > 0.
4. In dev, use a risedev profile that includes the compactor.

Example fix

// before: manual compaction without a running compactor
./risedev psql -c "CALL rw_iceberg_compaction('sink')"; // No iceberg compactor available
// after: launch profile with compactor
./risedev d # profile includes compactor node
./risedev psql -c "CALL rw_iceberg_compaction('sink')";

// Caller-side pre-check via meta metrics/logs or expose compactor_num
if meta.iceberg_compactor_count() == 0 {
    return Err("no iceberg compactor registered; start the compactor component first".into());
}

match trigger_manual_compaction(sink_id).await {
    Err(e) if e.to_string().contains("No iceberg compactor available") => {
        // start/scale compactor workers, then retry
        ensure_compactor_running().await?;
        retry(|| trigger_manual_compaction(sink_id)).await
    }
    other => other,
}

When it happens

Trigger: Calling trigger_manual_compaction(sink_id) when no compactor worker is connected to the meta (compactor_num() returns 0).

Common situations: Compactor process not started (or crashed) in the deployment; running a meta-only profile without the compactor component; network partitions disconnecting all compactor workers.

Understand the failure class

Background: 'Could not be found', 'does not exist', 'not found in database': the resource-not-found family when an ID, slug, key, or URI lookup comes back empty — this error's family across 20 libraries.

Related errors

• Manual iceberg compaction task {} for sink {} failed: {}
• Manual iceberg compaction waiter dropped unexpectedly for si
• Iceberg compaction task failed before dispatch for sink {}
• manual iceberg compaction is already waiting for sink {}
• Iceberg branch {} disappeared after manifest rewrite for sin

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/3aea8ec3e9fe5811. Report an issue: GitHub.