risingwavelabs/risingwave · error · HummockError

failed to build iceberg compaction execution config

Error message

failed to build iceberg compaction execution config

What it means

During iceberg compaction the runner builds an icebx/parquet writer ExecutionConfig via a typed builder (writer properties, target file size, concurrency, prefetch). If the builder's validation fails, the error is wrapped with this context and surfaced as a HummockError::compaction_executor. It means the local compaction parameters assembled from IcebergConfig/runner config were rejected, not that compaction itself failed.

Source

Thrown at src/storage/src/hummock/compactor/iceberg_compaction/iceberg_compactor_runner.rs:429

        // Build writer properties from sink configuration
        let write_parquet_properties = WriterProperties::builder()
            .set_compression(iceberg_config.get_parquet_compression())
            .set_max_row_group_bytes(iceberg_config.write_parquet_max_row_group_bytes())
            .set_created_by(concat!("risingwave version ", env!("CARGO_PKG_VERSION")).to_owned())
            .build();

        let compaction_execution_config = CompactionExecutionConfigBuilder::default()
            .enable_validate_compaction(config.enable_validate_compaction)
            .max_record_batch_rows(config.max_record_batch_rows)
            .write_parquet_properties(write_parquet_properties)
            .target_file_size_bytes(iceberg_config.target_file_size_mb() * 1024 * 1024)
            .max_concurrent_closes(config.max_concurrent_closes)
            .enable_prefetch(config.enable_prefetch)
            .build()
            .map_err(|e| {
                HummockError::compaction_executor(
                    anyhow::Error::new(e)
                        .context("failed to build iceberg compaction execution config"),
                )
            })?;

        tracing::info!(
            iceberg_component = "compaction_worker",
            iceberg_operation = "execute_plan",
            task_id = %task_id,
            plan_index = plan_index,
            task_type = ?compaction_kind,
            table = %table_ident,
            branch = %branch,
            input_parallelism = compaction_plan.recommended_executor_parallelism(),
            output_parallelism = compaction_plan.recommended_output_parallelism(),
            memory_reservation_bytes,
            statistics = ?statistics,
            "iceberg_compaction_plan_started",
        );

View on GitHub (pinned to 6469eb736d)

Solutions

1. Inspect the underlying error chained after this context message in the logs — it names the exact violated builder invariant.
2. Check the iceberg compaction config values (target_file_size_mb, max_concurrent_closes, enable_prefetch) in risingwave.toml / system parameters and reset them to valid positive values.
3. Re-apply default configuration (remove overrides for iceberg compaction keys) and restart the compactor.
4. If defaults themselves fail to build, this is an internal bug — report it with the full error report.

Example fix

// before
iceberg_compaction_target_file_size_mb = 0   # invalid

// after
iceberg_compaction_target_file_size_mb = 256  # positive, within allowed range

fn validate_iceberg_compaction_config(cfg: &IcebergCompactorRunnerConfig) -> Result<(), String> {
    if cfg.target_file_size_mb == 0 { return Err("target_file_size_mb must be > 0".into()); }
    if cfg.max_concurrent_closes == 0 { return Err("max_concurrent_closes must be > 0".into()); }
    Ok(())
}

match build_execution_config(&cfg) {
    Ok(c) => c,
    Err(e) => { tracing::error!(root = ?e.root_cause(), "iceberg execution config rejected"); return Err(e); }
}

When it happens

Trigger: IcebergCompactionExecutionConfig builder validation fails after applying iceberg_config.target_file_size_mb()*1024*1024, max_concurrent_closes, and enable_prefetch — typically a zero/out-of-range target_file_size_mb or invalid concurrency setting from config.

Common situations: A hand-edited risingwave.toml (or corrupted cluster parameter) sets iceberg target_file_size_mb to 0 or an absurd value; a user overrides internal config via system parameters with a wrong type; a bug in a config migration produces an invalid combination of writer options.

Understand the failure class

Background: "Invalid value" and "allowed values are" config errors: what your library rejected and how to fix it — this error's family across 41 libraries.

Related errors

• Manual iceberg compaction task {} for sink {} failed: {}
• No iceberg compactor available
• Manual iceberg compaction waiter dropped unexpectedly for si
• Iceberg compaction task failed before dispatch for sink {}
• manual iceberg compaction is already waiting for sink {}

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/512ee3445d0a9d87. Report an issue: GitHub.