risingwavelabs/risingwave · error · SinkError::Config

`enable_manifest_rewrite` is not supported for Iceberg…

Error message

`enable_manifest_rewrite` is not supported for Iceberg format version 3 because rewrite manifests cannot preserve row lineage

What it means

`validate_manifest_rewrite_format` rejects enabling manifest rewrite on Iceberg format version 3 tables. In v3, manifest files must preserve row lineage, which rewriting manifests would destroy, so the combination `enable_manifest_rewrite = true` + `format_version >= V3` is an error.

Solutions

1. Remove or set `enable_manifest_rewrite = false` in the sink options.
2. Or recreate/keep the target table at format version 2 if manifest rewrite is required.
3. Check table.metadata().format_version() before enabling the feature.
4. Update sink templates that predate format v3 support.

Example fix

-- before
WITH ( enable_manifest_rewrite = true )
-- after
WITH ( enable_manifest_rewrite = false )

let fv = table.metadata().format_version();
if enable_manifest_rewrite && fv >= FormatVersion::V3 {
    return Err("disable enable_manifest_rewrite for format v3 tables");
}

fn manifest_rewrite_allowed(fv: FormatVersion, enable: bool) -> bool { !enable || fv < FormatVersion::V3 }

match sink_result {
    Err(SinkError::Config(m)) if m.to_string().contains("enable_manifest_rewrite") => {
        // recreate sink without enable_manifest_rewrite
    }
    other => other?,
}

When it happens

Trigger: Creating/updating an Iceberg sink with `enable_manifest_rewrite` true while the target table metadata reports format version 3 (checked via `validate_manifest_rewrite_format(format_version)`).

Common situations: Table upgraded to format version 3 (e.g. by another engine) while the sink still carries `enable_manifest_rewrite = true`; new tables defaulting to v3 with old sink configs copied from v1/v2 setups.

Understand the failure class

Background: Conflicting config options: "cannot be used together" — configuration validation errors across open-source libraries — this error's family across 162 libraries.

Related errors

• `compaction-interval-sec` must be greater than 0 when…
• `compaction.max_snapshots_num` must be greater than 0, got
• `compaction.target_file_size_mb` must be greater than 0
• `compaction.type` must not be set when `write_mode` is…
• `compaction.write_parquet_compression` must be one of

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/a1670bac753403b5. Report an issue: GitHub.

Appendix: source

Thrown at src/connector/src/sink/iceberg/config.rs:775

            .await
            .map_err(Into::into)
    }

    pub fn full_table_name(&self) -> Result<TableIdent> {
        self.table.to_table_ident().map_err(Into::into)
    }

    pub fn catalog_name(&self) -> String {
        self.common.catalog_name()
    }

    pub fn table_format_version(&self) -> FormatVersion {
        self.format_version
    }

    pub fn validate_manifest_rewrite_format(&self, format_version: FormatVersion) -> Result<()> {
        if self.enable_manifest_rewrite && format_version >= FormatVersion::V3 {
            return Err(SinkError::Config(anyhow!(
                "`enable_manifest_rewrite` is not supported for Iceberg format version 3 because rewrite manifests cannot preserve row lineage"
            )));
        }
        Ok(())
    }

    pub fn compaction_interval_sec(&self) -> u64 {
        // default to 1 hour
        self.compaction_interval_sec.unwrap_or(3600)
    }

    /// Calculate the timestamp (in milliseconds) before which snapshots should be expired
    /// Returns `current_time_ms` - `max_age_millis`
    pub fn snapshot_expiration_timestamp_ms(&self, current_time_ms: i64) -> Option<i64> {
        self.snapshot_expiration_max_age_millis
            .map(|max_age_millis| current_time_ms - max_age_millis)
    }

View on GitHub (pinned to 6469eb736d)