risingwavelabs/risingwave · error · SinkError::Config

`enable_pk_index` requires an Iceberg table with format…

Error message

`enable_pk_index` requires an Iceberg table with format version >= 2, but the target table is format version {}

What it means

When `enable_pk_index` is set, `create_and_validate_table_impl` requires the target Iceberg table to use format version >= 2, because the PK index feature relies on v2 metadata (e.g. sequence numbers/row lineage prerequisites). Tables at v1 are rejected with this explicit message.

Solutions

1. Recreate the target Iceberg table at format version 2 (or later).
2. Or set the table's format-version property (e.g. `format-version=2` when creating via Spark/Flink/iceberg-rust).
3. Or disable `enable_pk_index` if v1 compatibility matters more.
4. Check the existing table's metadata before enabling the option.

Example fix

-- before (Spark DDL of target table)
CREATE TABLE catalog.db.events (...) TBLPROPERTIES ('format-version'='1');
-- after
CREATE TABLE catalog.db.events (...) TBLPROPERTIES ('format-version'='2');

let fv = table.metadata().format_version();
if enable_pk_index && fv < FormatVersion::V2 {
    return Err(format!("enable_pk_index needs format >= 2, table is {fv:?}"));
}

fn pk_index_supported(fv: FormatVersion) -> bool { fv >= FormatVersion::V2 }

When it happens

Trigger: Creating an Iceberg sink with `enable_pk_index = true` whose loaded table's `metadata().format_version()` is `V1`.

Common situations: Old tables created years ago at format version 1 being reused for a new sink; warehouse defaults producing v1 tables; configs enabling pk_index without checking table format.

Understand the failure class

Background: "Invalid value" and "allowed values are" config errors: what your library rejected and how to fix it — this error's family across 41 libraries.

Related errors

• bounded compaction requires Iceberg format V2 or V3
• `compaction-interval-sec` must be greater than 0 when…
• `compaction.max_snapshots_num` must be greater than 0, got
• `compaction.target_file_size_mb` must be greater than 0
• `compaction.type` must not be set when `write_mode` is…

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/fcd57306f267b1ac. Report an issue: GitHub.

Appendix: source

Thrown at src/connector/src/sink/iceberg/create_table.rs:78

}

pub async fn create_and_validate_table_impl(
    config: &IcebergConfig,
    param: &SinkParam,
) -> Result<Table> {
    if config.create_table_if_not_exists {
        create_table_if_not_exists_impl(config, param).await?;
    }

    let table = config
        .load_table()
        .await
        .map_err(|err| SinkError::Iceberg(anyhow!(err)))?;

    if config.enable_pk_index {
        let table_format_version = table.metadata().format_version();
        if table_format_version < FormatVersion::V2 {
            return Err(SinkError::Config(anyhow!(
                "`enable_pk_index` requires an Iceberg table with format version >= 2, \
                 but the target table is format version {}",
                table_format_version
            )));
        }
    }

    let sink_schema = param.schema();
    let iceberg_arrow_schema = schema_to_arrow_schema(table.metadata().current_schema())
        .map_err(|err| SinkError::Iceberg(anyhow!(err)))?;

    try_matches_arrow_schema(&sink_schema, &iceberg_arrow_schema)
        .map_err(|err| SinkError::Iceberg(anyhow!(err)))?;

    Ok(table)
}

/// Returns `true` if this call created the table, `false` if it already existed.

View on GitHub (pinned to 6469eb736d)