risingwavelabs/risingwave · error · SinkError::Iceberg

failed to add partition columns

Error message

failed to add partition columns

What it means

After building the partition field descriptors, they are attached to an UnboundPartitionSpec via add_partition_fields. If the iceberg crate rejects any field (invalid transform for the source type, duplicate names/ids, conflicting field ids), the error is wrapped with this static context message.

Solutions

  1. Read the chained underlying error to see which partition field was rejected.
  2. Match the transform to the column type (e.g. use `day` only on date/timestamp columns, `truncate`/`bucket` on appropriate types).
  3. Fix the partition config in the sink WITH options and recreate the sink.
  4. Check for duplicate partition column names.

Example fix

// before
WITH (connector='iceberg', partition.by='day(user_name)') -- day() on a VARCHAR
// after
WITH (connector='iceberg', partition.by='truncate(user_name, 10)')
-- or partition by a timestamp column: partition.by='day(event_time)'
Defensive patterns

Strategy: validation

Validate before calling

// Check each partition transform is valid for its column type before creating
// e.g. day/month/year only on date/timestamp; truncate/bucket per iceberg rules
for (col, transform) in partition_specs {
    if !transform_supports(transform, type_of(col)) {
        eprintln!("transform {transform} unsupported for column {col} type");
    }
}

Try / catch

match result {
    Err(e) if e.to_string().contains("failed to add partition columns") => {
        // fix transform/column pairing in partition.by, then recreate the sink
    }
    other => other?,
}

Prevention

When it happens

Trigger: create_table_if_not_exists_impl calls UnboundPartitionSpec::builder().with_spec_id(0).add_partition_fields(partition_fields) and it returns Err — typically a transform not applicable to the column's type (e.g. year/month/day on a non-date/timestamp column, bucket on unsupported types) or an invalid field id assignment.

Common situations: Partition transform (year/month/day/hour/truncate/bucket) applied to a column type it doesn't support; partitioning on VARCHAR with a temporal transform; multiple partition fields colliding on generated names (_p_<column>).

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/7a9c5743e48fd42b. Report an issue: GitHub.

Appendix: source

Thrown at src/connector/src/sink/iceberg/create_table.rs:206

                    Some(id) => partition_fields.push(
                        UnboundPartitionField::builder()
                            .source_id(id)
                            .transform(transform)
                            .name(format!("_p_{}", column))
                            .field_id(PARTITION_DATA_ID_START + i as i32)
                            .build(),
                    ),
                    None => bail!(format!(
                        "Partition source column does not exist in schema: {}",
                        column
                    )),
                };
            }
            Some(
                UnboundPartitionSpec::builder()
                    .with_spec_id(0)
                    .add_partition_fields(partition_fields)
                    .map_err(|e| SinkError::Iceberg(anyhow!(e)))
                    .context("failed to add partition columns")?
                    .build(),
            )
        }
        None => None,
    };

    let sort_order = match &config.order_key {
        Some(order_key) => Some(build_sort_order(order_key, &iceberg_schema)?),
        None => None,
    };

    // Some JNI catalogs extract `format-version` from table properties, while
    // native Rust Glue rejects reserved properties before creating metadata.
    let properties = if matches!(
        config.catalog_kind()?,
        IcebergCatalogKind::Glue(IcebergCatalogRuntime::NativeRust)
    ) {

View on GitHub (pinned to 6469eb736d)