risingwavelabs/risingwave · error · SinkError::Iceberg

failed to convert {}: {} to arrow type

Error message

failed to convert {}: {} to arrow type

What it means

This contextual error reports that one of the sink's columns could not be converted from its RisingWave type to an Arrow field, via IcebergCreateTableArrowConvert::to_arrow_field. The message names the column and its RisingWave data type, with the underlying converter error chained as context.

Source

Thrown at src/connector/src/sink/iceberg/create_table.rs:137

            .columns
            .iter()
            .find(|column| column.data_type.contains_variant())
    {
        return Err(SinkError::Config(anyhow!(
            "creating an Iceberg table with VARIANT column `{}` requires `format_version = '3'`",
            column.name
        )));
    }

    let iceberg_create_table_arrow_convert = IcebergCreateTableArrowConvert::default();
    // convert risingwave schema -> arrow schema -> iceberg schema
    let arrow_fields = param
        .columns
        .iter()
        .map(|column| {
            Ok(iceberg_create_table_arrow_convert
                .to_arrow_field(&column.name, &column.data_type)
                .map_err(|e| SinkError::Iceberg(anyhow!(e)))
                .context(format!(
                    "failed to convert {}: {} to arrow type",
                    column.name, column.data_type
                ))?)
        })
        .collect::<Result<Vec<ArrowField>>>()?;
    let arrow_schema = arrow_schema_iceberg::Schema::new(arrow_fields);
    let iceberg_schema = iceberg::arrow::arrow_schema_to_schema(&arrow_schema)
        .map_err(|e| SinkError::Iceberg(anyhow!(e)))
        .context("failed to convert arrow schema to iceberg schema")?;

    let location = {
        let mut names = namespace.clone().inner();
        names.push(table_name.clone());
        match &config.common.warehouse_path {
            Some(warehouse_path) => {
                let is_s3_tables = warehouse_path.starts_with("arn:aws:s3tables");
                // Lakehouse Iceberg REST catalog federation uses bq:// prefix for BigQuery-managed Iceberg tables.

View on GitHub (pinned to 6469eb736d)

Solutions

1. Read the chained underlying error to identify the offending type.
2. Cast the offending column to a supported type in the source query/mv (e.g. CAST(col AS VARCHAR)).
3. For VARIANT columns, set table.format-version='3'.
4. Remove unsupported columns from the sink definition.
5. Upgrade RisingWave/iceberg-rust if the type should be supported.

Example fix

// before: sinking a jsonb column without v3
CREATE MATERIALIZED VIEW mv AS SELECT payload FROM src; -- payload JSONB
// after
CREATE MATERIALIZED VIEW mv AS SELECT CAST(payload AS VARCHAR) AS payload FROM src;

// Verify all sink column types are convertible before creating the sink
for col in &param.columns {
    if let Err(e) = convert.to_arrow_field(&col.name, &col.data_type) {
        eprintln!("column {} ({}) unsupported: {e}", col.name, col.data_type);
    }
}

match result {
    Err(e) if e.to_string().contains("failed to convert") => {
        // cast the named column to a supported type in the MV
    }
    other => other?,
}

When it happens

Trigger: During create_table_if_not_exists_impl, mapping param.columns through to_arrow_field returns Err for a specific column — typically an unsupported RisingWave type (e.g. VARIANT under format v1/v2, struct/list with unsupported inner types) in the sink definition.

Common situations: Sinking columns whose types have no Iceberg/Arrow equivalent; VARIANT columns without format-version 3; deeply nested struct/list types with unsupported leaf types; type added in RisingWave newer than the iceberg-rust version in use.

Understand the failure class

Background: Type mismatch errors: IllegalArgumentException, TypeError and type guards across 150 open-source libraries — this error's family across 150 libraries.

Related errors

• Failed to convert Arrow type to Iceberg type
• failed to convert arrow schema to iceberg schema
• RecordBatch::try_new failed: {e}
• error converting Iceberg schema to Arrow schema: {err}
• support negative scale for arrow decimal

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/2c442a9c91e797a0. Report an issue: GitHub.