risingwavelabs/risingwave · error · SinkError::Iceberg

failed to convert : to arrow type

Error message

failed to convert {}: {} to arrow type

What it means

This contextual error reports that one of the sink's columns could not be converted from its RisingWave type to an Arrow field, via IcebergCreateTableArrowConvert::to_arrow_field. The message names the column and its RisingWave data type, with the underlying converter error chained as context.

Solutions

1. Read the chained underlying error to identify the offending type.
2. Cast the offending column to a supported type in the source query/mv (e.g. CAST(col AS VARCHAR)).
3. For VARIANT columns, set table.format-version='3'.
4. Remove unsupported columns from the sink definition.
5. Upgrade RisingWave/iceberg-rust if the type should be supported.

Example fix

// before: sinking a jsonb column without v3
CREATE MATERIALIZED VIEW mv AS SELECT payload FROM src; -- payload JSONB
// after
CREATE MATERIALIZED VIEW mv AS SELECT CAST(payload AS VARCHAR) AS payload FROM src;

// Verify all sink column types are convertible before creating the sink
for col in &param.columns {
    if let Err(e) = convert.to_arrow_field(&col.name, &col.data_type) {
        eprintln!("column {} ({}) unsupported: {e}", col.name, col.data_type);
    }
}

match result {
    Err(e) if e.to_string().contains("failed to convert") => {
        // cast the named column to a supported type in the MV
    }
    other => other?,
}

When it happens

Trigger: During create_table_if_not_exists_impl, mapping param.columns through to_arrow_field returns Err for a specific column — typically an unsupported RisingWave type (e.g. VARIANT under format v1/v2, struct/list with unsupported inner types) in the sink definition.

Common situations: Sinking columns whose types have no Iceberg/Arrow equivalent; VARIANT columns without format-version 3; deeply nested struct/list types with unsupported leaf types; type added in RisingWave newer than the iceberg-rust version in use.

Understand the failure class

Background: Type mismatch errors: IllegalArgumentException, TypeError and type guards across 150 open-source libraries — this error's family across 150 libraries.

Related errors

• error converting Iceberg schema to Arrow schema
• failed to convert arrow schema to iceberg schema
• Failed to convert Arrow type to Iceberg type
• RecordBatch::try_new failed
• bounded compaction branch

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/2c442a9c91e797a0. Report an issue: GitHub.

Appendix: source

Thrown at src/connector/src/sink/iceberg/create_table.rs:137

            .columns
            .iter()
            .find(|column| column.data_type.contains_variant())
    {
        return Err(SinkError::Config(anyhow!(
            "creating an Iceberg table with VARIANT column `{}` requires `format_version = '3'`",
            column.name
        )));
    }

    let iceberg_create_table_arrow_convert = IcebergCreateTableArrowConvert::default();
    // convert risingwave schema -> arrow schema -> iceberg schema
    let arrow_fields = param
        .columns
        .iter()
        .map(|column| {
            Ok(iceberg_create_table_arrow_convert
                .to_arrow_field(&column.name, &column.data_type)
                .map_err(|e| SinkError::Iceberg(anyhow!(e)))
                .context(format!(
                    "failed to convert {}: {} to arrow type",
                    column.name, column.data_type
                ))?)
        })
        .collect::<Result<Vec<ArrowField>>>()?;
    let arrow_schema = arrow_schema_iceberg::Schema::new(arrow_fields);
    let iceberg_schema = iceberg::arrow::arrow_schema_to_schema(&arrow_schema)
        .map_err(|e| SinkError::Iceberg(anyhow!(e)))
        .context("failed to convert arrow schema to iceberg schema")?;

    let location = {
        let mut names = namespace.clone().inner();
        names.push(table_name.clone());
        match &config.common.warehouse_path {
            Some(warehouse_path) => {
                let is_s3_tables = warehouse_path.starts_with("arn:aws:s3tables");
                // Lakehouse Iceberg REST catalog federation uses bq:// prefix for BigQuery-managed Iceberg tables.

View on GitHub (pinned to 6469eb736d)