risingwavelabs/risingwave · error · anyhow::Error

Field {} not found in our schema

Error message

Field {} not found in our schema

What it means

During sink-vs-Iceberg-table compatibility validation, `check_compatibility` looks up each Iceberg arrow field's name in the RisingWave schema field map. If the Iceberg table contains a column that does not exist in the RisingWave sink schema, it throws `Field {name} not found in our schema`. The two schemas must reference exactly the same column set.

Source

Thrown at src/connector/src/sink/iceberg/create_table.rs:377

                }
                if !field_is_compatible(rw_type, arrow_field)? {
                    return Ok(false);
                }
            }
            Ok(true)
        }
        (_, left, right) => Ok(left.equals_datatype(right)),
    }
}

fn check_compatibility(
    schema_fields: HashMap<&str, &risingwave_common::types::DataType>,
    fields: &ArrowFields,
) -> anyhow::Result<bool> {
    for arrow_field in fields {
        let our_field_type = schema_fields
            .get(arrow_field.name().as_str())
            .ok_or_else(|| anyhow!("Field {} not found in our schema", arrow_field.name()))?;

        if !field_is_compatible(our_field_type, arrow_field)? {
            let converted_arrow_data_type = IcebergArrowConvert
                .to_arrow_field("", our_field_type)
                .map_err(|e| anyhow!(e))?
                .data_type()
                .clone();
            bail!(
                "field {}'s type is incompatible\nRisingWave converted data type: {}\niceberg's data type: {}",
                arrow_field.name(),
                converted_arrow_data_type,
                arrow_field.data_type()
            );
        }
    }
    Ok(true)
}

View on GitHub (pinned to 6469eb736d)

Solutions

1. Align the sink query with the Iceberg table: SELECT exactly the same columns (names and set) as the table has.
2. If the table has evolved, alter the sink (recreate) to include the new columns in matching order.
3. Check for case mismatches (e.g., table column `Id` vs sink `id`) and quote/normalize names.

Example fix

// before: table has (a, b, c), sink only selects (a, b)
CREATE SINK s AS SELECT a, b FROM t;
// after
CREATE SINK s AS SELECT a, b, c FROM t;

// verify sink columns equal the Iceberg table columns before creating
let sink_cols: HashSet<&str> = rw_schema.fields.iter().map(|f| f.name.as_str()).collect();
let table_cols: HashSet<&str> = arrow_schema.fields().iter().map(|f| f.name()).collect();
if sink_cols != table_cols {
    return Err(format!("column set mismatch: sink-only={:?} table-only={:?}",
        sink_cols - table_cols, table_cols - sink_cols));
}

When it happens

Trigger: Called from `try_matches_arrow_schema` when validating an existing Iceberg table's arrow schema against the sink's RW schema: the Iceberg table has a column name absent from the RW schema (extra column in the table, or the sink selects a subset of columns).

Common situations: Iceberg table was created/altered externally with extra columns; sink SQL selects fewer/differently-named columns than the table; schema evolution added a column after sink creation; case-sensitivity differences between catalogs.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

• field {}'s type is incompatible RisingWave converted data ty
• Schema length mismatch, risingwave is {}, and iceberg is {}
• Column order mismatch at position {}: the sink has column `{
• {} unsupported in {} catalog
• Can't find schema by id {}

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/8780ad5ebc7e35a4. Report an issue: GitHub.