risingwavelabs/risingwave · error · SinkError::LanceDb

Columns mismatch. RisingWave schema has {} fields, LanceDB t

Error message

Columns mismatch. RisingWave schema has {} fields, LanceDB table has {} fields

What it means

validate_ordered_schema requires the existing LanceDB table's Arrow schema to have exactly the same number of columns as the RisingWave sink schema. When the counts differ, the sink cannot safely map values, so validation fails with the two field counts.

Source

Thrown at src/connector/src/sink/lancedb.rs:292

        LanceDbSink::new(config, param)
    }
}

// ---------------------------------------------------------------------------
// Writer
// ---------------------------------------------------------------------------

// Re-export arrow types from the LanceDb arrow module so they're used consistently.
use risingwave_common::array::arrow::{
    arrow_array_lancedb as arrow_array, arrow_schema_lancedb as arrow_schema,
};

fn validate_ordered_schema(
    rw_arrow_schema: &arrow_schema::Schema,
    lance_schema: &arrow_schema::Schema,
) -> Result<()> {
    if rw_arrow_schema.fields().len() != lance_schema.fields().len() {
        return Err(SinkError::LanceDb(anyhow!(
            "Columns mismatch. RisingWave schema has {} fields, LanceDB table has {} fields",
            rw_arrow_schema.fields().len(),
            lance_schema.fields().len()
        )));
    }

    for (idx, (rw_field, lance_field)) in rw_arrow_schema
        .fields()
        .iter()
        .zip_eq_fast(lance_schema.fields().iter())
        .enumerate()
    {
        if rw_field.name() != lance_field.name() {
            return Err(SinkError::LanceDb(anyhow!(
                "column order mismatch at position {}: LanceDB column is '{}', RisingWave column is '{}'",
                idx,
                lance_field.name(),
                rw_field.name()

View on GitHub (pinned to 6469eb736d)

Solutions

  1. Align the sink query's SELECT list to match the existing table's columns exactly (or vice versa)
  2. Drop and recreate the LanceDB table with the schema RisingWave expects
  3. Create the sink against a fresh table name and let RW create the table
  4. Compare `SELECT * FROM table LIMIT 0` schema with the RW sink schema and reconcile differences

Example fix

// before
CREATE SINK s AS SELECT a, b FROM t WITH ('connector'='lancedb', 'lancedb.table'='t_old'); -- t_old has 3 cols
// after
CREATE SINK s AS SELECT a, b, c FROM t WITH ('connector'='lancedb', 'lancedb.table'='t_old');
Defensive patterns

Strategy: validation

Validate before calling

async function checkColumnCount(tableName, expected) {
  const t = await conn.openTable(tableName);
  const schema = await t.schema();
  if (schema.fields.length !== expected)
    throw new Error(`table ${tableName} has ${schema.fields.length} cols, sink has ${expected}`);
}

Try / catch

catch (SinkError::LanceDb(e)) if e.includes("Columns mismatch") { align the SELECT projection or recreate the table }

Prevention

When it happens

Trigger: CREATE SINK into an existing LanceDB table whose column count differs from the RW schema — e.g., the table was created earlier with extra columns, or RW added/dropped a column since the table was created.

Common situations: Schema evolution drift: someone ALTERed the MV/table on one side only; reusing an old LanceDB table for a new sink with a different projection.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/220da93c31bdf6f7. Report an issue: GitHub.