risingwavelabs/risingwave · error · SinkError::LanceDb

column '{}' type mismatch: LanceDB type is {:?}, RisingWave

Error message

column '{}' type mismatch: LanceDB type is {:?}, RisingWave type is {:?}

What it means

Once names match positionally, each column's Arrow data type must be equal. If the LanceDB column's Arrow type differs from the type RisingWave derived for that column (e.g., Int64 vs Int32, Utf8 vs LargeUtf8), validation fails with both types.

Source

Thrown at src/connector/src/sink/lancedb.rs:315

    }

    for (idx, (rw_field, lance_field)) in rw_arrow_schema
        .fields()
        .iter()
        .zip_eq_fast(lance_schema.fields().iter())
        .enumerate()
    {
        if rw_field.name() != lance_field.name() {
            return Err(SinkError::LanceDb(anyhow!(
                "column order mismatch at position {}: LanceDB column is '{}', RisingWave column is '{}'",
                idx,
                lance_field.name(),
                rw_field.name()
            )));
        }

        if rw_field.data_type() != lance_field.data_type() {
            return Err(SinkError::LanceDb(anyhow!(
                "column '{}' type mismatch: LanceDB type is {:?}, RisingWave type is {:?}",
                rw_field.name(),
                lance_field.data_type(),
                rw_field.data_type()
            )));
        }
    }

    Ok(())
}

/// The writer writes data files directly to the Lance dataset storage using the
/// low-level `FileFragment::create_fragments()` API. On checkpoint, it returns
/// lightweight `Fragment` metadata instead of the actual data payload. A fragment
/// is a logical row segment that references one or more files containing columns
/// for those rows. The coordinator then commits these fragments atomically.
///
/// This follows the same pattern as the Iceberg sink, where writers handle I/O

View on GitHub (pinned to 6469eb736d)

Solutions

1. Recreate the table (or a new table) with types matching the RisingWave schema
2. Cast the RW sink column to the type matching the existing table (e.g., col::int for Int32)
3. Check rw_schema_to_arrow_schema's mapping and align the table types to it exactly
4. Verify timestamp units/timezones match between the two schemas

Example fix

// before
CREATE SINK s AS SELECT id FROM t WITH (...); -- RW: BIGINT(Int64), table column: Int32
// after
CREATE SINK s AS SELECT id::int AS id FROM t WITH (...);

async function checkColumnTypes(tableName, rwArrowSchema) {
  const schema = await (await conn.openTable(tableName)).schema();
  schema.fields.forEach((f, i) => {
    if (!f.type.equals(rwArrowSchema.fields[i].type))
      throw new Error(`${f.name}: table=${f.type} sink=${rwArrowSchema.fields[i].type}`);
  });
}

catch (SinkError::LanceDb(e)) if e.includes("type mismatch") { cast RW column or recreate table with matching Arrow types }

When it happens

Trigger: Existing LanceDB table column has a different Arrow type than the RW column (e.g., table created with INTEGER where RW schema says BIGINT; string vs binary; timestamp units differing).

Common situations: Table pre-created by another tool with narrower/different types; RW version change altering the Arrow mapping for a type; precision/units mismatch for timestamps.

Related errors

• Join key types are not aligned: LHS: {outer_type:?}, RHS: {i
• Join key types are not aligned: LHS: {outer_type:?}, RHS: {i
• (schema conversion error)
• Columns mismatch. RisingWave schema has {} fields, LanceDB t
• column order mismatch at position {}: LanceDB column is '{}'

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/dc67e29462fe7526. Report an issue: GitHub.