{"record":{"id":"dc67e29462fe7526","repo":"risingwavelabs/risingwave","slug":"column-type-mismatch-lancedb-type-is-r","errorCode":null,"errorMessage":"column '{}' type mismatch: LanceDB type is {:?}, RisingWave type is {:?}","messagePattern":"column '(.+?)' type mismatch: LanceDB type is (.+?), RisingWave type is (.+?)","errorType":"exception","errorClass":"SinkError::LanceDb","httpStatus":null,"severity":"error","filePath":"src/connector/src/sink/lancedb.rs","lineNumber":315,"sourceCode":"    }\n\n    for (idx, (rw_field, lance_field)) in rw_arrow_schema\n        .fields()\n        .iter()\n        .zip_eq_fast(lance_schema.fields().iter())\n        .enumerate()\n    {\n        if rw_field.name() != lance_field.name() {\n            return Err(SinkError::LanceDb(anyhow!(\n                \"column order mismatch at position {}: LanceDB column is '{}', RisingWave column is '{}'\",\n                idx,\n                lance_field.name(),\n                rw_field.name()\n            )));\n        }\n\n        if rw_field.data_type() != lance_field.data_type() {\n            return Err(SinkError::LanceDb(anyhow!(\n                \"column '{}' type mismatch: LanceDB type is {:?}, RisingWave type is {:?}\",\n                rw_field.name(),\n                lance_field.data_type(),\n                rw_field.data_type()\n            )));\n        }\n    }\n\n    Ok(())\n}\n\n/// The writer writes data files directly to the Lance dataset storage using the\n/// low-level `FileFragment::create_fragments()` API. On checkpoint, it returns\n/// lightweight `Fragment` metadata instead of the actual data payload. A fragment\n/// is a logical row segment that references one or more files containing columns\n/// for those rows. The coordinator then commits these fragments atomically.\n///\n/// This follows the same pattern as the Iceberg sink, where writers handle I/O","sourceCodeStart":297,"sourceCodeEnd":333,"githubUrl":"https://github.com/risingwavelabs/risingwave/blob/6469eb736d691e8e9b8a419a57edd6429ca77417/src/connector/src/sink/lancedb.rs#L297-L333","documentation":"Once names match positionally, each column's Arrow data type must be equal. If the LanceDB column's Arrow type differs from the type RisingWave derived for that column (e.g., Int64 vs Int32, Utf8 vs LargeUtf8), validation fails with both types.","triggerScenarios":"Existing LanceDB table column has a different Arrow type than the RW column (e.g., table created with INTEGER where RW schema says BIGINT; string vs binary; timestamp units differing).","commonSituations":"Table pre-created by another tool with narrower/different types; RW version change altering the Arrow mapping for a type; precision/units mismatch for timestamps.","solutions":["Recreate the table (or a new table) with types matching the RisingWave schema","Cast the RW sink column to the type matching the existing table (e.g., col::int for Int32)","Check rw_schema_to_arrow_schema's mapping and align the table types to it exactly","Verify timestamp units/timezones match between the two schemas"],"exampleFix":"// before\nCREATE SINK s AS SELECT id FROM t WITH (...); -- RW: BIGINT(Int64), table column: Int32\n// after\nCREATE SINK s AS SELECT id::int AS id FROM t WITH (...);","handlingStrategy":"validation","validationCode":"async function checkColumnTypes(tableName, rwArrowSchema) {\n  const schema = await (await conn.openTable(tableName)).schema();\n  schema.fields.forEach((f, i) => {\n    if (!f.type.equals(rwArrowSchema.fields[i].type))\n      throw new Error(`${f.name}: table=${f.type} sink=${rwArrowSchema.fields[i].type}`);\n  });\n}","typeGuard":null,"tryCatchPattern":"catch (SinkError::LanceDb(e)) if e.includes(\"type mismatch\") { cast RW column or recreate table with matching Arrow types }","preventionTips":["Pre-create tables using types derived from rw_schema_to_arrow_schema mappings","Watch for Int32/Int64 and Utf8/LargeUtf8 style near-misses","Re-verify types after RW version upgrades"],"tags":["rust","arrow","type-mismatch","schema"],"backgroundTag":"dtype-mismatch","analyzedSha":"6469eb736d691e8e9b8a419a57edd6429ca77417","analyzedAt":"2026-09-11T21:06:21.487Z","contentChangedAt":"2026-09-11T21:06:21.487Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}