{"record":{"id":"8e494b8560c921dc","repo":"risingwavelabs/risingwave","slug":"recordbatch-try-new-failed-e","errorCode":null,"errorMessage":"RecordBatch::try_new failed: {e}","messagePattern":"RecordBatch::try_new failed: (.+?)","errorType":"exception","errorClass":"anyhow::Error","httpStatus":null,"severity":"error","filePath":"src/connector/src/sink/iceberg/position_delete.rs","lineNumber":294,"sourceCode":"        .build()\n        .context(\"Failed to build position-delete file metadata\")\n}\n\n/// Writes one chunk of `positions` as a `(file_path, pos)` batch into `writer`. Every row shares\n/// `data_file_path` because the delete file is file-scoped.\nasync fn write_position_delete_chunk(\n    writer: &mut ParquetWriter,\n    arrow_schema: &ArrowSchemaRef,\n    data_file_path: &str,\n    positions: Vec<i64>,\n) -> Result<()> {\n    let path_column: ArrayRef = Arc::new(StringArray::from_iter_values(std::iter::repeat_n(\n        data_file_path,\n        positions.len(),\n    )));\n    let pos_column: ArrayRef = Arc::new(Int64Array::from(positions));\n    let batch = RecordBatch::try_new(arrow_schema.clone(), vec![path_column, pos_column])\n        .map_err(|e| anyhow!(e))?;\n    writer.write(&batch).await?;\n    Ok(())\n}\n","sourceCodeStart":276,"sourceCodeEnd":298,"githubUrl":"https://github.com/risingwavelabs/risingwave/blob/6469eb736d691e8e9b8a419a57edd6429ca77417/src/connector/src/sink/iceberg/position_delete.rs#L276-L298","documentation":"Arrow's `RecordBatch::try_new` validates that every provided column array matches the given schema (field count, types, nullability, length equality). When the path StringArray or pos Int64Array disagrees with the arrow schema's declared fields, try_new returns an ArrowError which this code rethrows as an anyhow error prefixed with 'RecordBatch::try_new failed'.","triggerScenarios":"Calling write_position_delete_chunk with an arrow_schema whose fields are not exactly [String (file_path), Int64 (pos)] — e.g. after an iceberg-rs upgrade changed PositionDeleteWriterBuilder's expected schema — or passing an empty positions vec to a schema marked non-nullable.","commonSituations":"Version mismatch between the RisingWave connector and the iceberg-rs crate where the position-delete schema ordering/types changed; a refactor accidentally swapping the two columns; schema generated from a different source than the arrays.","solutions":["Log/print the arrow_schema fields and the array data types at failure; confirm the schema is exactly [Utf8, Int64] in that order","Ensure the schema used to build the writer (from PositionDeleteWriterBuilder) is the same Arc<arrow_schema> passed here rather than a locally constructed one","Pin/upgrade iceberg-rs and RisingWave together so both agree on the position-delete record format","Build the batch with RecordBatch::try_new_with_options only after asserting arrays.len() == schema.fields().len() and matching data_type() per column"],"exampleFix":"// before\nlet batch = RecordBatch::try_new(arrow_schema.clone(), vec![path_column, pos_column])\n    .map_err(|e| anyhow!(e))?;\n// after\nlet batch = RecordBatch::try_new(\n    arrow_schema.clone(),\n    vec![Arc::clone(&path_column), Arc::clone(&pos_column)],\n)\n.inspect_err(|e| tracing::error!(?e, schema = ?arrow_schema, \"position-delete batch build failed\"))\n.map_err(|e| anyhow!(e))?;","handlingStrategy":"validation","validationCode":"assert_eq!(arrow_schema.fields().len(), 2);\nassert_eq!(arrow_schema.field(0).data_type(), &arrow::datatypes::DataType::Utf8);\nassert_eq!(arrow_schema.field(1).data_type(), &arrow::datatypes::DataType::Int64);","typeGuard":"fn batch_matches(schema: &arrow::datatypes::SchemaRef, cols: &[ArrayRef]) -> bool {\n    schema.fields().len() == cols.len()\n        && schema.fields().iter().zip(cols).all(|(f, a)| f.data_type() == a.data_type())\n        && cols.iter().all(|a| a.len() == cols[0].len())\n}","tryCatchPattern":"let batch = RecordBatch::try_new(arrow_schema.clone(), vec![path_column, pos_column])\n    .map_err(|e| anyhow!(\"RecordBatch::try_new failed: {e}; schema = {:?}\", arrow_schema))?;","preventionTips":["Always derive the arrow schema from the same builder that created the writer, never construct it inline twice","Check array lengths equal each other and the number of rows expected","Re-verify column order/types after any iceberg-rs or arrow upgrade","Log the schema on failure for fast diagnosis"],"tags":["rust","arrow","iceberg","schema-mismatch"],"backgroundTag":"schema-validation-failed","analyzedSha":"6469eb736d691e8e9b8a419a57edd6429ca77417","analyzedAt":"2026-09-11T21:06:21.487Z","contentChangedAt":"2026-09-11T21:06:21.487Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}