{"record":{"id":"0aaf93ac248d6ced","repo":"influxdata/influxdb","slug":"unexpected-batch-schema-mismatch-expected-columns-got","errorCode":null,"errorMessage":"unexpected batch schema mismatch: expected {} columns, got {}","messagePattern":"unexpected batch schema mismatch: expected (.+?) columns, got (.+?)","errorType":"validation","errorClass":"anyhow::Error","httpStatus":null,"severity":"error","filePath":"influxdb3_py_api/src/py_conversion.rs","lineNumber":39,"sourceCode":"    batches: &[RecordBatch],\n) -> Result<Bound<'py, PyList>, anyhow::Error> {\n    // Pre-create Python strings for field/tag names once for all batches;\n    // schema must be the same across batches.\n    let Some(first_batch) = batches.first() else {\n        return Ok(PyList::empty(py));\n    };\n    let field_names: Vec<Bound<'_, PyString>> = first_batch\n        .schema()\n        .fields()\n        .iter()\n        .map(|f| PyString::new(py, f.name().as_str()))\n        .collect();\n\n    let total_rows: usize = batches.iter().map(|b| b.num_rows()).sum();\n    let mut rows: Vec<Py<PyAny>> = Vec::with_capacity(total_rows);\n\n    for batch in batches {\n        ensure!(\n            batch.num_columns() == field_names.len(),\n            \"unexpected batch schema mismatch: expected {} columns, got {}\",\n            field_names.len(),\n            batch.num_columns()\n        );\n        let num_rows = batch.num_rows();\n        for row_idx in 0..num_rows {\n            let row = PyDict::new(py);\n            for (col_idx, field_name) in field_names.iter().enumerate() {\n                let array = batch.column(col_idx);\n                let value = extract_arrow_value_to_py(py, array, row_idx)?;\n                row.set_item(field_name, value).context(\"set dict item\")?;\n            }\n            rows.push(row.into());\n        }\n    }\n\n    let list = PyList::new(py, rows)?;","sourceCodeStart":21,"sourceCodeEnd":57,"githubUrl":"https://github.com/influxdata/influxdb/blob/06200ef96ba82c5f6727e5038a83af8e722c6875/influxdb3_py_api/src/py_conversion.rs#L21-L57","documentation":"record_batches_to_py_rows converts Arrow RecordBatches to Python rows using a column-name list derived from the first/schema fields. Before processing each batch it verifies the batch's column count matches that field list; a mismatch means the batch's schema diverged from what was expected. This is an internal invariant violation raised via anyhow's ensure! macro.","triggerScenarios":"Calling query() or wal_flush_to_py() when result batches have differing column counts (e.g. schema evolved mid-query, batches from mixed schemas concatenated).","commonSituations":"Writing to a table with altered schema between flushes so WAL batches differ from query schema; mixed-version writers producing inconsistent batch schemas; upstream query engine bugs.","solutions":["Ensure all data written to the table conforms to one schema; fix writers producing mismatched column counts.","Re-check the query/table schema (SHOW/modern schema APIs) and align clients with the current schema.","If batches are legitimately heterogeneous, split them per schema before conversion instead of one call."],"exampleFix":null,"handlingStrategy":"validation","validationCode":"// verify all batches share the expected schema before conversion\nlet expected = batches[0].schema();\nfor b in &batches {\n    assert_eq!(b.num_columns(), expected.fields().len());\n}","typeGuard":null,"tryCatchPattern":"match record_batches_to_py_rows(py, &batches, &field_names) {\n    Ok(rows) => ..., \n    Err(e) => log::error!(\"schema mismatch during row conversion: {e:#}\"),\n}","preventionTips":["Enforce a single table schema across all writers and versions.","Split heterogeneous batches per schema before conversion.","Validate schema after schema migrations before running Python conversions."],"tags":["rust","arrow","schema","internal"],"backgroundTag":"schema-mismatch","analyzedSha":"06200ef96ba82c5f6727e5038a83af8e722c6875","analyzedAt":"2026-09-19T12:55:30.003Z","contentChangedAt":"2026-09-19T12:55:30.003Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}