{"record":{"id":"613941013cd5aa1a","repo":"quickwit-oss/quickwit","slug":"input-is-missing-column","errorCode":null,"errorMessage":"input {} is missing column '{}'","messagePattern":"input (.+?) is missing column '(.+?)'","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"quickwit/quickwit-parquet-engine/src/merge/merge_order.rs","lineNumber":417,"sourceCode":"            }\n\n            transitions_remaining -= 1;\n        }\n    }\n\n    Ok(boundaries)\n}\n\n/// Get a column by name from a RecordBatch, with a clear error message.\nfn get_column(\n    batch: &RecordBatch,\n    name: &str,\n    input_index: usize,\n) -> Result<arrow::array::ArrayRef> {\n    let idx = batch\n        .schema()\n        .index_of(name)\n        .map_err(|_| anyhow::anyhow!(\"input {} is missing column '{}'\", input_index, name))?;\n    Ok(Arc::clone(batch.column(idx)))\n}\n","sourceCodeStart":399,"sourceCodeEnd":420,"githubUrl":"https://github.com/quickwit-oss/quickwit/blob/a39730c5cdcd1a4fe798403737ae293999ea21f8/quickwit/quickwit-parquet-engine/src/merge/merge_order.rs#L399-L420","documentation":"get_column extracts a named column from an arrow RecordBatch and converts a missing-column error from Schema::index_of into this anyhow error, reporting which input batch (by index) lacked the column. Merge order computation requires every input batch to carry the timestamp and sort columns.","triggerScenarios":"compute_merge_order processing a RecordBatch whose schema lacks a column required by the merge order (e.g. the timestamp column or a sort key), typically the batch at the reported input index.","commonSituations":"Mixing parquet files with different schemas (one missing the timestamp field); schema evolution where a field was renamed; partial reads returning batches projected to fewer columns.","solutions":["Inspect the schema of the input file at the reported index and add/restore the missing column.","Ensure all files in the merge set share the same arrow schema — filter out or re-write divergent files.","If using column projection, project all sort/timestamp columns in the read."],"exampleFix":"// before: reading only a subset of columns\nlet batch = reader.read_columns(&[\"service\"])?;\n// after: include required merge columns\nlet batch = reader.read_columns(&[\"service\", \"_timestamp\"])?;","handlingStrategy":"validation","validationCode":"// before merging, verify schema parity across inputs\nlet schema0 = first_batch.schema();\nfor (i, batch) in batches.iter().enumerate() {\n    for name in required_columns {\n        assert!(batch.schema().index_of(name).is_ok(), \"input {} missing {}\", i, name);\n    }\n}","typeGuard":"fn has_column(batch: &RecordBatch, name: &str) -> bool {\n    batch.schema().index_of(name).is_ok()\n}","tryCatchPattern":"match compute_merge_order(&batches) {\n    Err(e) if e.to_string().contains(\"is missing column\") => {\n        // re-read inputs with full projection or drop the offending file\n    }\n    other => other?,\n}","preventionTips":["Project all sort/timestamp columns when reading parquet for merge.","Enforce a single arrow schema across all files in a merge set.","Detect schema drift between files before starting the merge."],"tags":["parquet","arrow","schema","merge"],"backgroundTag":"schema-validation-failed","analyzedSha":"a39730c5cdcd1a4fe798403737ae293999ea21f8","analyzedAt":"2026-09-08T13:19:37.784Z","contentChangedAt":"2026-09-08T13:19:37.784Z","schemaVersion":2},"datasetVersion":"2026-09-14T11:17:12.474Z"}