{"record":{"id":"94a20430adb69d8a","repo":"quickwit-oss/quickwit","slug":"type-conflict-for-column-input-0-has-i","errorCode":null,"errorMessage":"type conflict for column '{}': input 0 has {:?}, input {} has {:?} (normalized: {:?} vs {:?})","messagePattern":"type conflict for column '(.+?)': input 0 has (.+?), input (.+?) has (.+?) \\(normalized: (.+?) vs (.+?)\\)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"quickwit/quickwit-parquet-engine/src/merge/schema.rs","lineNumber":85,"sourceCode":"    // The previous version always defaulted new fields to nullable on\n    // first sight, which broke columns whose nullability must be\n    // preserved (e.g. `List<Float64>` — the writer's non-nullable-\n    // list contract requires the union field to stay non-nullable).\n    struct FieldInfo {\n        normalized_type: DataType,\n        any_nullable: bool,\n        appears_in: usize,\n    }\n    let mut field_map: BTreeMap<String, FieldInfo> = BTreeMap::new();\n\n    for (input_idx, batch) in inputs.iter().enumerate() {\n        for field in batch.schema().fields() {\n            let normalized_type = normalize_type(field.data_type());\n\n            match field_map.get_mut(field.name().as_str()) {\n                Some(existing) => {\n                    if existing.normalized_type != normalized_type {\n                        bail!(\n                            \"type conflict for column '{}': input 0 has {:?}, input {} has {:?} \\\n                             (normalized: {:?} vs {:?})\",\n                            field.name(),\n                            existing.normalized_type,\n                            input_idx,\n                            field.data_type(),\n                            existing.normalized_type,\n                            normalized_type,\n                        );\n                    }\n                    if field.is_nullable() {\n                        existing.any_nullable = true;\n                    }\n                    existing.appears_in += 1;\n                }\n                None => {\n                    field_map.insert(\n                        field.name().clone(),","sourceCodeStart":67,"sourceCodeEnd":103,"githubUrl":"https://github.com/quickwit-oss/quickwit/blob/a39730c5cdcd1a4fe798403737ae293999ea21f8/quickwit/quickwit-parquet-engine/src/merge/schema.rs#L67-L103","documentation":"While building the union schema, each column's normalized data type is tracked across all input batches. If two inputs contain the same column name with different (post-normalization) types, the batches cannot be combined into one union column, so the alignment bails naming the conflicting column, the input indices, and both types.","triggerScenarios":"Calling align_inputs_to_union_schema where input batches disagree on a column's arrow type (e.g. Int64 vs Utf8 for the same field name) after normalize_type.","commonSituations":"Splits written with different schema versions (a field type changed between releases); documents with inconsistent types ingested into the same index; JSON ingestion coercing values differently per batch.","solutions":["Find which splits contain the divergent column type and re-index them so the field has a consistent type.","Update the index config / doc mapper so the field type matches the data actually ingested.","Extend normalize_type if the conflicting types should be considered compatible (e.g. widening numerics).","Reject or quarantine documents producing the off-type values at ingestion time."],"exampleFix":"// before: field 'status' changed Int64 -> Utf8 between writer versions; merge fails\n// after: pin the field type in the doc mapper\n// quickwit.yaml index config\n// field_mappings:\n//   - name: status\n//     type: u64   # keep stable across writer versions","handlingStrategy":"validation","validationCode":"fn schemas_compatible(schemas: &[SchemaRef]) -> anyhow::Result<()> {\n    let mut seen: HashMap<String, DataType> = HashMap::new();\n    for s in schemas {\n        for f in s.fields() {\n            let dt = normalize_type(f.data_type());\n            match seen.insert(f.name().clone(), dt.clone()) {\n                Some(prev) if prev != dt => anyhow::bail!(\"column '{}' type conflict: {:?} vs {:?}\", f.name(), prev, dt),\n                _ => {}\n            }\n        }\n    }\n    Ok(())\n}","typeGuard":null,"tryCatchPattern":"match align_inputs_to_union_schema(&batches, &sort) {\n    Err(e) if e.to_string().contains(\"type conflict for column\") => {\n        // identify the offending splits, re-index them, then retry\n    }\n    other => other.map(|_| ()),\n}","preventionTips":["Pin field types in the doc mapper and never change them without a migration.","Validate document field types at ingestion time.","Before merging across writer versions, compare split schemas."],"tags":["arrow","schema","type-conflict"],"backgroundTag":"type-mismatch","analyzedSha":"a39730c5cdcd1a4fe798403737ae293999ea21f8","analyzedAt":"2026-09-08T13:19:37.784Z","contentChangedAt":"2026-09-08T13:19:37.784Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}