type conflict for column '{}': input 0 has {:?}, input {} ha

Error message

type conflict for column '{}': input 0 has {:?}, input {} has {:?} (normalized: {:?} vs {:?})

What it means

While building the union schema, each column's normalized data type is tracked across all input batches. If two inputs contain the same column name with different (post-normalization) types, the batches cannot be combined into one union column, so the alignment bails naming the conflicting column, the input indices, and both types.

Source

Thrown at quickwit/quickwit-parquet-engine/src/merge/schema.rs:85

    // The previous version always defaulted new fields to nullable on
    // first sight, which broke columns whose nullability must be
    // preserved (e.g. `List<Float64>` — the writer's non-nullable-
    // list contract requires the union field to stay non-nullable).
    struct FieldInfo {
        normalized_type: DataType,
        any_nullable: bool,
        appears_in: usize,
    }
    let mut field_map: BTreeMap<String, FieldInfo> = BTreeMap::new();

    for (input_idx, batch) in inputs.iter().enumerate() {
        for field in batch.schema().fields() {
            let normalized_type = normalize_type(field.data_type());

            match field_map.get_mut(field.name().as_str()) {
                Some(existing) => {
                    if existing.normalized_type != normalized_type {
                        bail!(
                            "type conflict for column '{}': input 0 has {:?}, input {} has {:?} \
                             (normalized: {:?} vs {:?})",
                            field.name(),
                            existing.normalized_type,
                            input_idx,
                            field.data_type(),
                            existing.normalized_type,
                            normalized_type,
                        );
                    }
                    if field.is_nullable() {
                        existing.any_nullable = true;
                    }
                    existing.appears_in += 1;
                }
                None => {
                    field_map.insert(
                        field.name().clone(),

View on GitHub (pinned to a39730c5cd)

Solutions

Find which splits contain the divergent column type and re-index them so the field has a consistent type.
Update the index config / doc mapper so the field type matches the data actually ingested.
Extend normalize_type if the conflicting types should be considered compatible (e.g. widening numerics).
Reject or quarantine documents producing the off-type values at ingestion time.

Example fix

// before: field 'status' changed Int64 -> Utf8 between writer versions; merge fails
// after: pin the field type in the doc mapper
// quickwit.yaml index config
// field_mappings:
//   - name: status
//     type: u64   # keep stable across writer versions

Defensive patterns

Strategy: validation

Validate before calling

fn schemas_compatible(schemas: &[SchemaRef]) -> anyhow::Result<()> {
    let mut seen: HashMap<String, DataType> = HashMap::new();
    for s in schemas {
        for f in s.fields() {
            let dt = normalize_type(f.data_type());
            match seen.insert(f.name().clone(), dt.clone()) {
                Some(prev) if prev != dt => anyhow::bail!("column '{}' type conflict: {:?} vs {:?}", f.name(), prev, dt),
                _ => {}
            }
        }
    }
    Ok(())
}

Try / catch

match align_inputs_to_union_schema(&batches, &sort) {
    Err(e) if e.to_string().contains("type conflict for column") => {
        // identify the offending splits, re-index them, then retry
    }
    other => other.map(|_| ()),
}

Prevention

Pin field types in the doc mapper and never change them without a migration.
Validate document field types at ingestion time.
Before merging across writer versions, compare split schemas.

When it happens

Trigger: Calling align_inputs_to_union_schema where input batches disagree on a column's arrow type (e.g. Int64 vs Utf8 for the same field name) after normalize_type.

Common situations: Splits written with different schema versions (a field type changed between releases); documents with inconsistent types ingested into the same index; JSON ingestion coercing values differently per batch.

Understand the failure class

Background: Type mismatch errors: IllegalArgumentException, TypeError and type guards across 150 open-source libraries — this error's family across 150 libraries.

Related errors

AI-assisted analysis of quickwit-oss/quickwit@a39730c5cd (2026-09-08). Data as JSON: /api/errors/94a20430adb69d8a. Report an issue: GitHub.