quickwit-oss/quickwit · error

input {} has sort_fields '{}', expected '{}'

Error message

input {} has sort_fields '{}', expected '{}'

What it means

merge_parquet_split_metadata requires all input splits to carry identical `sort_fields` (the sort schema string). This bail fires when input i's sort_fields serialization differs from inputs[0]'s. The merged output inherits one sort schema; mixing sorted orders (different fields or directions) would make the output's declared sort key meaningless and break zonemap/sorted-search assumptions.

Source

Thrown at quickwit/quickwit-parquet-engine/src/merge/metadata_aggregation.rs:89

        }
        if input.index_uid != first.index_uid {
            bail!(
                "input {} has index_uid '{}', expected '{}'",
                i,
                input.index_uid,
                first.index_uid
            );
        }
        if input.partition_id != first.partition_id {
            bail!(
                "input {} has partition_id {}, expected {}",
                i,
                input.partition_id,
                first.partition_id
            );
        }
        if input.sort_fields != first.sort_fields {
            bail!(
                "input {} has sort_fields '{}', expected '{}'",
                i,
                input.sort_fields,
                first.sort_fields
            );
        }
        if input.window != first.window {
            bail!(
                "input {} has window {:?}, expected {:?}",
                i,
                input.window,
                first.window
            );
        }
        if !mixed_prefix_ok && input.rg_partition_prefix_len != first.rg_partition_prefix_len {
            bail!(
                "input {} has rg_partition_prefix_len {}, expected {} — splits with different \
                 prefix lengths must not appear in the same regular merge (legacy-promotion \

View on GitHub (pinned to a39730c5cd)

Solutions

  1. Filter merge candidates by exact sort_fields match before grouping into merge tasks.
  2. Force a 'sort rebuild' path (or delete/reindex) for splits written under the old sort configuration instead of compacting them with new ones.
  3. If the mismatch is only formatting, normalize sort_fields serialization on write so equivalent schemas compare equal (see equivalent_schemas_for_compaction used by the merge engine).
  4. Check recent index-config changes that explain which splits carry which sort_fields.

Example fix

// before
let candidates: Vec<_> = splits.into_iter().filter(|s| s.index_uid == uid).collect();
// after
let candidates: Vec<_> = splits.into_iter()
    .filter(|s| s.index_uid == uid && s.sort_fields == expected_sort_fields)
    .collect();
Defensive patterns

Strategy: validation

Validate before calling

fn sort_fields_consistent(inputs: &[ParquetSplitMetadata]) -> bool {
    inputs.iter().all(|s| s.sort_fields == inputs[0].sort_fields)
}

Type guard

fn homogeneous_sort_fields(inputs: &[ParquetSplitMetadata]) -> Option<&str> {
    let sf = &inputs.first()?.sort_fields;
    inputs.iter().all(|s| &s.sort_fields == sf).then_some(sf.as_str())
}

Try / catch

match merge_parquet_split_metadata(&inputs, &output, mixed) {
    Err(e) if e.to_string().contains("sort_fields") => {
        warn!("sort-schema drift in merge batch: {e:#}; scheduling rebuild for old splits");
        schedule_sort_rebuild(&inputs);
    }
    other => other?,
}

Prevention

When it happens

Trigger: Calling merge_parquet_split_metadata with splits whose sort_fields differ — e.g. the index config's sort fields changed between the times different splits were created, and a compaction sweep now picks up splits from both eras.

Common situations: User edits sort configuration of an existing index; a schema-migration rolled out mid-lifecycle leaving old splits with the old sort key; timezone or field-name formatting differences producing string-unequal but logically similar sort fields (this function compares strings/structs exactly).

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

AI-assisted analysis of quickwit-oss/quickwit@a39730c5cd (2026-09-08). Data as JSON: /api/errors/9b20bec51c33547b. Report an issue: GitHub.