quickwit-oss/quickwit · error

input {} has kind {:?}, expected {:?}

Error message

input {} has kind {:?}, expected {:?}

What it means

merge_parquet_split_metadata validates that every input ParquetSplitMetadata beyond the first agrees with the first input on invariant fields. This bail fires when an input's `kind` (split kind enum) differs from `inputs[0].kind`. The invariant exists because compaction groups splits by scope (MP-3): a merged output can only inherit one kind, so mixing kinds would produce an ill-defined metastore record.

Source

Thrown at quickwit/quickwit-parquet-engine/src/merge/metadata_aggregation.rs:65

/// skipped because inputs come from different prefix buckets — the
/// output's prefix_len is taken from the writer's KV stamp via
/// `output.output_rg_partition_prefix_len` (CS-1), so the input-side
/// equality is no longer load-bearing for the metastore record.
pub fn merge_parquet_split_metadata(
    inputs: &[ParquetSplitMetadata],
    output: &MergeOutputFile,
    mixed_prefix_ok: bool,
) -> Result<ParquetSplitMetadata> {
    if inputs.is_empty() {
        bail!("merge_parquet_split_metadata requires at least one input split");
    }

    let first = &inputs[0];

    // Validate invariant fields: all inputs must agree on these.
    for (i, input) in inputs.iter().enumerate().skip(1) {
        if input.kind != first.kind {
            bail!(
                "input {} has kind {:?}, expected {:?}",
                i,
                input.kind,
                first.kind
            );
        }
        if input.index_uid != first.index_uid {
            bail!(
                "input {} has index_uid '{}', expected '{}'",
                i,
                input.index_uid,
                first.index_uid
            );
        }
        if input.partition_id != first.partition_id {
            bail!(
                "input {} has partition_id {}, expected {}",
                i,

View on GitHub (pinned to a39730c5cd)

Solutions

  1. Fix the upstream grouping so all splits selected for one merge share the same `kind` (include kind in the merge-scope/grouping key in ParquetMergeExecutor).
  2. Log which splits were selected for the merge and compare their kind values to locate the off-by-one or stale-selection source.
  3. In tests, build all input ParquetSplitMetadata with a shared helper that sets a single consistent kind.
  4. If kinds legitimately differ, run them as separate merge operations instead of one.

Example fix

// before
let inputs = vec![created_split, merged_split];
merge_parquet_split_metadata(&inputs, &output, false)?;
// after
let inputs = vec![created_split, created_split_2]; // same kind per merge
merge_parquet_split_metadata(&inputs, &output, false)?;
Defensive patterns

Strategy: validation

Validate before calling

fn kinds_consistent(inputs: &[ParquetSplitMetadata]) -> bool {
    inputs.iter().all(|s| s.kind == inputs[0].kind)
}
if !kinds_consistent(&inputs) { /* skip or split the merge task */ }

Type guard

fn all_same_kind(inputs: &[ParquetSplitMetadata]) -> Option<Kind> {
    let first = inputs.first()?.kind;
    inputs.iter().all(|s| s.kind == first).then_some(first)
}

Try / catch

match merge_parquet_split_metadata(&inputs, &output, mixed_prefix_ok) {
    Err(e) if e.to_string().contains("has kind") => {
        warn!("mixed-kind merge task rejected: {e:#}; splitting task");
        // re-group inputs by kind and retry each group
    }
    other => other?,
}

Prevention

When it happens

Trigger: Calling merge_parquet_split_metadata(inputs, output, mixed_prefix_ok) with a slice whose first element has, say, kind Kind::Created, but a later element has Kind::Merged or another variant. Typically caused by a bug in the merge-scope grouping logic (e.g. grouping by index/partition but forgetting kind), or by test fixtures hand-constructing split metadata with inconsistent kinds.

Common situations: A merge planner bug selects splits across kinds; a new split kind was added and the grouping key in ParquetMergeExecutor wasn't updated; hand-written test inputs where the first split's kind differs from the rest; metastore records written by an older version whose kind field changed.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

AI-assisted analysis of quickwit-oss/quickwit@a39730c5cd (2026-09-08). Data as JSON: /api/errors/8fdbf4874a3ebe8f. Report an issue: GitHub.