quickwit-oss/quickwit · error

window_duration_secs mismatch in {}: expected {}, found {}

Error message

window_duration_secs mismatch in {}: expected {}, found {}

What it means

Same consensus check as window_start but for the `window_duration_secs` parquet metadata key. All inputs to a merge must declare the same window duration (missing treated as 0); a differing value aborts the merge because merged output could not carry a coherent window duration.

Source

Thrown at quickwit/quickwit-parquet-engine/src/merge/mod.rs:430

                    );
                }
            }
            None => {
                consensus_window_start = Some(file_window_start);
            }
        }

        // Window duration: must be consistent.
        let file_window_duration = find_kv(PARQUET_META_WINDOW_DURATION)
            .map(|s| s.parse::<u32>())
            .transpose()
            .with_context(|| format!("parsing window_duration from {}", path.display()))?
            .unwrap_or(0);

        match &consensus_window_duration {
            Some(expected) => {
                if file_window_duration != *expected {
                    bail!(
                        "window_duration_secs mismatch in {}: expected {}, found {}",
                        path.display(),
                        expected,
                        file_window_duration
                    );
                }
            }
            None => {
                consensus_window_duration = Some(file_window_duration);
            }
        }

        // Merge ops: take the max across all inputs.
        let file_merge_ops = find_kv(PARQUET_META_NUM_MERGE_OPS)
            .map(|s| s.parse::<u32>())
            .transpose()
            .with_context(|| format!("parsing num_merge_ops from {}", path.display()))?
            .unwrap_or(0);

View on GitHub (pinned to a39730c5cd)

Solutions

  1. Verify all inputs share the same `window_duration_secs` metadata; exclude divergent files from the merge group.
  2. Re-index legacy splits lacking the key so they carry the correct duration.
  3. Fix cluster config so all indexers use the same window duration.
  4. Update the merge planner to partition split candidates by window duration.

Example fix

// before: legacy split without duration merged with new splits
merge(vec![legacy_split, new_split])?;
// after: filter by consistent window_duration_secs first
let expected = splits[0].window_duration_secs;
let compatible: Vec<_> = splits.into_iter().filter(|s| s.window_duration_secs == expected).collect();
merge(compatible)?;
Defensive patterns

Strategy: validation

Validate before calling

fn durations_consistent(files: &[PathBuf]) -> anyhow::Result<()> {
    let mut vals: HashSet<u64> = HashSet::new();
    for f in files { vals.insert(read_parquet_kv(f, "window_duration_secs").unwrap_or(0)); }
    anyhow::ensure!(vals.len() <= 1, "mixed window_duration_secs across merge inputs");
    Ok(())
}

Try / catch

match merge(...).await {
    Err(e) if e.to_string().contains("window_duration_secs mismatch") => {
        // quarantine legacy split, re-index, then retry merge
    }
    other => other?,
}

Prevention

When it happens

Trigger: Calling merge_sorted_parquet_files_impl with input files whose `window_duration_secs` KV metadata differs (including one file missing the key, which defaults to 0).

Common situations: Mixing splits written before windowing metadata was introduced (missing key => 0) with newer splits that carry a real duration; misconfigured windowing on one indexing node in the cluster.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

AI-assisted analysis of quickwit-oss/quickwit@a39730c5cd (2026-09-08). Data as JSON: /api/errors/51916cd290ec97a7. Report an issue: GitHub.