{"record":{"id":"d8c0fd245360d3c8","repo":"quickwit-oss/quickwit","slug":"window-start-mismatch-in-expected-found","errorCode":null,"errorMessage":"window_start mismatch in {}: expected {:?}, found {:?}","messagePattern":"window_start mismatch in (.+?): expected (.+?), found (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"quickwit/quickwit-parquet-engine/src/merge/mod.rs","lineNumber":407,"sourceCode":"                        \"parsing sort schema from {}: '{}'\",\n                        path.display(),\n                        file_sort_fields\n                    )\n                })?;\n                consensus_sort_fields = Some(file_sort_fields.clone());\n            }\n        }\n\n        // Window start: must be consistent.\n        let file_window_start = find_kv(PARQUET_META_WINDOW_START)\n            .map(|s| s.parse::<i64>())\n            .transpose()\n            .with_context(|| format!(\"parsing window_start from {}\", path.display()))?;\n\n        match &consensus_window_start {\n            Some(expected) => {\n                if file_window_start != *expected {\n                    bail!(\n                        \"window_start mismatch in {}: expected {:?}, found {:?}\",\n                        path.display(),\n                        expected,\n                        file_window_start\n                    );\n                }\n            }\n            None => {\n                consensus_window_start = Some(file_window_start);\n            }\n        }\n\n        // Window duration: must be consistent.\n        let file_window_duration = find_kv(PARQUET_META_WINDOW_DURATION)\n            .map(|s| s.parse::<u32>())\n            .transpose()\n            .with_context(|| format!(\"parsing window_duration from {}\", path.display()))?\n            .unwrap_or(0);","sourceCodeStart":389,"sourceCodeEnd":425,"githubUrl":"https://github.com/quickwit-oss/quickwit/blob/a39730c5cdcd1a4fe798403737ae293999ea21f8/quickwit/quickwit-parquet-engine/src/merge/mod.rs#L389-L425","documentation":"During a parquet merge, extract_and_validate_input_metadata reads the `window_start` metadata key from every input split's parquet file metadata and requires all inputs to agree. The first observed value becomes the consensus; any file whose window_start differs from that consensus aborts the merge. This prevents merging splits that cover different time windows, which would corrupt the merged output's window semantics.","triggerScenarios":"Calling merge_sorted_parquet_files_impl (or execute_merge_operation) with a set of parquet files whose embedded KV metadata `window_start` values are not all identical.","commonSituations":"Merging splits hand-picked from different time windows, mixing splits from an older writer that stamped windows differently, or a scheduling bug in the merge planner that groups non-contiguous windows.","solutions":["Check each input file's parquet KV metadata `window_start` and only group files with identical values.","Fix the merge/placement logic that selected the splits so it partitions by window_start.","If the offending file was produced by a buggy writer, re-index or re-ingest it with correct window metadata.","If merging across windows is intended, use an operation path that does not require window consensus."],"exampleFix":"// before: mixing splits from different windows in one merge op\nlet op = ParquetMergeOperation { splits: vec![split_a_win1, split_b_win2], .. };\n// after: group splits by window_start before merging\nlet mut groups: HashMap<Option<String>, Vec<Split>> = HashMap::new();\nfor s in splits { groups.entry(s.window_start.clone()).or_default().push(s); }\nfor group in groups.values() { merge(group.clone()).await?; }","handlingStrategy":"validation","validationCode":"fn windows_consistent(files: &[PathBuf]) -> anyhow::Result<()> {\n    let mut vals: HashSet<Option<String>> = HashSet::new();\n    for f in files { vals.insert(read_parquet_kv(f, \"window_start\")); }\n    anyhow::ensure!(vals.len() <= 1, \"mixed window_start across merge inputs\");\n    Ok(())\n}","typeGuard":null,"tryCatchPattern":"match merge_sorted_parquet_files_impl(...).await {\n    Err(e) if e.to_string().contains(\"window_start mismatch\") => {\n        // re-group splits by window and reschedule\n    }\n    other => other?,\n}","preventionTips":["Always group merge candidates by window_start before scheduling.","Verify writer versions stamp window metadata consistently across the cluster.","Log window_start at merge-task creation time for debugging."],"tags":["merge","parquet","metadata-validation"],"backgroundTag":"schema-validation-failed","analyzedSha":"a39730c5cdcd1a4fe798403737ae293999ea21f8","analyzedAt":"2026-09-08T13:19:37.784Z","contentChangedAt":"2026-09-08T13:19:37.784Z","schemaVersion":2},"datasetVersion":"2026-09-14T11:17:12.474Z"}