{"record":{"id":"a1c471d0d9899a26","repo":"quickwit-oss/quickwit","slug":"merge-requires-at-least-one-input-file","errorCode":null,"errorMessage":"merge requires at least one input file","messagePattern":"merge requires at least one input file","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"quickwit/quickwit-parquet-engine/src/merge/mod.rs","lineNumber":178,"sourceCode":"/// the multi-RecordBatch concatenation path.\n#[cfg(test)]\npub(crate) fn merge_sorted_parquet_files_with_read_batch_size(\n    input_paths: &[PathBuf],\n    output_dir: &Path,\n    config: &MergeConfig,\n    read_batch_size: usize,\n) -> Result<Vec<MergeOutputFile>> {\n    merge_sorted_parquet_files_impl(input_paths, output_dir, config, Some(read_batch_size))\n}\n\nfn merge_sorted_parquet_files_impl(\n    input_paths: &[PathBuf],\n    output_dir: &Path,\n    config: &MergeConfig,\n    read_batch_size: Option<usize>,\n) -> Result<Vec<MergeOutputFile>> {\n    if input_paths.is_empty() {\n        bail!(\"merge requires at least one input file\");\n    }\n    if config.num_outputs == 0 {\n        bail!(\"num_outputs must be at least 1\");\n    }\n\n    // Step 0: Read and validate metadata from all input files.\n    // Sort schema, window, and merge ops are derived from the files themselves.\n    let input_meta = extract_and_validate_input_metadata(input_paths)?;\n\n    info!(\n        num_inputs = input_paths.len(),\n        num_outputs = config.num_outputs,\n        sort_fields = %input_meta.sort_fields,\n        \"starting sorted parquet merge\"\n    );\n\n    // Step 1: Read all input files into RecordBatches.\n    let inputs = read_inputs(input_paths, read_batch_size)?;","sourceCodeStart":160,"sourceCodeEnd":196,"githubUrl":"https://github.com/quickwit-oss/quickwit/blob/a39730c5cdcd1a4fe798403737ae293999ea21f8/quickwit/quickwit-parquet-engine/src/merge/mod.rs#L160-L196","documentation":"merge_sorted_parquet_files_impl bails immediately if `input_paths` is empty. A merge with zero input files has nothing to produce; the API treats that as a caller bug rather than returning an empty result, since callers are expected to only invoke the merge engine with at least one compaction candidate.","triggerScenarios":"Calling merge_sorted_parquet_files(input_paths, output_dir, config) with an empty slice — e.g. a merge policy that built a task before checking candidate count, or a filter that removed all paths after task creation.","commonSituations":"Race where candidate splits were deleted (retention/expiry) between planning and execution; a merge task persisted in the metastore whose inputs were already merged by another worker; unit tests calling the merge function directly with no fixtures.","solutions":["Guard at the call site: skip the merge entirely if the candidate list is empty instead of invoking the engine.","Re-validate that all planned input paths still exist right before the merge and drop empty task sets.","Check for concurrent merge workers double-claiming tasks and adding appropriate task claiming/locking.","In tests, assert candidate lists are non-empty before constructing merge tasks."],"exampleFix":"// before\nrun_merge_task(task)?; // task.input_paths may be empty\n// after\nif task.input_paths.is_empty() {\n    info!(task_id = %task.id, \"no inputs left, skipping merge\");\n    return Ok(());\n}\nrun_merge_task(task)?;","handlingStrategy":"validation","validationCode":"if input_paths.is_empty() {\n    // skip merge entirely\n    return Ok(());\n}","typeGuard":"fn non_empty<'a, T>(paths: &'a [T]) -> Option<&'a [T]> {\n    (!paths.is_empty()).then_some(paths)\n}","tryCatchPattern":"match merge_sorted_parquet_files(&paths, out_dir, &config) {\n    Err(e) if e.to_string().contains(\"at least one input file\") => {\n        info!(\"merge task became empty (inputs deleted?); skipping\");\n        Ok(())\n    }\n    other => other,\n}","preventionTips":["Check candidate count before creating/persisting a merge task.","Re-verify input paths exist immediately before execution (guard against retention races).","Use task claiming/locking so concurrent workers don't empty a task's inputs."],"tags":["rust","merge","empty-input","validation"],"backgroundTag":"empty-required-field","analyzedSha":"a39730c5cdcd1a4fe798403737ae293999ea21f8","analyzedAt":"2026-09-08T13:19:37.784Z","contentChangedAt":"2026-09-08T13:19:37.784Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}