quickwit-oss/quickwit · error

merge requires at least one input

Error message

merge requires at least one input

What it means

streaming_merge_sorted_parquet_files requires at least one input column-page stream; an empty inputs vector cannot produce a merged file and is rejected immediately. Callers should decide themselves what an empty merge means (typically no output).

Source

Thrown at quickwit/quickwit-parquet-engine/src/merge/streaming.rs:154

}

#[cfg(not(any(test, feature = "testsuite")))]
pub(crate) fn record_body_col_page_cache_len(_len: usize) {}

/// Streaming N-input → M-output column-major merge.
///
/// See module docs for the four phases. Returns one
/// [`MergeOutputFile`] per output file produced (zero-row outputs are
/// dropped). Caller's `config.num_outputs` is the upper bound on the
/// number of files; fewer are returned when there are not enough
/// `sorted_series` transitions to split at.
pub async fn streaming_merge_sorted_parquet_files(
    inputs: Vec<Box<dyn ColumnPageStream>>,
    output_dir: &Path,
    config: &MergeConfig,
) -> Result<Vec<MergeOutputFile>> {
    if inputs.is_empty() {
        bail!("merge requires at least one input");
    }
    if config.num_outputs == 0 {
        bail!("num_outputs must be at least 1");
    }

    let input_meta = extract_and_validate_input_metadata(&inputs)?;

    // Reject legacy multi-RG inputs (`rg_partition_prefix_len == 0`
    // AND any input has >1 row group). These have no alignment claim,
    // so RG boundaries are arbitrary row counts that may split a
    // single sort-key value across two RGs. The streaming engine
    // cannot determine merge regions without column-chunk-bounded
    // buffering; such inputs must go through `LegacyInputAdapter`
    // (from PR-5, see `storage::legacy_adapter`), which presents
    // them as one synthetic single-RG stream.
    //
    // This guard catches caller bugs — production code always routes
    // legacy splits through the adapter (see `merge::execute_merge_operation`

View on GitHub (pinned to a39730c5cd)

Solutions

1. Check inputs.is_empty() before calling and skip the merge (emit no output) in that case.
2. Fix the stream-building stage so every selected split yields exactly one stream.
3. If an empty merge should be an error in your workflow, surface a clearer error upstream naming the empty split set.

Example fix

// before
let outputs = streaming_merge_sorted_parquet_files(streams, &out, &cfg).await?;
// after
if streams.is_empty() {
    return Ok(Vec::new()); // nothing to merge
}
let outputs = streaming_merge_sorted_parquet_files(streams, &out, &cfg).await?;

if streams.is_empty() { return Ok(Vec::new()); } // nothing to merge

match streaming_merge_sorted_parquet_files(streams, &out, &cfg).await {
    Err(e) if e.to_string() == "merge requires at least one input" => Ok(Vec::new()),
    other => other,
}

When it happens

Trigger: Calling streaming_merge_sorted_parquet_files with an empty Vec<Box<dyn ColumnPageStream>> — e.g. no parquet files to merge, or all inputs filtered out before building streams.

Common situations: Merge tasks scheduled on an empty split set; callers opening streams lazily and skipping files that failed or were deleted, ending up with zero streams.

Understand the failure class

Background: "must not be empty", "cannot be empty" — required-field validation errors across open-source libraries — this error's family across 41 libraries.

Related errors

• merge_parquet_split_metadata requires at least one input spl
• merge requires at least one input file
• timestamp_secs must be UInt64 or Int64 for MC-3 check
• attempted to merge splits with different doc mapping uid
• input {} has kind {:?}, expected {:?}

AI-assisted analysis of quickwit-oss/quickwit@a39730c5cd (2026-09-08). Data as JSON: /api/errors/00604cbf6dd8d5a5. Report an issue: GitHub.