quickwit-oss/quickwit · error

num_outputs must be at least 1

Error message

num_outputs must be at least 1

What it means

merge_sorted_parquet_files_impl validates that MergeConfig.num_outputs is at least 1. num_outputs controls how many output files the merge produces by splitting at sorted_series transitions; a value of 0 would make boundary computation and the writer produce no output for non-empty input, so it is rejected up front as an invalid configuration.

Source

Thrown at quickwit/quickwit-parquet-engine/src/merge/mod.rs:181

    input_paths: &[PathBuf],
    output_dir: &Path,
    config: &MergeConfig,
    read_batch_size: usize,
) -> Result<Vec<MergeOutputFile>> {
    merge_sorted_parquet_files_impl(input_paths, output_dir, config, Some(read_batch_size))
}

fn merge_sorted_parquet_files_impl(
    input_paths: &[PathBuf],
    output_dir: &Path,
    config: &MergeConfig,
    read_batch_size: Option<usize>,
) -> Result<Vec<MergeOutputFile>> {
    if input_paths.is_empty() {
        bail!("merge requires at least one input file");
    }
    if config.num_outputs == 0 {
        bail!("num_outputs must be at least 1");
    }

    // Step 0: Read and validate metadata from all input files.
    // Sort schema, window, and merge ops are derived from the files themselves.
    let input_meta = extract_and_validate_input_metadata(input_paths)?;

    info!(
        num_inputs = input_paths.len(),
        num_outputs = config.num_outputs,
        sort_fields = %input_meta.sort_fields,
        "starting sorted parquet merge"
    );

    // Step 1: Read all input files into RecordBatches.
    let inputs = read_inputs(input_paths, read_batch_size)?;
    let total_rows: usize = inputs.iter().map(|b| b.num_rows()).sum();

    if total_rows == 0 {

View on GitHub (pinned to a39730c5cd)

Solutions

1. Clamp the computed value before calling: `config.num_outputs = config.num_outputs.max(1)` (compute as `.max(1)` after division).
2. Fix the derivation formula to use saturating/ceiling logic so tiny inputs still yield 1 output.
3. Validate MergeConfig at construction/deserialization time (reject num_outputs = 0 there with a clearer message).
4. Check the config source for a literal 0 or an unset field defaulting to 0.

Example fix

// before
let num_outputs = total_input_bytes / target_bytes_per_output;
let config = MergeConfig { num_outputs, ..base };
// after
let num_outputs = (total_input_bytes / target_bytes_per_output).max(1);
let config = MergeConfig { num_outputs, ..base };

if config.num_outputs == 0 {
    return Err(anyhow::anyhow!("num_outputs must be >= 1; got {}", config.num_outputs));
}

fn valid_merge_config(config: &MergeConfig) -> bool {
    config.num_outputs >= 1
}

match merge_sorted_parquet_files(&paths, out_dir, &config) {
    Err(e) if e.to_string().contains("num_outputs must be at least 1") => {
        warn!("num_outputs=0, retrying with 1");
        merge_sorted_parquet_files(&paths, out_dir, &MergeConfig { num_outputs: 1, ..config })
    }
    other => other,
}

When it happens

Trigger: Calling merge_sorted_parquet_files with a MergeConfig where num_outputs == 0 — e.g. num_outputs computed as `total_bytes / target_bytes_per_output` with integer division rounding to 0 for very small inputs, or a misconfigured/zero-valued merge config field.

Common situations: Integer-division rounding when deriving num_outputs from input size vs. a large target file size; config file with num_outputs = 0; tests constructing MergeConfig::default() variants and forgetting to set the field.

Understand the failure class

Background: "Invalid value" and "allowed values are" config errors: what your library rejected and how to fix it — this error's family across 41 libraries.

Related errors

• merge requires at least one input file
• num_outputs must be at least 1
• shard_burst_limit ({}) must be at least 1.5*content_length_l
• index ID pattern `{pattern}` is invalid: patterns must not c
• index ID pattern `{pattern}` is invalid: an index ID must ha

AI-assisted analysis of quickwit-oss/quickwit@a39730c5cd (2026-09-08). Data as JSON: /api/errors/c83254ea54a512bb. Report an issue: GitHub.