quickwit-oss/quickwit · error

`desired_num_pipelines` must be strictly positive

Error message

`desired_num_pipelines` must be strictly positive

What it means

SourceConfig validation requires the number of indexing pipelines (`num_pipelines`) to be a positive non-zero value. The builder converts it into a `NonZeroUsize`, and if the user supplied 0 the construction fails with this message naming the YAML/JSON key `desired_num_pipelines`. It is a config validation error raised before any indexing work starts.

Source

Thrown at quickwit/quickwit-config/src/source_config/serialize.rs:105

    Ok(new_source_config)
}

impl SourceConfigForSerialization {
    /// Checks the validity of the `SourceConfig` as a "deserializable source".
    ///
    /// Two remarks:
    /// - This does not check connectivity, it just validate configuration, without performing any
    ///   IO. See `check_connectivity(..)`.
    /// - This is used each time the `SourceConfig` is deserialized (at creation but also during
    ///   communications with the metastore). When ingesting from stdin, we programmatically create
    ///   an invalid `SourceConfig` and only use it locally.
    fn validate_and_build(self) -> anyhow::Result<SourceConfig> {
        if !RESERVED_SOURCE_IDS.contains(&self.source_id.as_str()) {
            validate_identifier("source", &self.source_id)?;
        }
        let num_pipelines = NonZeroUsize::new(self.num_pipelines)
            .ok_or_else(|| anyhow::anyhow!("`desired_num_pipelines` must be strictly positive"))?;
        match &self.source_params {
            SourceParams::Stdin => {
                bail!(
                    "stdin can only be used as source through the CLI command `quickwit tool \
                     local-ingest`"
                );
            }
            SourceParams::File(_)
            | SourceParams::Kafka(_)
            | SourceParams::Kinesis(_)
            | SourceParams::Pulsar(_) => {
                // TODO consider any validation opportunity
            }
            SourceParams::PubSub(_)
            | SourceParams::Ingest
            | SourceParams::IngestApi
            | SourceParams::IngestCli
            | SourceParams::Vec(_)

View on GitHub (pinned to a39730c5cd)

Solutions

1. Set `desired_num_pipelines` to a positive integer (>= 1) in the source config.
2. If the count comes from a variable/templeting, default it to 1 when the computed value is 0.
3. To effectively disable a source, remove the source or set `enabled: false` instead of setting pipelines to 0.

Example fix

# before
source:
  desired_num_pipelines: 0
# after
source:
  desired_num_pipelines: 1

if let Some(n) = source_config.desired_num_pipelines {
    if n == 0 {
        return Err("desired_num_pipelines must be >= 1");
    }
}

When it happens

Trigger: Creating or updating a source config (via `load_source_config_from_user_config` or `load_source_config_update` such as REST POST/PATCH on a source, or index config deserialization) with `desired_num_pipelines: 0` or the field set to 0 programmatically.

Common situations: Templated YAML where a variable defaulted to 0, automation computing pipeline counts from an empty/zero value, or copy-pasted configs where the field was zeroed to 'disable' the source.

Understand the failure class

Background: "Invalid value" and "allowed values are" config errors: what your library rejected and how to fix it — this error's family across 41 libraries.

Related errors

• `index_id` in config file {} does not match updated `index_i
• shard_burst_limit ({}) must be at least 1.5*content_length_l
• index ID pattern `{pattern}` is invalid: patterns must not c
• index ID pattern `{pattern}` is invalid: an index ID must ha
• index config merge policy `max_merge_factor` must be superio

AI-assisted analysis of quickwit-oss/quickwit@a39730c5cd (2026-09-08). Data as JSON: /api/errors/b54892bee8407b72. Report an issue: GitHub.