quickwit-oss/quickwit · error

shard_burst_limit ({}) must be at least 1.5*content_length_l

Error message

shard_burst_limit ({}) must be at least 1.5*content_length_limit ({})

What it means

Quickwit validates that the indexing service's `shard_burst_limit` is at least the estimated per-shard persist size (derived as 1.5x the ingest API content-length limit). If the configured burst limit is smaller than the estimated persist size, a single ingest request could exceed the shard's capacity and indexing would fail, so node config validation rejects it at startup.

Source

Thrown at quickwit/quickwit-config/src/node_config/mod.rs:837

            "ingestion shard throughput limit: {}",
            self.shard_throughput_limit.display().si()
        );
        ensure!(
            self.shard_throughput_limit >= ByteSize::mib(1)
                && self.shard_throughput_limit <= ByteSize::mib(20),
            "shard_throughput_limit ({}) must be within 1mb and 20mb",
            self.shard_throughput_limit.display().si()
        );
        // The newline delimited format is persisted as something a bit larger
        // (lines prefixed with their length)
        let estimated_persist_size = ByteSize::b(3 * self.content_length_limit.as_u64() / 2);
        ensure!(
            self.shard_burst_limit >= estimated_persist_size,
            "shard_burst_limit ({}) must be at least 1.5*content_length_limit ({})",
            self.shard_burst_limit,
            estimated_persist_size,
        );
        ensure!(
            self.shard_scale_up_factor > 1.0,
            "shard_scale_up_factor ({}) must be greater than 1",
            self.shard_scale_up_factor,
        );
        Ok(())
    }
}

#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct JaegerConfig {
    /// Enables the gRPC endpoint that allows the Jaeger Query Service to connect and retrieve
    /// traces.
    #[serde(default = "JaegerConfig::default_enable_endpoint")]
    pub enable_endpoint: bool,
    /// How far back in time we look for spans when queries at not time-bound (`get_services`,
    /// `get_operations`, `get_trace` operations).
    #[serde(default = "JaegerConfig::default_lookback_period_hours")]

View on GitHub (pinned to a39730c5cd)

Solutions

1. Raise `shard_burst_limit` in the node config so it is >= 1.5 * the ingest content_length_limit (the value shown in the message).
2. Alternatively lower the ingest `content_length_limit` so the estimated persist size fits within the configured shard_burst_limit.
3. Remove the custom override entirely and rely on the default derived relationship between the two settings.

Example fix

// before (quickwit.yaml)
indexer:
  shard_burst_limit: 5000000   # 5MB
# ingest content_length_limit: 20000000 (20MB) -> needs >= 30MB

// after (quickwit.yaml)
indexer:
  shard_burst_limit: 30000000  # 30MB, >= 1.5 * content_length_limit

let estimated_persist_size = 3 * ingest_config.content_length_limit / 2;
assert!(indexer_config.shard_burst_limit >= estimated_persist_size,
    "shard_burst_limit must be >= 1.5 * content_length_limit");

When it happens

Trigger: Loading a node config (load_node_config_with_env -> NodeConfig validation) where `indexer.shard_burst_limit` (or equivalent) is set lower than the estimated persist size computed from the ingest `content_length_limit`.

Common situations: Operators tune shard_burst_limit down to throttle indexing throughput without realizing it is coupled to the ingest API content-length limit; configs copied from small deployments then reused with a larger content_length_limit.

Understand the failure class

Background: "Invalid value" and "allowed values are" config errors: what your library rejected and how to fix it — this error's family across 41 libraries.

Related errors

• num_outputs must be at least 1
• `desired_num_pipelines` must be strictly positive
• `index_id` in config file {} does not match updated `index_i
• position of a Kinesis shard should never be EOF
• index ID pattern `{pattern}` is invalid: patterns must not c

AI-assisted analysis of quickwit-oss/quickwit@a39730c5cd (2026-09-08). Data as JSON: /api/errors/e4fc30d2b9157436. Report an issue: GitHub.