quickwit-oss/quickwit · error

max_queue_disk_usage

Error message

max_queue_disk_usage ({}) must be at least max_queue_memory_usage ({})

What it means

Ingest queue validation requires that the on-disk queue capacity is at least as large as the in-memory queue capacity (`max_queue_disk_usage >= max_queue_memory_usage`). The disk queue is the spill/overflow area for the memory queue, so a smaller disk queue is an invalid configuration and produces this error listing both SI-formatted sizes.

Solutions

1. Increase `max_queue_disk_usage` until it is >= `max_queue_memory_usage`
2. Decrease `max_queue_memory_usage` to fit under the current disk limit
3. Set both explicitly in the config so their relationship is obvious

Example fix

// before
max_queue_memory_usage: 1gib
max_queue_disk_usage: 512mib
// after
max_queue_memory_usage: 512mib
max_queue_disk_usage: 1gib

fn ensure_disk_geq_memory(memory: ByteSize, disk: ByteSize) -> anyhow::Result<()> {
    if disk < memory {
        anyhow::bail!(
            "max_queue_disk_usage ({}) must be >= max_queue_memory_usage ({})",
            disk.display().si(), memory.display().si()
        );
    }
    Ok(())
}

if let Err(e) = ingest_config.validate() {
    if e.to_string().contains("must be at least max_queue_memory_usage") {
        eprintln!("Increase max_queue_disk_usage or decrease max_queue_memory_usage.");
    }
}

When it happens

Trigger: Node config where `max_queue_disk_usage` is smaller than `max_queue_memory_usage`, detected during ingest config validation at startup.

Common situations: Increasing `max_queue_memory_usage` (e.g. to 1gib) without raising the disk limit; tuned-down disk quota on a small node while memory defaults stayed high; unit confusion between mib/gib values.

Understand the failure class

Background: Conflicting config options: "cannot be used together" — configuration validation errors across open-source libraries — this error's family across 162 libraries.

Related errors

• max_queue_disk_usage must be at least 256 MiB, got
• shard_throughput_limit
• concatenate field uses an unknown field
• contains an empty label name
• contains an empty label value

AI-assisted analysis of quickwit-oss/quickwit@a39730c5cd (2026-09-08). Data as JSON: /api/errors/a2f2fdb1819386a7. Report an issue: GitHub.

Appendix: source

Thrown at quickwit/quickwit-config/src/node_config/mod.rs:812

    }

    pub fn grpc_compression_encoding(&self) -> Option<CompressionEncoding> {
        self.grpc_compression_algorithm
            .as_ref()
            .map(|algorithm| match algorithm {
                CompressionAlgorithm::Gzip => CompressionEncoding::Gzip,
                CompressionAlgorithm::Zstd => CompressionEncoding::Zstd,
            })
    }

    fn validate(&self) -> anyhow::Result<()> {
        self.warn_if_replication_factor_is_set();
        ensure!(
            self.max_queue_disk_usage > ByteSize::mib(256),
            "max_queue_disk_usage must be at least 256 MiB, got `{}`",
            self.max_queue_disk_usage.display().si()
        );
        ensure!(
            self.max_queue_disk_usage >= self.max_queue_memory_usage,
            "max_queue_disk_usage ({}) must be at least max_queue_memory_usage ({})",
            self.max_queue_disk_usage.display().si(),
            self.max_queue_memory_usage.display().si()
        );
        info!(
            "ingestion shard throughput limit: {}",
            self.shard_throughput_limit.display().si()
        );
        ensure!(
            self.shard_throughput_limit >= ByteSize::mib(1)
                && self.shard_throughput_limit <= ByteSize::mib(20),
            "shard_throughput_limit ({}) must be within 1mb and 20mb",
            self.shard_throughput_limit.display().si()
        );
        // The newline delimited format is persisted as something a bit larger
        // (lines prefixed with their length)
        let estimated_persist_size = ByteSize::b(3 * self.content_length_limit.as_u64() / 2);

View on GitHub (pinned to a39730c5cd)