{"record":{"id":"78070fa235dfa460","repo":"quickwit-oss/quickwit","slug":"shard-throughput-limit-must-be-within-1mb-and","errorCode":null,"errorMessage":"shard_throughput_limit ({}) must be within 1mb and 20mb","messagePattern":"shard_throughput_limit \\((.+?)\\) must be within 1mb and 20mb","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"quickwit/quickwit-config/src/node_config/mod.rs","lineNumber":822,"sourceCode":"\n    fn validate(&self) -> anyhow::Result<()> {\n        self.warn_if_replication_factor_is_set();\n        ensure!(\n            self.max_queue_disk_usage > ByteSize::mib(256),\n            \"max_queue_disk_usage must be at least 256 MiB, got `{}`\",\n            self.max_queue_disk_usage.display().si()\n        );\n        ensure!(\n            self.max_queue_disk_usage >= self.max_queue_memory_usage,\n            \"max_queue_disk_usage ({}) must be at least max_queue_memory_usage ({})\",\n            self.max_queue_disk_usage.display().si(),\n            self.max_queue_memory_usage.display().si()\n        );\n        info!(\n            \"ingestion shard throughput limit: {}\",\n            self.shard_throughput_limit.display().si()\n        );\n        ensure!(\n            self.shard_throughput_limit >= ByteSize::mib(1)\n                && self.shard_throughput_limit <= ByteSize::mib(20),\n            \"shard_throughput_limit ({}) must be within 1mb and 20mb\",\n            self.shard_throughput_limit.display().si()\n        );\n        // The newline delimited format is persisted as something a bit larger\n        // (lines prefixed with their length)\n        let estimated_persist_size = ByteSize::b(3 * self.content_length_limit.as_u64() / 2);\n        ensure!(\n            self.shard_burst_limit >= estimated_persist_size,\n            \"shard_burst_limit ({}) must be at least 1.5*content_length_limit ({})\",\n            self.shard_burst_limit,\n            estimated_persist_size,\n        );\n        ensure!(\n            self.shard_scale_up_factor > 1.0,\n            \"shard_scale_up_factor ({}) must be greater than 1\",\n            self.shard_scale_up_factor,","sourceCodeStart":804,"sourceCodeEnd":840,"githubUrl":"https://github.com/quickwit-oss/quickwit/blob/a39730c5cdcd1a4fe798403737ae293999ea21f8/quickwit/quickwit-config/src/node_config/mod.rs#L804-L840","documentation":"The ingestion shard throughput limit must be between 1 MiB and 20 MiB per second; values outside this range are rejected during ingest config validation. This bounds per-shard consumption so a single shard can neither starve (too low) nor overwhelm the node (too high).","triggerScenarios":"Setting `shard_throughput_limit` in the node config below 1 MiB or above 20 MiB and starting/validating the node.","commonSituations":"Aggressive tuning for high-throughput ingestion (setting 50mb); setting a near-zero limit by mistake; copying a value in bytes rather than a byte-size string and ending up far outside the range.","solutions":["Set `shard_throughput_limit` within [1mib, 20mib], e.g. `10mib`","Remove the setting to use the default if custom tuning isn't required","To raise total ingestion throughput, add shards rather than exceeding the per-shard cap"],"exampleFix":"// before\nshard_throughput_limit: 50mb\n// after\nshard_throughput_limit: 20mb","handlingStrategy":"validation","validationCode":"fn ensure_shard_throughput(v: ByteSize) -> anyhow::Result<()> {\n    if !(ByteSize::mib(1)..=ByteSize::mib(20)).contains(&v) {\n        anyhow::bail!(\"shard_throughput_limit ({}) must be within 1mb and 20mb\", v.display().si());\n    }\n    Ok(())\n}","typeGuard":null,"tryCatchPattern":"if let Err(e) = ingest_config.validate() {\n    if e.to_string().contains(\"shard_throughput_limit\") {\n        eprintln!(\"Set shard_throughput_limit between 1 MiB and 20 MiB.\");\n    }\n}","preventionTips":["Clamp generated values to [1mib, 20mib] in provisioning code","Scale ingestion by adding shards, not by raising the per-shard cap","Validate the whole node config in CI before deploying"],"tags":["config","ingest","validation"],"backgroundTag":"value-out-of-range","analyzedSha":"a39730c5cdcd1a4fe798403737ae293999ea21f8","analyzedAt":"2026-09-08T13:19:37.784Z","contentChangedAt":"2026-09-08T13:19:37.784Z","schemaVersion":2},"datasetVersion":"2026-09-17T15:17:12.973Z"}