{"record":{"id":"48ac926e01898186","repo":"vectordotdev/vector","slug":"zstd-encoder-should-not-fail-on-init","errorCode":null,"errorMessage":"Zstd encoder should not fail on init.","messagePattern":"Zstd encoder should not fail on init\\.","errorType":"panic","errorClass":null,"httpStatus":null,"severity":"error","filePath":"src/sinks/util/buffer/mod.rs","lineNumber":65,"sourceCode":"        }\n    }\n\n    fn buffer(&mut self) -> &mut InnerBuffer {\n        let bytes = self.settings.bytes;\n        let compression = self.compression;\n        self.inner.get_or_insert_with(|| {\n            let writer = BytesMut::with_capacity(bytes).writer();\n            match compression {\n                Compression::None => InnerBuffer::Plain(writer),\n                Compression::Gzip(level) => {\n                    InnerBuffer::Gzip(GzEncoder::new(writer, level.as_flate2()))\n                }\n                Compression::Zlib(level) => {\n                    InnerBuffer::Zlib(ZlibEncoder::new(writer, level.as_flate2()))\n                }\n                Compression::Zstd(level) => InnerBuffer::Zstd(\n                    ZstdEncoder::new(writer, level.into())\n                        .expect(\"Zstd encoder should not fail on init.\"),\n                ),\n                Compression::Snappy => InnerBuffer::Snappy(SnappyEncoder::new(writer)),\n            }\n        })\n    }\n\n    pub fn push(&mut self, input: &[u8]) {\n        self.num_items += 1;\n        match self.buffer() {\n            InnerBuffer::Plain(inner) => {\n                inner.write_all(input).unwrap();\n            }\n            InnerBuffer::Gzip(inner) => {\n                inner.write_all(input).unwrap();\n            }\n            InnerBuffer::Zlib(inner) => {\n                inner.write_all(input).unwrap();\n            }","sourceCodeStart":47,"sourceCodeEnd":83,"githubUrl":"https://github.com/vectordotdev/vector/blob/bdb87aeaa4c4ff27c0ba643c1c77b21bf2ef4013/src/sinks/util/buffer/mod.rs#L47-L83","documentation":"This is a Rust panic assertion (`.expect`) on `ZstdEncoder::new(writer, level)`, which returns a Result only because zstd allocation/compression-parameter validation can fail in theory. Vector wraps it in an expect because with an in-memory `BytesMut` writer and a validated `Compression` level, initialization is treated as an infallible internal invariant. If this panic fires, the zstd crate rejected the encoder creation itself (e.g. allocation failure or an out-of-range compression level), meaning a broken invariant rather than a runtime condition the caller can handle.","triggerScenarios":"Calling `BatchSerializerBuffer::buffer()`/`push()` with `compression = Compression::Zstd(level)` when `ZstdEncoder::new` returns Err — practically only on memory allocation failure or an invalid zstd compression level being passed into `level.into()`.","commonSituations":"Configuring a sink with `compression = \"zstd\"` and an extreme/unsupported compression level (e.g. level above zstd's max of 22 or a negative level after conversion); running under severe memory pressure where the zstd internal allocation fails; using a custom-built zstd build with restricted limits.","solutions":["Check the sink's compression configuration and set zstd level to a valid range (1-22, typically 3 as default).","Reduce memory pressure in the process/container (raise memory limits) since the failure mode is allocation failure inside zstd.","If the panic reproduces reliably, verify the zstd crate version in Cargo.lock for regressions and pin/update it.","Switch compression to gzip/zlib/snappy as a workaround while investigating."],"exampleFix":"// before (config vector.yaml)\ncompression: zstd\ncompression_level: 100\n// after\ncompression: zstd\ncompression_level: 3","handlingStrategy":"validation","validationCode":"// Validate zstd compression level before configuring the sink\nfn validate_zstd_level(level: i32) -> Result<(), String> {\n    if (1..=22).contains(&level) {\n        Ok(())\n    } else {\n        Err(format!(\"zstd compression_level must be 1-22, got {level}\"))\n    }\n}","typeGuard":null,"tryCatchPattern":"// Rust panics are not catchable via Result; abort the task.\n// Use std::panic::catch_unwind only at an isolation boundary:\nlet result = std::panic::catch_unwind(|| buffer_with_zstd_config());\nmatch result {\n    Ok(inner) => inner,\n    Err(_) => restart_sink_task_with_default_compression(),\n}","preventionTips":["Keep zstd compression_level within the documented 1-22 range in sink configs.","Provision adequate memory for the Vector process; zstd init allocates internally.","Pin the zstd crate version and test compression configs after dependency upgrades."],"tags":["rust","compression","zstd","panic","internal-invariant"],"backgroundTag":"internal-invariant-violation","analyzedSha":"bdb87aeaa4c4ff27c0ba643c1c77b21bf2ef4013","analyzedAt":"2026-09-16T02:53:35.741Z","contentChangedAt":"2026-09-16T02:53:35.741Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}