pulumi/pulumi · critical
creating zstd encoder
Error message
creating zstd encoder: %v
What it means
This error is raised (as a panic) when the Pulumi Go SDK initializes its package-level zstd compression encoder in sdk/go/common/encoding/marshal.go. The encoder is created lazily with zstd.NewWriter(nil, zstd.WithZeroFrames(true)); because construction is expected to be infallible, any error is treated as unrecoverable and panicked. It signals a fundamental problem with the compression library setup or runtime environment rather than user data.
Solutions
- Verify the github.com/klauspost/compress dependency version: run `go mod tidy` and remove any replace directives or vendor corruption, then rebuild.
- Rebuild with a clean module cache (`go clean -modcache && go build ./...`) to rule out a corrupted build.
- Check available memory on the host; encoder allocation fails under OOM conditions.
- If reproducible, report the underlying zstd.NewWriter error text upstream, since this path is expected to be infallible.
Example fix
// before (application crashes at first compressed marshal) out := encoding.MarshalCompressed(data) // panic: creating zstd encoder: ... // after (pin a healthy dependency, no replaces) // go.mod require github.com/klauspost/compress v1.17.9
Defensive patterns
Strategy: try-catch
Try / catch
// Go panics are not catchable via errors.Is; guard at process level
func safeMarshal(data []byte) (out []byte, err error) {
defer func() {
if r := recover(); r != nil {
if s, ok := r.(string); ok && strings.Contains(s, "creating zstd encoder") {
err = fmt.Errorf("zstd init failed: %s", s)
} else {
panic(r)
}
}
}()
return encoding.MarshalCompressed(data)
} Prevention
- Pin github.com/klauspost/compress to a known-good version and avoid replace directives
- Build from a clean module cache in CI
- Monitor memory limits on hosts running the binary
When it happens
Trigger: Calling any marshaling code path that first touches the package-level zstdEncoder sync.Once (e.g. marshaling data compressed with zstd) when zstd.NewWriter(nil, zstd.WithZeroFrames(true)) returns an error — practically only under pathological conditions such as memory allocation failure or a broken build of github.com/klauspost/compress.
Common situations: Corrupted vendor/module cache or incompatible replace directives for github.com/klauspost/compress; severe memory exhaustion at process start; running a misbuilt binary.
Related errors
- applier's invalid function signature (panic re-raises the…
- callback panicked
- cannot convert an input of type %T to a value of type
- DIY_BACKEND_GZIP and DIY_BACKEND_ZSTD cannot both be enabled
- entryPoint ' ' was not a valid file name when creating…
AI-assisted analysis of pulumi/pulumi@793f7b2e16 (2026-08-31).
Data as JSON: /api/errors/3fbd0e6b82f6cf95.
Report an issue: GitHub.
Appendix: source
Thrown at sdk/go/common/encoding/marshal.go:179
if err := reader.Close(); err != nil {
return err
}
return m.inner.Unmarshal(inflated, v)
}
type zstdMarshaller struct {
inner Marshaler
}
var (
zstdEncoder = newZstdEncoder()
zstdDecoder = newZstdDecoder()
)
func newZstdEncoder() *zstd.Encoder {
enc, err := zstd.NewWriter(nil, zstd.WithZeroFrames(true))
if err != nil {
panic(fmt.Sprintf("creating zstd encoder: %v", err))
}
return enc
}
func newZstdDecoder() *zstd.Decoder {
dec, err := zstd.NewReader(nil)
if err != nil {
panic(fmt.Sprintf("creating zstd decoder: %v", err))
}
return dec
}
func (m *zstdMarshaller) Marshal(v any) ([]byte, error) {
b, err := m.inner.Marshal(v)
if err != nil {
return nil, err
}
return zstdEncoder.EncodeAll(b, nil), nilView on GitHub (pinned to 793f7b2e16)