{"record":{"id":"3fbd0e6b82f6cf95","repo":"pulumi/pulumi","slug":"creating-zstd-encoder-v","errorCode":null,"errorMessage":"creating zstd encoder: %v","messagePattern":"creating zstd encoder: (.+?)","errorType":"panic","errorClass":null,"httpStatus":null,"severity":"critical","filePath":"sdk/go/common/encoding/marshal.go","lineNumber":179,"sourceCode":"\tif err := reader.Close(); err != nil {\n\t\treturn err\n\t}\n\treturn m.inner.Unmarshal(inflated, v)\n}\n\ntype zstdMarshaller struct {\n\tinner Marshaler\n}\n\nvar (\n\tzstdEncoder = newZstdEncoder()\n\tzstdDecoder = newZstdDecoder()\n)\n\nfunc newZstdEncoder() *zstd.Encoder {\n\tenc, err := zstd.NewWriter(nil, zstd.WithZeroFrames(true))\n\tif err != nil {\n\t\tpanic(fmt.Sprintf(\"creating zstd encoder: %v\", err))\n\t}\n\treturn enc\n}\n\nfunc newZstdDecoder() *zstd.Decoder {\n\tdec, err := zstd.NewReader(nil)\n\tif err != nil {\n\t\tpanic(fmt.Sprintf(\"creating zstd decoder: %v\", err))\n\t}\n\treturn dec\n}\n\nfunc (m *zstdMarshaller) Marshal(v any) ([]byte, error) {\n\tb, err := m.inner.Marshal(v)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\treturn zstdEncoder.EncodeAll(b, nil), nil","sourceCodeStart":161,"sourceCodeEnd":197,"githubUrl":"https://github.com/pulumi/pulumi/blob/793f7b2e160db4321fb7fb6b0607461e01cb251e/sdk/go/common/encoding/marshal.go#L161-L197","documentation":"This error is raised (as a panic) when the Pulumi Go SDK initializes its package-level zstd compression encoder in sdk/go/common/encoding/marshal.go. The encoder is created lazily with zstd.NewWriter(nil, zstd.WithZeroFrames(true)); because construction is expected to be infallible, any error is treated as unrecoverable and panicked. It signals a fundamental problem with the compression library setup or runtime environment rather than user data.","triggerScenarios":"Calling any marshaling code path that first touches the package-level zstdEncoder sync.Once (e.g. marshaling data compressed with zstd) when zstd.NewWriter(nil, zstd.WithZeroFrames(true)) returns an error — practically only under pathological conditions such as memory allocation failure or a broken build of github.com/klauspost/compress.","commonSituations":"Corrupted vendor/module cache or incompatible replace directives for github.com/klauspost/compress; severe memory exhaustion at process start; running a misbuilt binary.","solutions":["Verify the github.com/klauspost/compress dependency version: run `go mod tidy` and remove any replace directives or vendor corruption, then rebuild.","Rebuild with a clean module cache (`go clean -modcache && go build ./...`) to rule out a corrupted build.","Check available memory on the host; encoder allocation fails under OOM conditions.","If reproducible, report the underlying zstd.NewWriter error text upstream, since this path is expected to be infallible."],"exampleFix":"// before (application crashes at first compressed marshal)\nout := encoding.MarshalCompressed(data) // panic: creating zstd encoder: ...\n// after (pin a healthy dependency, no replaces)\n// go.mod\nrequire github.com/klauspost/compress v1.17.9","handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"// Go panics are not catchable via errors.Is; guard at process level\nfunc safeMarshal(data []byte) (out []byte, err error) {\n  defer func() {\n    if r := recover(); r != nil {\n      if s, ok := r.(string); ok && strings.Contains(s, \"creating zstd encoder\") {\n        err = fmt.Errorf(\"zstd init failed: %s\", s)\n      } else {\n        panic(r)\n      }\n    }\n  }()\n  return encoding.MarshalCompressed(data)\n}","preventionTips":["Pin github.com/klauspost/compress to a known-good version and avoid replace directives","Build from a clean module cache in CI","Monitor memory limits on hosts running the binary"],"tags":["go","zstd","compression","panic","initialization"],"backgroundTag":"zstd-encoder-init-failed","analyzedSha":"793f7b2e160db4321fb7fb6b0607461e01cb251e","analyzedAt":"2026-08-31T09:36:43.099Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}