hashicorp/packer · error

failed to create zstd encoder: %s

Error message

failed to create zstd encoder: %s

What it means

Thrown when initializing the klauspost/compression zstd encoder fails. This is rare — zstd.NewWriter with only an encoder-level option essentially never fails unless memory for the encoder cannot be allocated.

Source

Thrown at packer/provisioner.go:403

	packerSbom, err := os.Open(tmpFileName)
	if err != nil {
		return fmt.Errorf("failed to open Packer SBOM file %q: %s", tmpFileName, err)
	}
	defer func() {
		if err := packerSbom.Close(); err != nil {
			log.Printf("[WARN] Failed to close Packer SBOM file: %s", err)
		}
	}()

	provisionerOut := &hcpSbomProvisioner.PackerSBOM{}
	err = json.NewDecoder(packerSbom).Decode(provisionerOut)
	if err != nil {
		return fmt.Errorf("malformed packer SBOM output from file %q: %s", tmpFileName, err)
	}

	encoder, err := zstd.NewWriter(nil, zstd.WithEncoderLevel(zstd.SpeedBestCompression))
	if err != nil {
		return fmt.Errorf("failed to create zstd encoder: %s", err)
	}
	p.CompressedData = encoder.EncodeAll(provisionerOut.RawSBOM, nil)
	p.SBOMFormat = provisionerOut.Format
	p.SBOMName = provisionerOut.Name

	return nil
}

View on GitHub (pinned to eb36e3c3e4)

Solutions

1. Increase the memory available to the Packer process/container and retry
2. Lower the compression level by patching/using a Packer build without SpeedBestCompression if you build from source
3. Check dmesg/OOM-killer logs to confirm memory pressure caused the failure

// Go: ensure the process has headroom before heavy compression
var ms runtime.MemStats
runtime.ReadMemStats(&ms)
// e.g. require at least 128 MiB heap headroom in constrained containers
if ms.Sys > 4<<30 { log.Println("memory pressure: expect slow zstd best-compression") }

if err := p.Provision(ctx, ui, comm, data); err != nil {
    if strings.Contains(err.Error(), "failed to create zstd encoder") {
        // raise container memory limit and retry
    }
    return err
}

When it happens

Trigger: Provision calls zstd.NewWriter(nil, zstd.WithEncoderLevel(zstd.SpeedBestCompression)) and it returns an error, which only realistically happens on allocation failure (severe memory pressure / cgroup OOM limits).

Common situations: Containers with very low memory limits; hosts under extreme memory exhaustion while compressing a large SBOM.

Related errors

• signing_mode %q does not support Sigstore bundle emission
• decode envelope payload: %w
• decode envelope signature: %w
• read verifier %q: %w
• load verifier %q: %w

AI-assisted analysis of hashicorp/packer@eb36e3c3e4 (2026-09-05). Data as JSON: /api/errors/a29e1a89eb54be08. Report an issue: GitHub.