crowdsecurity/crowdsec · error

closing gzip writer: %w

Error message

closing gzip writer: %w

What it means

PrepareRequest wraps the error from gzipWriter.Close(), which flushes the gzip footer/trailer to the buffer. Close returns an error only if the pending write to the underlying bytes.Buffer fails, so like its sibling this points at in-memory compression failing unexpectedly rather than any user-fixable input problem.

Source

Thrown at pkg/apiclient/client_http.go:52

	if body != nil {
		jsonBuf := &bytes.Buffer{}
		enc := json.NewEncoder(jsonBuf)
		enc.SetEscapeHTML(false)

		if err = enc.Encode(body); err != nil {
			return nil, err
		}

		jsonBytes := jsonBuf.Bytes()
		if len(jsonBytes) > compressionMinSize {
			compressedBody = true
			buf = &bytes.Buffer{}
			gzipWriter := gzip.NewWriter(buf)
			if _, err = gzipWriter.Write(jsonBytes); err != nil {
				return nil, fmt.Errorf("writing to gzip writer: %w", err)
			}
			if err = gzipWriter.Close(); err != nil {
				return nil, fmt.Errorf("closing gzip writer: %w", err)
			}
		} else {
			buf = jsonBuf
		}
	}

	req, err := http.NewRequestWithContext(ctx, method, u.String(), buf)
	if err != nil {
		return nil, err
	}

	if body != nil {
		req.Header.Set("Content-Type", "application/json")
		if compressedBody {
			req.Header.Set("Content-Encoding", "gzip")
		}
	}

View on GitHub (pinned to 909b515798)

Solutions

1. Free up host memory / check for OOM conditions and retry
2. If it persists on a healthy host, capture the inner error and report it to the crowdsec maintainers
3. As a workaround, reduce batch size so payloads stay under the 5KB compression threshold

req, err := client.PrepareRequest(ctx, method, u, body)
if err != nil && strings.Contains(err.Error(), "gzip") {
    // retry once after a short delay; failure is resource-related, not input-related
    time.Sleep(500 * time.Millisecond)
    req, err = client.PrepareRequest(ctx, method, u, body)
}
return req, err

When it happens

Trigger: len(jsonBytes) > 5KB, Write succeeded, but gzipWriter.Close() returns non-nil — pending buffered data could not be written to the in-memory buffer (resource exhaustion) or the writer was already in an error state.

Common situations: Severe memory pressure on the machine; otherwise practically unreachable — this is a defensive wrap around the gzip writer contract.

Related errors

• writing to gzip writer: %w
• context must be non-nil
• failed to read gz %s: %w
• failed to read header of object %s/%s: %w
• failed to create gzip reader for object %s/%s: %w

AI-assisted analysis of crowdsecurity/crowdsec@909b515798 (2026-09-06). Data as JSON: /api/errors/c1df3e44b8a0e843. Report an issue: GitHub.