What causes "failed to read gz %s: %w" in crowdsecurity/crowdsec?

Verify the file integrity: `gzip -t /path/to/file.gz`.

failed to read gz %s: %w

Error message

failed to read gz %s: %w

What it means

When the one-shot filename ends in .gz, readFile wraps it in gzip.NewReader before scanning. If the gzip stream is corrupt or the file isn't valid gzip data, it logs 'Failed to read gz file' and returns "failed to read gz %s: %w".

Source

Thrown at pkg/acquisition/modules/file/run.go:385

}

func (s *Source) readFile(ctx context.Context, filename string, out chan pipeline.Event) error {
	var scanner *bufio.Scanner

	logger := s.logger.WithField("oneshot", filename)

	fd, err := os.Open(filename)
	if err != nil {
		return fmt.Errorf("failed opening %s: %w", filename, err)
	}

	defer fd.Close()

	if strings.HasSuffix(filename, ".gz") {
		gz, err := gzip.NewReader(fd)
		if err != nil {
			logger.Errorf("Failed to read gz file: %s", err)
			return fmt.Errorf("failed to read gz %s: %w", filename, err)
		}

		defer gz.Close()

		scanner = bufio.NewScanner(gz)
	} else {
		scanner = bufio.NewScanner(fd)
	}

	scanner.Split(bufio.ScanLines)

	if s.config.MaxBufferSize > 0 {
		buf := make([]byte, 0, 64*1024)
		scanner.Buffer(buf, s.config.MaxBufferSize)
	}

	for scanner.Scan() {
		select {

View on GitHub (pinned to 909b515798)

Solutions

Verify the file integrity: `gzip -t /path/to/file.gz`.
Confirm the file is real gzip: `file /path/to/file.gz` — if not gzip, rename to drop .gz and replay as plain text.
If truncated by a bad rotation job, obtain the complete log from backup.
Re-decompress and replay the uncompressed file if gzip repair fails.

Example fix

// before
mv app.log app.log.gz  # renamed plain text
// after
gzip app.log  # actually compress it, or keep .log and replay uncompressed

Defensive patterns

Strategy: validation

Validate before calling

// validate gzip integrity before replay
gzip -t /path/to/file.gz && crowdsec -file /path/to/file.gz -type syslog

Prevention

Never rename plain-text logs to .gz; compress them properly.
Ensure log rotation finishes compressing before replaying archives.
Run `file` on the archive to confirm it is gzip data.

When it happens

Trigger: gzip.NewReader(fd) errors in readFile when filename has a .gz suffix during OneShot replay.

Common situations: File named .gz but not actually gzip-compressed (renamed plain text); partially written/truncated rotation (rotation compressed mid-write); corrupted download.

Related errors

AI-assisted analysis of crowdsecurity/crowdsec@909b515798 (2026-09-06). Data as JSON: /api/errors/9fa6c62ac50ece3b. Report an issue: GitHub.