owasp-amass/amass · error

error gz-reading the file %s: %v

Error message

error gz-reading the file %s: %v

What it means

This error is returned by getGzipReader (called from GetListFromFile) when the file's content type was detected as gzip (via http.DetectContentType on the first 512 bytes), but gzip.NewReader subsequently failed to initialize a decompression stream over it. It means the file looks like gzip at a glance but the gzip header/stream is malformed, truncated, or corrupted. The library throws it so callers know the wordlist cannot be decompressed and GetListFromFile aborts instead of silently reading garbage.

Source

Thrown at config/wordlist.go:135

	}

	// We need to determine if this is a gzipped file or a plain text file, so we
	// first read the first 512 bytes to pass them down to http.DetectContentType
	// for mime detection. The file is rewinded before passing it along to the
	// next reader
	head := make([]byte, 512)
	if _, err = file.Read(head); err != nil {
		return nil, fmt.Errorf("error reading the first 512 bytes from %s: %s", absPath, err)
	}
	if _, err = file.Seek(0, 0); err != nil {
		return nil, fmt.Errorf("error rewinding the file %s: %s", absPath, err)
	}

	// Read the file as gzip if it's actually compressed
	if mt := http.DetectContentType(head); mt == "application/gzip" || mt == "application/x-gzip" {
		gzReader, err := gzip.NewReader(file)
		if err != nil {
			return nil, fmt.Errorf("error gz-reading the file %s: %v", absPath, err)
		}

		return gzReader, nil
	}

	return nil, fmt.Errorf("%s is not compressed", absPath)
}

func GetWordList(reader io.Reader) ([]string, error) {
	var words []string

	scanner := bufio.NewScanner(reader)
	for scanner.Scan() {
		// Get the next word in the list
		w := strings.TrimSpace(scanner.Text())
		if err := scanner.Err(); err == nil && w != "" {
			words = append(words, w)
		}

View on GitHub (pinned to 79299dce87)

Solutions

1. Verify the file integrity: run `gzip -t <file>` or `gunzip -t` to confirm the archive is valid; re-download or regenerate it if not.
2. Check the file is a complete gzip stream and not truncated: compare its size against the source, or decompress fully with `gunzip -c file.gz > /dev/null`.
3. Re-compress the wordlist properly (`gzip -9 words.txt`) instead of hand-crafting or concatenating gzip data.
4. Ensure the file was transferred in binary mode (FTP 'bin') or copied with a binary-safe method, then retry.
5. If the file is actually plain text that happens to trip gzip detection, rename/remove the .gz and confirm with `file <name>` that the content matches the extension.

Example fix

// before: trusting the extension and ignoring the sniff fallback
words, err := config.GetListFromFile("wl.txt.gz")
_ = err

// after: verify gzip integrity first, then load
if err := verifyGzip("wl.txt.gz"); err != nil {
    return fmt.Errorf("wordlist is corrupt: %w", err)
}
words, err := config.GetListFromFile("wl.txt.gz")

func isLikelyValidGzip(path string) error {
    f, err := os.Open(path)
    if err != nil { return err }
    defer f.Close()
    gz, err := gzip.NewReader(f)
    if err != nil { return fmt.Errorf("not a valid gzip stream: %w", err) }
    return gz.Close()
}
// call before GetListFromFile: if err := isLikelyValidGzip(p); err != nil { skip/corrupt }

func isGzipFile(path string) bool {
    f, err := os.Open(path)
    if err != nil { return false }
    defer f.Close()
    head := make([]byte, 2)
    n, err := f.Read(head)
    return err == nil && n == 2 && head[0] == 0x1f && head[1] == 0x8b
}

When it happens

Trigger: Calling config.GetListFromFile on a file whose first 512 bytes make http.DetectContentType return application/gzip or application/x-gzip, but where gzip.NewReader fails — e.g. a truncated .gz, a corrupt/incomplete download, a multi-member gz with a bad header, or a file that merely starts with the gzip magic bytes 0x1f 0x8b but is not a valid gzip stream. Note GetListFromFile ignores the getGzipReader error (line 101) and falls back to reading the file as plain text, so this error only surfaces through direct use or manifests as garbled wordlist output.

Common situations: Partially downloaded or interrupted gzip wordlists; files corrupted by transferring in ASCII/FTP mode; concatenating a gz file onto a text file so the head sniff sees gzip magic; compressed files produced by tools writing non-standard gzip headers; storage/truncation issues on disk.

Understand the failure class

Background: "failed to read file", EACCES, ENOENT and "could not read <path>" errors: when a program can't read a file from disk — this error's family across 49 libraries.

Related errors

• %s is not compressed
• failed to get absolute path for wordlist file: %w
• unable to load the file in the bruteforce wordlist_file sett
• error reading the first 512 bytes from %s: %s
• the file is empty

AI-assisted analysis of owasp-amass/amass@79299dce87 (2026-09-06). Data as JSON: /api/errors/e5873af01b461277. Report an issue: GitHub.