What causes "file cannot be checked for compression" in owasp-amass/amass?

Pad or use a wordlist file of at least 512 bytes (add more entries).

file cannot be checked for compression

Error message

file cannot be checked for compression

What it means

getGzipReader stats the file to decide whether compression detection is possible; the detection routine needs at least 512 bytes (the http.DetectContentType sniff size). Files smaller than 512 bytes cannot be reliably checked for gzip content, so this error is returned rather than guessing the format.

Source

Thrown at config/wordlist.go:116

		return nil, errors.New("the file is empty")
	}

	if gz, err := getGzipReader(file, absPath); err == nil {
		defer func() { _ = gz.Close() }()
		reader = gz
	}

	return GetWordList(reader)
}

func getGzipReader(file *os.File, absPath string) (*gzip.Reader, error) {
	finfo, err := file.Stat()
	if err != nil {
		return nil, err
	}

	if finfo.Size() < 512 {
		return nil, errors.New("file cannot be checked for compression")
	}

	// We need to determine if this is a gzipped file or a plain text file, so we
	// first read the first 512 bytes to pass them down to http.DetectContentType
	// for mime detection. The file is rewinded before passing it along to the
	// next reader
	head := make([]byte, 512)
	if _, err = file.Read(head); err != nil {
		return nil, fmt.Errorf("error reading the first 512 bytes from %s: %s", absPath, err)
	}
	if _, err = file.Seek(0, 0); err != nil {
		return nil, fmt.Errorf("error rewinding the file %s: %s", absPath, err)
	}

	// Read the file as gzip if it's actually compressed
	if mt := http.DetectContentType(head); mt == "application/gzip" || mt == "application/x-gzip" {
		gzReader, err := gzip.NewReader(file)
		if err != nil {

View on GitHub (pinned to 79299dce87)

Solutions

Pad or use a wordlist file of at least 512 bytes (add more entries).
Combine several small lists into one larger file.
Skip compression detection by supplying a known plain-text path if the library supports it.
Check the file size before calling and warn the user.

Example fix

// before
data := []byte("a.com\nb.com\n") // 11 bytes
_ = os.WriteFile("small.txt", data, 0644)
// after
data := bytes.Repeat([]byte("a.com\n"), 100) // > 512 bytes
_ = os.WriteFile("wordlist.txt", data, 0644)

Defensive patterns

Strategy: validation

Validate before calling

info, err := os.Stat(path)
if err != nil { return err }
if info.Size() < 512 { return fmt.Errorf("wordlist %s is smaller than 512 bytes; compression cannot be detected", path) }

Try / catch

list, err := GetListFromFile(path)
if err != nil {
    if strings.Contains(err.Error(), "cannot be checked for compression") {
        // pad the wordlist to >=512 bytes or concatenate with a larger list
    }
    return err
}

Prevention

Use wordlists larger than 512 bytes in tests and CI.
Warn users about tiny wordlists at startup.
Concatenate small lists before use.
Know the 512-byte sniff threshold when creating custom lists.

When it happens

Trigger: Calling GetListFromFile on a wordlist smaller than 512 bytes (e.g. a handful of subdomain entries in a plain-text file), causing getGzipReader to hit finfo.Size() < 512 and fail.

Common situations: Testing with a tiny wordlist of a few lines, a partially written file, or an intentionally minimal custom list — common in CI or quick experiments.

Understand the failure class

Background: "File too large" / "file size exceeds limit" errors: why libraries cap file sizes and how to fix them — this error's family across 46 libraries.

Related errors

AI-assisted analysis of owasp-amass/amass@79299dce87 (2026-09-06). Data as JSON: /api/errors/5b0bde46769715f8. Report an issue: GitHub.