{"record":{"id":"e5873af01b461277","repo":"owasp-amass/amass","slug":"error-gz-reading-the-file-s-v","errorCode":null,"errorMessage":"error gz-reading the file %s: %v","messagePattern":"error gz-reading the file (.+?): (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"config/wordlist.go","lineNumber":135,"sourceCode":"\t}\n\n\t// We need to determine if this is a gzipped file or a plain text file, so we\n\t// first read the first 512 bytes to pass them down to http.DetectContentType\n\t// for mime detection. The file is rewinded before passing it along to the\n\t// next reader\n\thead := make([]byte, 512)\n\tif _, err = file.Read(head); err != nil {\n\t\treturn nil, fmt.Errorf(\"error reading the first 512 bytes from %s: %s\", absPath, err)\n\t}\n\tif _, err = file.Seek(0, 0); err != nil {\n\t\treturn nil, fmt.Errorf(\"error rewinding the file %s: %s\", absPath, err)\n\t}\n\n\t// Read the file as gzip if it's actually compressed\n\tif mt := http.DetectContentType(head); mt == \"application/gzip\" || mt == \"application/x-gzip\" {\n\t\tgzReader, err := gzip.NewReader(file)\n\t\tif err != nil {\n\t\t\treturn nil, fmt.Errorf(\"error gz-reading the file %s: %v\", absPath, err)\n\t\t}\n\n\t\treturn gzReader, nil\n\t}\n\n\treturn nil, fmt.Errorf(\"%s is not compressed\", absPath)\n}\n\nfunc GetWordList(reader io.Reader) ([]string, error) {\n\tvar words []string\n\n\tscanner := bufio.NewScanner(reader)\n\tfor scanner.Scan() {\n\t\t// Get the next word in the list\n\t\tw := strings.TrimSpace(scanner.Text())\n\t\tif err := scanner.Err(); err == nil && w != \"\" {\n\t\t\twords = append(words, w)\n\t\t}","sourceCodeStart":117,"sourceCodeEnd":153,"githubUrl":"https://github.com/owasp-amass/amass/blob/79299dce87b0085db0f2f4ef3e9c52cccb49f514/config/wordlist.go#L117-L153","documentation":"This error is returned by getGzipReader (called from GetListFromFile) when the file's content type was detected as gzip (via http.DetectContentType on the first 512 bytes), but gzip.NewReader subsequently failed to initialize a decompression stream over it. It means the file looks like gzip at a glance but the gzip header/stream is malformed, truncated, or corrupted. The library throws it so callers know the wordlist cannot be decompressed and GetListFromFile aborts instead of silently reading garbage.","triggerScenarios":"Calling config.GetListFromFile on a file whose first 512 bytes make http.DetectContentType return application/gzip or application/x-gzip, but where gzip.NewReader fails — e.g. a truncated .gz, a corrupt/incomplete download, a multi-member gz with a bad header, or a file that merely starts with the gzip magic bytes 0x1f 0x8b but is not a valid gzip stream. Note GetListFromFile ignores the getGzipReader error (line 101) and falls back to reading the file as plain text, so this error only surfaces through direct use or manifests as garbled wordlist output.","commonSituations":"Partially downloaded or interrupted gzip wordlists; files corrupted by transferring in ASCII/FTP mode; concatenating a gz file onto a text file so the head sniff sees gzip magic; compressed files produced by tools writing non-standard gzip headers; storage/truncation issues on disk.","solutions":["Verify the file integrity: run `gzip -t <file>` or `gunzip -t` to confirm the archive is valid; re-download or regenerate it if not.","Check the file is a complete gzip stream and not truncated: compare its size against the source, or decompress fully with `gunzip -c file.gz > /dev/null`.","Re-compress the wordlist properly (`gzip -9 words.txt`) instead of hand-crafting or concatenating gzip data.","Ensure the file was transferred in binary mode (FTP 'bin') or copied with a binary-safe method, then retry.","If the file is actually plain text that happens to trip gzip detection, rename/remove the .gz and confirm with `file <name>` that the content matches the extension."],"exampleFix":"// before: trusting the extension and ignoring the sniff fallback\nwords, err := config.GetListFromFile(\"wl.txt.gz\")\n_ = err\n\n// after: verify gzip integrity first, then load\nif err := verifyGzip(\"wl.txt.gz\"); err != nil {\n    return fmt.Errorf(\"wordlist is corrupt: %w\", err)\n}\nwords, err := config.GetListFromFile(\"wl.txt.gz\")","handlingStrategy":"validation","validationCode":"func isLikelyValidGzip(path string) error {\n    f, err := os.Open(path)\n    if err != nil { return err }\n    defer f.Close()\n    gz, err := gzip.NewReader(f)\n    if err != nil { return fmt.Errorf(\"not a valid gzip stream: %w\", err) }\n    return gz.Close()\n}\n// call before GetListFromFile: if err := isLikelyValidGzip(p); err != nil { skip/corrupt }","typeGuard":"func isGzipFile(path string) bool {\n    f, err := os.Open(path)\n    if err != nil { return false }\n    defer f.Close()\n    head := make([]byte, 2)\n    n, err := f.Read(head)\n    return err == nil && n == 2 && head[0] == 0x1f && head[1] == 0x8b\n}","tryCatchPattern":null,"preventionTips":["Always validate gzip wordlists with `gzip -t` or a probe gzip.NewReader before feeding them to the library.","Download wordlists over checksum-verified channels (sha256) so truncation/corruption is caught early.","Transfer and store archives in binary mode; never ASCII-mode FTP or lossy text transformations.","Don't hand-concatenate .gz files; re-compress from the original text instead.","Note that GetListFromFile silently falls back to plain-text reading on this error — treat garbled output as a sign of a corrupt gz."],"tags":["gzip","file-corruption","wordlist","go"],"backgroundTag":"file-read-failed","analyzedSha":"79299dce87b0085db0f2f4ef3e9c52cccb49f514","analyzedAt":"2026-09-06T08:22:48.198Z","contentChangedAt":"2026-09-06T08:22:48.198Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}