{"record":{"id":"18c753e6f7914b55","repo":"gitleaks/gitleaks","slug":"failed-to-get-stdout-pipe-w","errorCode":null,"errorMessage":"failed to get stdout pipe: %w","messagePattern":"failed to get stdout pipe: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"sources/git.go","lineNumber":213,"sourceCode":"}\n\n// NewBlobReader returns an io.ReadCloser that can be used to read a blob\n// within the git repo used to create the GitCmd.\n//\n// The caller is responsible for closing the reader.\nfunc (c *GitCmd) NewBlobReader(commit, path string) (io.ReadCloser, error) {\n\treturn c.NewBlobReaderContext(context.Background(), commit, path)\n}\n\n// NewBlobReaderContext is the same as NewBlobReader but supports passing in a\n// context to use for timeouts\nfunc (c *GitCmd) NewBlobReaderContext(ctx context.Context, commit, path string) (io.ReadCloser, error) {\n\tgitArgs := []string{\"-C\", c.repoPath, \"cat-file\", \"blob\", commit + \":\" + path}\n\tcmd := exec.CommandContext(ctx, \"git\", gitArgs...)\n\tcmd.Stderr = io.Discard\n\tstdout, err := cmd.StdoutPipe()\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"failed to get stdout pipe: %w\", err)\n\t}\n\tif err := cmd.Start(); err != nil {\n\t\treturn nil, fmt.Errorf(\"failed to start git command: %w\", err)\n\t}\n\treturn &blobReader{\n\t\tReadCloser: stdout,\n\t\tcmd:        cmd,\n\t}, nil\n}\n\n// listenForStdErr listens for stderr output from git, prints it to stdout,\n// sends to errCh and closes it.\nfunc listenForStdErr(stderr io.ReadCloser, errCh chan<- error) {\n\tdefer close(errCh)\n\n\tvar errEncountered bool\n\n\tscanner := bufio.NewScanner(stderr)","sourceCodeStart":195,"sourceCodeEnd":231,"githubUrl":"https://github.com/gitleaks/gitleaks/blob/b58d3f102cf3a2c84cb7f923d05c25c9b1aed84b/sources/git.go#L195-L231","documentation":"Returned by GitCmd.NewBlobReaderContext (sources/git.go:213) when cmd.StdoutPipe() fails for the `git -C <repo> cat-file blob <commit>:<path>` child process used to read file contents at a specific commit. StdoutPipe only fails when the OS cannot create the pipe (os.Pipe error), which in practice means file-descriptor exhaustion (EMFILE). It says nothing about git itself — the command has not started yet.","triggerScenarios":"Scanning git history (`gitleaks git` / detect on a repo) with enough fragment/concurrency pressure to exhaust the process fd limit; low `ulimit -n`; code embedding gitleaks that leaks blob readers without closing them so NewBlobReaderContext eventually cannot allocate a pipe.","commonSituations":"Containers and CI runners with the common 1024 nofile limit; large monorepos with many commits × files; embedding gitleaks as a library in a long-lived service that does not close every io.ReadCloser returned by NewBlobReader/NewBlobReaderContext.","solutions":["Raise the fd limit before scanning: `ulimit -n 8192` (or set LimitNOFILE= in systemd / nofile in the container runtime)","Reduce scan concurrency or split the history into smaller commit ranges","Check for fd leaks while scanning: `ls /proc/$(pgrep gitleaks)/fd | wc -l`; ensure every blob reader is closed with defer rc.Close()","Update gitleaks — reader lifecycle around blobReader has been tightened across versions"],"exampleFix":"# before\n$ ulimit -n\n1024\n$ gitleaks git --source ./big-repo -v   # failed to get stdout pipe: too many open files\n\n# after\n$ ulimit -n 8192\n$ gitleaks git --source ./big-repo -v","handlingStrategy":"retry","validationCode":"// Check fd headroom before a git-history scan (Linux).\nfunc fdHeadroomOK(min uint64) bool {\n\tfds, err := os.ReadDir(\"/proc/self/fd\")\n\tif err != nil {\n\t\treturn true\n\t}\n\tvar rl syscall.Rlimit\n\tif err := syscall.Getrlimit(syscall.RLIMIT_NOFILE, &rl); err != nil {\n\t\treturn true\n\t}\n\treturn uint64(len(fds))+min < rl.Cur\n}","typeGuard":null,"tryCatchPattern":"if rc, err := gitCmd.NewBlobReaderContext(ctx, commit, path); err != nil {\n\tif errors.Is(err, syscall.EMFILE) {\n\t\t// back off and retry after closing in-flight readers / lowering concurrency\n\t\ttime.Sleep(backoff)\n\t\treturn retry()\n\t}\n\treturn err\n}","preventionTips":["Raise ulimit -n (or container nofile / systemd LimitNOFILE) above expected open pipes","Always `defer rc.Close()` every reader returned by NewBlobReader/NewBlobReaderContext","Cap scan concurrency so pipes are bounded well under the fd limit","Watch /proc/<pid>/fd growth during embedding use to catch leaks early"],"tags":["gitleaks","git","exec","file-descriptors","resource-exhaustion"],"backgroundTag":null,"analyzedSha":"b58d3f102cf3a2c84cb7f923d05c25c9b1aed84b","analyzedAt":"2026-08-15T22:17:28.846Z","schemaVersion":2},"datasetVersion":"2026-08-16T03:17:38.424Z"}