gitleaks/gitleaks · error

failed to get stdout pipe: %w

Error message

failed to get stdout pipe: %w

What it means

Returned by GitCmd.NewBlobReaderContext (sources/git.go:213) when cmd.StdoutPipe() fails for the `git -C <repo> cat-file blob <commit>:<path>` child process used to read file contents at a specific commit. StdoutPipe only fails when the OS cannot create the pipe (os.Pipe error), which in practice means file-descriptor exhaustion (EMFILE). It says nothing about git itself — the command has not started yet.

Source

Thrown at sources/git.go:213

}

// NewBlobReader returns an io.ReadCloser that can be used to read a blob
// within the git repo used to create the GitCmd.
//
// The caller is responsible for closing the reader.
func (c *GitCmd) NewBlobReader(commit, path string) (io.ReadCloser, error) {
	return c.NewBlobReaderContext(context.Background(), commit, path)
}

// NewBlobReaderContext is the same as NewBlobReader but supports passing in a
// context to use for timeouts
func (c *GitCmd) NewBlobReaderContext(ctx context.Context, commit, path string) (io.ReadCloser, error) {
	gitArgs := []string{"-C", c.repoPath, "cat-file", "blob", commit + ":" + path}
	cmd := exec.CommandContext(ctx, "git", gitArgs...)
	cmd.Stderr = io.Discard
	stdout, err := cmd.StdoutPipe()
	if err != nil {
		return nil, fmt.Errorf("failed to get stdout pipe: %w", err)
	}
	if err := cmd.Start(); err != nil {
		return nil, fmt.Errorf("failed to start git command: %w", err)
	}
	return &blobReader{
		ReadCloser: stdout,
		cmd:        cmd,
	}, nil
}

// listenForStdErr listens for stderr output from git, prints it to stdout,
// sends to errCh and closes it.
func listenForStdErr(stderr io.ReadCloser, errCh chan<- error) {
	defer close(errCh)

	var errEncountered bool

	scanner := bufio.NewScanner(stderr)

View on GitHub (pinned to b58d3f102c)

Solutions

1. Raise the fd limit before scanning: `ulimit -n 8192` (or set LimitNOFILE= in systemd / nofile in the container runtime)
2. Reduce scan concurrency or split the history into smaller commit ranges
3. Check for fd leaks while scanning: `ls /proc/$(pgrep gitleaks)/fd | wc -l`; ensure every blob reader is closed with defer rc.Close()
4. Update gitleaks — reader lifecycle around blobReader has been tightened across versions

Example fix

# before
$ ulimit -n
1024
$ gitleaks git --source ./big-repo -v   # failed to get stdout pipe: too many open files

# after
$ ulimit -n 8192
$ gitleaks git --source ./big-repo -v

// Check fd headroom before a git-history scan (Linux).
func fdHeadroomOK(min uint64) bool {
	fds, err := os.ReadDir("/proc/self/fd")
	if err != nil {
		return true
	}
	var rl syscall.Rlimit
	if err := syscall.Getrlimit(syscall.RLIMIT_NOFILE, &rl); err != nil {
		return true
	}
	return uint64(len(fds))+min < rl.Cur
}

if rc, err := gitCmd.NewBlobReaderContext(ctx, commit, path); err != nil {
	if errors.Is(err, syscall.EMFILE) {
		// back off and retry after closing in-flight readers / lowering concurrency
		time.Sleep(backoff)
		return retry()
	}
	return err
}

When it happens

Trigger: Scanning git history (`gitleaks git` / detect on a repo) with enough fragment/concurrency pressure to exhaust the process fd limit; low `ulimit -n`; code embedding gitleaks that leaks blob readers without closing them so NewBlobReaderContext eventually cannot allocate a pipe.

Common situations: Containers and CI runners with the common 1024 nofile limit; large monorepos with many commits × files; embedding gitleaks as a library in a long-lived service that does not close every io.ReadCloser returned by NewBlobReader/NewBlobReaderContext.

Related errors

• failed to start git command: %w
• stderr is not empty
• command failed (%d): %w, stderr: %s
• unable to parse remote URL: %w
• the diagnostics directory should not be set in http mode

AI-assisted analysis of gitleaks/gitleaks@b58d3f102c (2026-08-15). Data as JSON: /api/errors/18c753e6f7914b55. Report an issue: GitHub.