vxcontrol/pentagi · error

file '%s' size %d exceeds maximum allowed size %d

Error message

file '%s' size %d exceeds maximum allowed size %d

What it means

readFileFromContainer enforces a hard 100 MB cap (maxReadFileSize) on any single file pulled out of the container via CopyFromContainer. If the tar header reports a size above the cap the read is refused with this error, protecting the caller (an LLM tool result) and host memory from unbounded allocations (`make([]byte, tarHeader.Size)`).

Source

Thrown at backend/pkg/tools/terminal.go:406

			return "", fmt.Errorf("failed to read tar header: %w", err)
		}

		if tarHeader.FileInfo().IsDir() {
			continue
		}

		if stats.Mode.IsDir() {
			buffer.WriteString("--------------------------------------------------\n")
			buffer.WriteString(
				fmt.Sprintf("'%s' file content (with size %d bytes) shown below:\n",
					tarHeader.Name, tarHeader.Size,
				),
			)
		}

		const maxReadFileSize int64 = 100 * 1024 * 1024 // 100 MB limit
		if tarHeader.Size > maxReadFileSize {
			return "", fmt.Errorf("file '%s' size %d exceeds maximum allowed size %d", tarHeader.Name, tarHeader.Size, maxReadFileSize)
		}
		if tarHeader.Size < 0 {
			return "", fmt.Errorf("file '%s' has invalid size %d", tarHeader.Name, tarHeader.Size)
		}

		var fileContent = make([]byte, tarHeader.Size)
		_, err = tarReader.Read(fileContent)
		if err != nil && err != io.EOF {
			return "", fmt.Errorf("failed to read file '%s' content: %w", tarHeader.Name, err)
		}
		buffer.Write(fileContent)

		if stats.Mode.IsDir() {
			buffer.WriteString("\n\n")
		}
	}

	return buffer.String(), nil

View on GitHub (pinned to ea665308ba)

Solutions

1. Read only what you need: `docker exec <container> head/tail/grep -m1 -c ...` or `sed -n '1,100p' <path>` instead of the whole file
2. Compress the file inside the container first (`gzip -c file > file.gz`) and read the smaller artifact, or split it (`split -b 50m`)
3. If the size is legitimately needed, copy the file out via a volume mount or `docker cp` on the host instead of the tool API
4. Raise maxReadFileSize in terminal.go only with an explicit memory budget — the buffer is allocated fully in RAM

Example fix

// before
content, _ := tool.ReadFile(ctx, flowID, "/captures/full.pcap")
// after
out, _ := t.exec(ctx, flowID, "tcpdump -r /captures/full.pcap -c 1000 -nn") // bounded excerpt
// or inside terminal.go: stream in chunks instead of make([]byte, tarHeader.Size)

size, err := executor.Run(ctx, flowID, fmt.Sprintf("stat -c %%s %s", path))
n, _ := strconv.ParseInt(strings.TrimSpace(size), 10, 64)
if n > 100*1024*1024 {
    return fmt.Errorf("file %s is %d bytes; read a subset instead", path, n)
}

func withinReadLimit(size int64) bool {
    const maxReadFileSize int64 = 100 * 1024 * 1024
    return size >= 0 && size <= maxReadFileSize
}

content, err := tool.ReadFile(ctx, flowID, path)
if err != nil && strings.Contains(err.Error(), "exceeds maximum allowed size") {
    // fall back to head/tail/grep for a bounded excerpt
    content, err = tool.Execute(ctx, flowID, fmt.Sprintf("head -c 1M %s", path))
}

When it happens

Trigger: ReadFile or EditFile invoked on a file whose tar-reported size exceeds 104857600 bytes — e.g. large packet captures, core dumps, extracted archives, log files, or datasets accumulated in the sandbox container during a pentest.

Common situations: An agent tool call tries to read a multi-hundred-MB pcap or wordlist output; a binary artifact (memory dump, image) grew during the flow; EditFile targets a file that was appended to beyond the limit between reads.

Understand the failure class

Background: "File too large" / "file size exceeds limit" errors: why libraries cap file sizes and how to fix them — this error's family across 46 libraries.

Related errors

• tar archive content serialization failed: %w
• failed to close tar writer: %w
• Internal
• failed to stop flow %d: %w
• failed to create extension %q in schema %q (a privileged use

AI-assisted analysis of vxcontrol/pentagi@ea665308ba (2026-09-01). Data as JSON: /api/errors/9cf93f83567fc825. Report an issue: GitHub.