alibaba/open-code-review · warning

[ocr] WARNING: skipping %s (%d bytes exceeds %d-byte scan li

Error message

[ocr] WARNING: skipping %s (%d bytes exceeds %d-byte scan limit; raise MaxTokens if the real concern is token budget, not memory)

What it means

The scan provider's file enumeration (Enumerate, used by Run and preview) skips any regular file larger than maxFileSizeBytes and prints this warning to stderr. It is a budget guard: oversized files would blow the token limit / memory, so they are excluded from review rather than failing the scan.

Source

Thrown at internal/scan/provider.go:116

			return nil, err
		}
		if rel == "" {
			continue
		}
		if diff.IsPathExcluded(p.repoDir, rel, gitignorePatterns) {
			continue
		}
		full := filepath.Join(p.repoDir, rel)
		info, err := os.Lstat(full)
		if err != nil {
			fmt.Fprintf(os.Stderr, "[ocr] WARNING: cannot stat %s: %v\n", rel, err)
			continue
		}
		if !info.Mode().IsRegular() {
			continue
		}
		if info.Size() > p.maxFileSizeBytes {
			fmt.Fprintf(os.Stderr, "[ocr] WARNING: skipping %s (%d bytes exceeds %d-byte scan limit; raise MaxTokens if the real concern is token budget, not memory)\n",
				rel, info.Size(), p.maxFileSizeBytes)
			continue
		}
		binary, err := isBinaryFile(full)
		if err != nil {
			fmt.Fprintf(os.Stderr, "[ocr] WARNING: cannot sniff %s: %v\n", rel, err)
			continue
		}
		if binary {
			// Emit placeholder so preview can display [B], but do not
			// read the file body — saves memory on large binaries.
			out = append(out, model.ScanItem{
				Path:     rel,
				IsBinary: true,
			})
			continue
		}
		content, err := os.ReadFile(full)

View on GitHub (pinned to 5cf97d0d15)

Solutions

1. Raise the MaxTokens setting in your config if token budget, not memory, is the concern (the message explicitly suggests this).
2. Add the oversized file to exclude/ignore rules if it never needs review, silencing the repeated warning.
3. Split or minify the offending file, or point the scan at a narrower directory that excludes it.
4. Accept the skip if the file is unreviewable anyway (generated artifacts); the warning is informational and the scan continues.

Example fix

// before
provider := scan.NewProvider(scan.Options{})
// after
provider := scan.NewProvider(scan.Options{MaxTokens: 200000}) // raises maxFileSizeBytes so large files are scanned

info, err := os.Stat(path)
if err == nil && info.Mode().IsRegular() && info.Size() > maxFileSizeBytes {
    fmt.Fprintf(os.Stderr, "pre-check: %s (%d bytes) will be skipped\n", path, info.Size())
}

func scannable(info os.FileInfo, maxFileSizeBytes int64) bool {
    return info.Mode().IsRegular() && info.Size() <= maxFileSizeBytes
}

When it happens

Trigger: Calling scanner.Run or preview on a repository containing a regular file whose os.Stat size exceeds p.maxFileSizeBytes (derived from MaxTokens).

Common situations: Repo containing vendored minified JS, generated lockfiles (package-lock.json, go.sum at extreme size), datasets, or binaries > the size cap; lowering MaxTokens via config shrinks the cap and newly triggers skips.

Understand the failure class

Background: "File too large" / "file size exceeds limit" errors: why libraries cap file sizes and how to fix them — this error's family across 46 libraries.

Related errors

• background file %q is %d bytes, exceeding the maximum of %d
• scan failed: %w
• scan preview failed: %w
• file too large (%d bytes, max %d)
• scan: max_tokens must be positive

AI-assisted analysis of alibaba/open-code-review@5cf97d0d15 (2026-09-02). Data as JSON: /api/errors/3a3186013edf84f2. Report an issue: GitHub.