binary-husky/gpt_academic · warning · ValueError

File size ({file_size_mb:.1f}MB) exceeds limit of {max_size_

Error message

File size ({file_size_mb:.1f}MB) exceeds limit of {max_size_mb}MB

What it means

UnstructuredReader's English size guard: ValueError raised when the file's size in MB exceeds max_size_mb. It fires after existence/is_file/read checks and before the extension allowlist check.

Source

Thrown at crazy_functions/doc_fns/read_fns/unstructured_all/unstructured_reader.py:125

        Raises:
            ValueError: 文件不存在、格式不支持或大小超限
            PermissionError: 没有读取权限
        """
        path = Path(file_path).resolve()

        if not path.exists():
            raise ValueError(f"File not found: {path}")

        if not path.is_file():
            raise ValueError(f"Not a file: {path}")

        if not os.access(path, os.R_OK):
            raise PermissionError(f"No read permission: {path}")

        file_size_mb = path.stat().st_size / (1024 * 1024)
        if file_size_mb > max_size_mb:
            raise ValueError(
                f"File size ({file_size_mb:.1f}MB) exceeds limit of {max_size_mb}MB"
            )

        if path.suffix.lower() not in self.SUPPORTED_EXTENSIONS:
            raise ValueError(
                f"Unsupported format: {path.suffix}. "
                f"Supported: {', '.join(sorted(self.SUPPORTED_EXTENSIONS))}"
            )

        return path

    def _cleanup_text(self, text: str) -> str:
        """清理文本

        Args:
            text: 原始文本

        Returns:

View on GitHub (pinned to d6bde0fa54)

Solutions

1. Increase max_size_mb on the call/constructor if your environment can afford the memory.
2. Compress or split the document (ghostscript -dPDFSETTINGS=/ebook, qpdf --split-pages).
3. Reject oversized uploads earlier in the flow with a friendly message (413-style).
4. Stream large files through a lighter extractor instead of the full unstructured pipeline.

Example fix

# before
text = reader.read(fp)  # ValueError: exceeds limit

# after
LIMIT_MB = 200
size_mb = fp.stat().st_size / (1024 * 1024)
text = reader.read(fp, max_size_mb=max(LIMIT_MB, int(size_mb) + 1)) if size_mb < 500 else None
if text is None:
    raise HTTPException(413, 'file too large to process')

MB = 1024 * 1024
size_mb = os.path.getsize(fp) / MB
if size_mb > limit:
    return reject_413(fp, size_mb, limit)

def size_ok(path: str, limit_mb: float) -> bool:
    import os
    return os.path.getsize(path) / (1024 * 1024) <= limit_mb

try:
    reader.read(fp)
except ValueError as e:
    if 'exceeds limit' in str(e):
        split_or_compress_then_retry(fp)
    raise

When it happens

Trigger: Calling the reader with a document larger than the configured max_size_mb — huge scanned PDFs, Word files full of embedded media, or accidentally pointing at a bundled archive-like file that is actually parsed as a document.

Common situations: Default limit too small for real corpora; scanned thesis PDFs at hundreds of MB; users zipping content or embedding media; memory pressure or OOM that motivated the limit being forgotten when it blocks a legitimate file.

Related errors

• 文件大小 ({file_size_mb:.1f}MB) 超过限制 {max_size_mb}MB
• 文件大小 ({file_size_mb:.1f}MB) 超过限制 {max_size_mb}MB
• 没有读取权限: {path}
• 不支持的文件格式: {path.suffix}. 支持的格式: {', '.join(sorted(self.SUPPO
• File not found: {path}

AI-assisted analysis of binary-husky/gpt_academic@d6bde0fa54 (2026-08-14). Data as JSON: /api/errors/eee3c50fa92ac1f5. Report an issue: GitHub.