binary-husky/gpt_academic · warning · ValueError

文件大小 ({file_size_mb:.1f}MB) 超过限制 {max_size_mb}MB

Error message

文件大小 ({file_size_mb:.1f}MB) 超过限制 {max_size_mb}MB

What it means

Raised by PaperMetadataExtractor._validate_file when the file's size in MB exceeds the max_size_mb parameter (computed from path.stat().st_size / 1024^2). It protects downstream parsers (and memory) from huge PDFs/Word files before extraction begins.

Source

Thrown at crazy_functions/doc_fns/read_fns/unstructured_all/paper_metadata_extractor.py:114

        Raises:
            ValueError: 文件不存在、格式不支持或大小超限
            PermissionError: 没有读取权限
        """
        path = Path(file_path).resolve()

        if not path.exists():
            raise ValueError(f"文件不存在: {path}")

        if not path.is_file():
            raise ValueError(f"不是文件: {path}")

        if not os.access(path, os.R_OK):
            raise PermissionError(f"没有读取权限: {path}")

        file_size_mb = path.stat().st_size / (1024 * 1024)
        if file_size_mb > max_size_mb:
            raise ValueError(
                f"文件大小 ({file_size_mb:.1f}MB) 超过限制 {max_size_mb}MB"
            )

        if path.suffix.lower() not in self.SUPPORTED_EXTENSIONS:
            raise ValueError(
                f"不支持的文件格式: {path.suffix}. "
                f"支持的格式: {', '.join(sorted(self.SUPPORTED_EXTENSIONS))}"
            )

        return path

    def _cleanup_text(self, text: str) -> str:
        """清理文本

        Args:
            text: 原始文本

        Returns:

View on GitHub (pinned to d6bde0fa54)

Solutions

1. Raise max_size_mb when constructing/calling the extractor if your hardware can handle larger files.
2. Split or compress the document (e.g. qpdf/gs to drop embedded images) before feeding it in.
3. Pre-filter oversized files in the upload pipeline and give the user a clear message instead of letting the raise hit.
4. If large files are routine, move to a streaming/chunked extraction pipeline rather than raising the cap.

Example fix

# before
result = extractor.extract_metadata(fp)  # ValueError: size over limit

# after
MB = 1024 * 1024
if fp.stat().st_size > 200 * MB:
    raise HTTPException(413, 'file too large')
result = extractor.extract_metadata(fp, max_size_mb=200)

MB = 1024 * 1024
if os.path.getsize(fp) / MB > max_size_mb:
    reject_early(f'{fp} exceeds {max_size_mb}MB')

def within_size_limit(path: str, limit_mb: float) -> bool:
    import os
    return os.path.getsize(path) / (1024 * 1024) <= limit_mb

try:
    extractor.extract_metadata(fp)
except ValueError as e:
    if '超过限制' in str(e):
        return too_large_response()  # 413 to user
    raise

When it happens

Trigger: Passing a PDF/DOCX/tex file larger than max_size_mb (default configured on the extractor) to the metadata-extraction or read API; e.g. a 300MB scanned PDF against a 50MB default limit.

Common situations: Scanned-image PDFs of whole books; datasets of preprints where a few are huge; the default limit left unchanged while users upload bigger assets; MB vs MiB confusion right at the boundary (size marginally over the limit).

Related errors

• 文件大小 ({file_size_mb:.1f}MB) 超过限制 {max_size_mb}MB
• File size ({file_size_mb:.1f}MB) exceeds limit of {max_size_
• 文件不存在: {path}
• 不是一个文件: {path}
• 没有读取权限: {path}

AI-assisted analysis of binary-husky/gpt_academic@d6bde0fa54 (2026-08-14). Data as JSON: /api/errors/c13a5a936aca4278. Report an issue: GitHub.