{"record":{"id":"c13a5a936aca4278","repo":"binary-husky/gpt_academic","slug":"file-size-mb-1f-mb-max-size-mb-mb-c13a5a","errorCode":null,"errorMessage":"文件大小 ({file_size_mb:.1f}MB) 超过限制 {max_size_mb}MB","messagePattern":"文件大小 \\((.+?)MB\\) 超过限制 (.+?)MB","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"warning","filePath":"crazy_functions/doc_fns/read_fns/unstructured_all/paper_metadata_extractor.py","lineNumber":114,"sourceCode":"\n        Raises:\n            ValueError: 文件不存在、格式不支持或大小超限\n            PermissionError: 没有读取权限\n        \"\"\"\n        path = Path(file_path).resolve()\n\n        if not path.exists():\n            raise ValueError(f\"文件不存在: {path}\")\n\n        if not path.is_file():\n            raise ValueError(f\"不是文件: {path}\")\n\n        if not os.access(path, os.R_OK):\n            raise PermissionError(f\"没有读取权限: {path}\")\n\n        file_size_mb = path.stat().st_size / (1024 * 1024)\n        if file_size_mb > max_size_mb:\n            raise ValueError(\n                f\"文件大小 ({file_size_mb:.1f}MB) 超过限制 {max_size_mb}MB\"\n            )\n\n        if path.suffix.lower() not in self.SUPPORTED_EXTENSIONS:\n            raise ValueError(\n                f\"不支持的文件格式: {path.suffix}. \"\n                f\"支持的格式: {', '.join(sorted(self.SUPPORTED_EXTENSIONS))}\"\n            )\n\n        return path\n\n    def _cleanup_text(self, text: str) -> str:\n        \"\"\"清理文本\n\n        Args:\n            text: 原始文本\n\n        Returns:","sourceCodeStart":96,"sourceCodeEnd":132,"githubUrl":"https://github.com/binary-husky/gpt_academic/blob/d6bde0fa54373309bd05823a49bda8da019d2c77/crazy_functions/doc_fns/read_fns/unstructured_all/paper_metadata_extractor.py#L96-L132","documentation":"Raised by PaperMetadataExtractor._validate_file when the file's size in MB exceeds the max_size_mb parameter (computed from path.stat().st_size / 1024^2). It protects downstream parsers (and memory) from huge PDFs/Word files before extraction begins.","triggerScenarios":"Passing a PDF/DOCX/tex file larger than max_size_mb (default configured on the extractor) to the metadata-extraction or read API; e.g. a 300MB scanned PDF against a 50MB default limit.","commonSituations":"Scanned-image PDFs of whole books; datasets of preprints where a few are huge; the default limit left unchanged while users upload bigger assets; MB vs MiB confusion right at the boundary (size marginally over the limit).","solutions":["Raise max_size_mb when constructing/calling the extractor if your hardware can handle larger files.","Split or compress the document (e.g. qpdf/gs to drop embedded images) before feeding it in.","Pre-filter oversized files in the upload pipeline and give the user a clear message instead of letting the raise hit.","If large files are routine, move to a streaming/chunked extraction pipeline rather than raising the cap."],"exampleFix":"# before\nresult = extractor.extract_metadata(fp)  # ValueError: size over limit\n\n# after\nMB = 1024 * 1024\nif fp.stat().st_size > 200 * MB:\n    raise HTTPException(413, 'file too large')\nresult = extractor.extract_metadata(fp, max_size_mb=200)","handlingStrategy":"validation","validationCode":"MB = 1024 * 1024\nif os.path.getsize(fp) / MB > max_size_mb:\n    reject_early(f'{fp} exceeds {max_size_mb}MB')","typeGuard":"def within_size_limit(path: str, limit_mb: float) -> bool:\n    import os\n    return os.path.getsize(path) / (1024 * 1024) <= limit_mb","tryCatchPattern":"try:\n    extractor.extract_metadata(fp)\nexcept ValueError as e:\n    if '超过限制' in str(e):\n        return too_large_response()  # 413 to user\n    raise","preventionTips":["Enforce an upload size cap before files hit storage.","Compress scanned PDFs at ingest time.","Keep max_size_mb in config, tuned to your RAM budget.","Log near-miss sizes to spot creeping file sizes."],"tags":["file-size","validation","pdf","python"],"backgroundTag":null,"analyzedSha":"d6bde0fa54373309bd05823a49bda8da019d2c77","analyzedAt":"2026-08-14T22:48:35.038Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}