{"record":{"id":"3532cfd8debbd15c","repo":"binary-husky/gpt_academic","slug":"10","errorCode":null,"errorMessage":"文件内容提取超时（10秒）","messagePattern":"文件内容提取超时（10秒）","errorType":"exception","errorClass":"TimeoutError","httpStatus":null,"severity":"error","filePath":"crazy_functions/Document_Conversation.py","lineNumber":115,"sourceCode":"            if os.path.getsize(fp) > self.max_file_size:\n                self.failed_files.append((fp, f\"文件过大：超过{self.max_file_size / 1024 / 1024}MB\"))\n                mutable_status[2] = \"文件过大\"\n                return fragments\n\n            # 更新状态\n            mutable_status[0] = \"提取文件内容\"\n            mutable_status[1] = time.time()\n\n            # 提取内容 - 使用单独的超时控制\n            content = None\n            extract_start_time = time.time()\n            try:\n                while True:\n                    check_timeout()  # 检查全局超时\n\n                    # 检查提取过程是否超时（10秒）\n                    if time.time() - extract_start_time > 10:\n                        raise TimeoutError(\"文件内容提取超时（10秒）\")\n\n                    try:\n                        content = extract_text(fp)\n                        break\n                    except Exception as e:\n                        if \"timeout\" in str(e).lower():\n                            continue  # 如果是临时超时，重试\n                        raise  # 其他错误直接抛出\n\n            except Exception as e:\n                self.failed_files.append((fp, f\"文件读取失败：{str(e)}\"))\n                mutable_status[2] = \"读取失败\"\n                return fragments\n\n            if content is None:\n                self.failed_files.append((fp, \"文件解析失败：不支持的格式或文件损坏\"))\n                mutable_status[2] = \"格式不支持\"\n                return fragments","sourceCodeStart":97,"sourceCodeEnd":133,"githubUrl":"https://github.com/binary-husky/gpt_academic/blob/d6bde0fa54373309bd05823a49bda8da019d2c77/crazy_functions/Document_Conversation.py#L97-L133","documentation":"This is the fixed 10-second wall-clock budget for extract_text(fp). The loop retries extractor exceptions whose string contains 'timeout'; once more than 10 seconds have elapsed at the top of the loop, it raises TimeoutError, records the file as read-failed, and continues the batch. The check is cooperative and cannot stop a currently blocked extractor.","triggerScenarios":"SimpleDirectoryReader.load_data() or its fallback repeatedly raises timeout-like exceptions for more than 10 seconds, or control returns to the loop after the extractor exceeded the budget. Large PDFs/PPTX/DOCX files, malformed documents, and slow markitdown conversion are common.","commonSituations":"A complex document near the 10MB limit; scanned PDFs; password-protected or corrupt Office files; high concurrency; Docker with slow volume I/O; a parser dependency hanging rather than enforcing its own timeout.","solutions":["Identify the file from failed_files and test extract_text(fp) in isolation.","Convert or repair the document before upload, and OCR scanned PDFs first.","Increase the extraction budget or make it configurable if the document is valid but large.","Reduce max_workers when many parser subprocesses compete.","Use a process-level timeout around extract_text so a hung parser is terminated instead of retried in a spin loop."],"exampleFix":"# before\nwhile True:\n    if time.time() - extract_start_time > 10:\n        raise TimeoutError(\"文件内容提取超时（10秒）\")\n    try:\n        content = extract_text(fp)\n        break\n\n# after\nwith concurrent.futures.ThreadPoolExecutor(max_workers=1) as executor:\n    future = executor.submit(extract_text, fp)\n    try:\n        content = future.result(timeout=10)\n    except concurrent.futures.TimeoutError as e:\n        raise TimeoutError(\"文件内容提取超时（10秒）\") from e\n","handlingStrategy":"validation","validationCode":"fp = file_info[0]\nif os.path.splitext(fp.lower())[1] not in supports_format:\n    skip_reason = \"unsupported format\"\nelif os.path.getsize(fp) > 10 * 1024 * 1024:\n    skip_reason = \"file too large\"\n","typeGuard":"def is_supported_extract_file(fp: str) -> bool:\n    return os.path.isfile(fp) and os.path.splitext(fp.lower())[1] in supports_format\n","tryCatchPattern":"try:\n    content = extract_text(fp)\nexcept (TimeoutError, concurrent.futures.TimeoutError) as e:\n    record_failed_file(fp, f\"extraction timeout: {e}\")\n    continue\n","preventionTips":["Repair or convert unusual documents before extraction.","OCR scanned PDFs before passing them to a text-only reader.","Measure extraction time for large formats and set a realistic timeout.","Do not retry a deterministic parser failure indefinitely."],"tags":["timeout","text-extraction","llama-index","documents"],"backgroundTag":null,"analyzedSha":"d6bde0fa54373309bd05823a49bda8da019d2c77","analyzedAt":"2026-08-14T22:48:35.038Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}