binary-husky/gpt_academic · error · TimeoutError

文件内容提取超时（10秒）

Error message

文件内容提取超时（10秒）

What it means

This is the fixed 10-second wall-clock budget for extract_text(fp). The loop retries extractor exceptions whose string contains 'timeout'; once more than 10 seconds have elapsed at the top of the loop, it raises TimeoutError, records the file as read-failed, and continues the batch. The check is cooperative and cannot stop a currently blocked extractor.

Source

Thrown at crazy_functions/Document_Conversation.py:115

            if os.path.getsize(fp) > self.max_file_size:
                self.failed_files.append((fp, f"文件过大：超过{self.max_file_size / 1024 / 1024}MB"))
                mutable_status[2] = "文件过大"
                return fragments

            # 更新状态
            mutable_status[0] = "提取文件内容"
            mutable_status[1] = time.time()

            # 提取内容 - 使用单独的超时控制
            content = None
            extract_start_time = time.time()
            try:
                while True:
                    check_timeout()  # 检查全局超时

                    # 检查提取过程是否超时（10秒）
                    if time.time() - extract_start_time > 10:
                        raise TimeoutError("文件内容提取超时（10秒）")

                    try:
                        content = extract_text(fp)
                        break
                    except Exception as e:
                        if "timeout" in str(e).lower():
                            continue  # 如果是临时超时，重试
                        raise  # 其他错误直接抛出

            except Exception as e:
                self.failed_files.append((fp, f"文件读取失败：{str(e)}"))
                mutable_status[2] = "读取失败"
                return fragments

            if content is None:
                self.failed_files.append((fp, "文件解析失败：不支持的格式或文件损坏"))
                mutable_status[2] = "格式不支持"
                return fragments

View on GitHub (pinned to d6bde0fa54)

Solutions

1. Identify the file from failed_files and test extract_text(fp) in isolation.
2. Convert or repair the document before upload, and OCR scanned PDFs first.
3. Increase the extraction budget or make it configurable if the document is valid but large.
4. Reduce max_workers when many parser subprocesses compete.
5. Use a process-level timeout around extract_text so a hung parser is terminated instead of retried in a spin loop.

Example fix

# before
while True:
    if time.time() - extract_start_time > 10:
        raise TimeoutError("文件内容提取超时（10秒）")
    try:
        content = extract_text(fp)
        break

# after
with concurrent.futures.ThreadPoolExecutor(max_workers=1) as executor:
    future = executor.submit(extract_text, fp)
    try:
        content = future.result(timeout=10)
    except concurrent.futures.TimeoutError as e:
        raise TimeoutError("文件内容提取超时（10秒）") from e

fp = file_info[0]
if os.path.splitext(fp.lower())[1] not in supports_format:
    skip_reason = "unsupported format"
elif os.path.getsize(fp) > 10 * 1024 * 1024:
    skip_reason = "file too large"

def is_supported_extract_file(fp: str) -> bool:
    return os.path.isfile(fp) and os.path.splitext(fp.lower())[1] in supports_format

try:
    content = extract_text(fp)
except (TimeoutError, concurrent.futures.TimeoutError) as e:
    record_failed_file(fp, f"extraction timeout: {e}")
    continue

When it happens

Trigger: SimpleDirectoryReader.load_data() or its fallback repeatedly raises timeout-like exceptions for more than 10 seconds, or control returns to the loop after the extractor exceeded the budget. Large PDFs/PPTX/DOCX files, malformed documents, and slow markitdown conversion are common.

Common situations: A complex document near the 10MB limit; scanned PDFs; password-protected or corrupt Office files; high concurrency; Docker with slow volume I/O; a parser dependency hanging rather than enforcing its own timeout.

Related errors

• 处理文件 {os.path.basename(fp)} 超时（{TIMEOUT_SECONDS}秒）
• 文本分割超时（5秒）
• 等待用户输入超时
• Doc2x processing timeout after maximum attempts
• Doc2x conversion timeout after maximum attempts

AI-assisted analysis of binary-husky/gpt_academic@d6bde0fa54 (2026-08-14). Data as JSON: /api/errors/3532cfd8debbd15c. Report an issue: GitHub.