crewAIInc/crewAI · error · ProcessingDependencyError

pypdf is required for PDF chunking

Error message

pypdf is required for PDF chunking

What it means

chunk_pdf() lazily imports pypdf (PdfReader/PdfWriter) and raises ProcessingDependencyError('pypdf is required for PDF chunking') with dependency='pypdf' when it is missing. Chunking activates when a PDF has more pages than max_pages (or when FileHandling.CHUNK is selected), so environments without pypdf only fail when a multi-chunk PDF arrives.

Source

Thrown at lib/crewai-files/src/crewai_files/processing/transformers.py:183

    """Split a PDF into chunks of maximum page count.

    Yields chunks one at a time to minimize memory usage.

    Args:
        file: The PDF file to chunk.
        max_pages: Maximum pages per chunk.
        overlap_pages: Number of overlapping pages between chunks (for context).

    Yields:
        PDFFile objects, one per chunk.

    Raises:
        ProcessingDependencyError: If pypdf is not installed.
    """
    try:
        from pypdf import PdfReader, PdfWriter
    except ImportError as e:
        raise ProcessingDependencyError(
            "pypdf is required for PDF chunking",
            dependency="pypdf",
            install_command="pip install pypdf",
        ) from e

    content = file.read()
    reader = PdfReader(io.BytesIO(content))
    total_pages = len(reader.pages)

    if total_pages <= max_pages:
        yield file
        return

    filename = file.filename or "document.pdf"
    base_filename = filename.rsplit(".", 1)[0]
    step = max_pages - overlap_pages

    chunk_num = 0

View on GitHub (pinned to 754d7323be)

Solutions

1. Install pypdf: pip install pypdf.
2. Include pypdf in the deployment dependency set whenever CHUNK mode or PDF limits are configured.
3. Check importlib.util.find_spec('pypdf') at startup and warn/abort early if CHUNK is configured without it.
4. Alternatively split PDFs upstream before ingestion.

Example fix

# before
# pypdf not installed; pdf has 120 pages, max_pages=50
chunks = list(chunk_pdf(pdf_file, max_pages=50))  # ProcessingDependencyError

# after
# shell: pip install pypdf
chunks = list(chunk_pdf(pdf_file, max_pages=50))

import importlib.util

if importlib.util.find_spec("pypdf") is None and handling == FileHandling.CHUNK:
    raise RuntimeError("PDF chunking requires pypdf: pip install pypdf")

from crewai_files.processing.exceptions import ProcessingDependencyError

try:
    chunks = list(chunk_pdf(pdf_file, max_pages=50))
except ProcessingDependencyError as e:
    if e.dependency == "pypdf":
        raise RuntimeError(f"missing optional dependency; run: {e.install_command}") from e
    raise

When it happens

Trigger: Processing a PDF whose page count exceeds max_pages with FileHandling.CHUNK (or calling chunk_pdf directly) while pypdf is not installed. PDFs within the limit yield the original file early and never hit the import.

Common situations: Enabling CHUNK handling for large documents after testing only with small PDFs; production image lacking pypdf because it is optional; long reports/manuals (>50 pages) hitting the chunk path for the first time.

Related errors

• Pillow is required for image resizing
• Pillow is required for image optimization
• PDF '{filename}' page count ({page_count}) exceeds maximum (
• Error loading chunker for {self}: {e}
• The 'couchbase' package is required to use the CouchbaseFTSV

AI-assisted analysis of crewAIInc/crewAI@754d7323be (2026-08-15). Data as JSON: /api/errors/a6fd85e9e76f1c97. Report an issue: GitHub.