crewAIInc/crewAI · error · ValueError

Error loading chunker for {self}: {e}

Error message

Error loading chunker for {self}: {e}

What it means

get_chunker() dynamically imports crewai_tools.rag.chunkers.<module> and instantiates <Class>; any exception during import or construction (missing optional dependency like langchain_text_splitters, ImportError inside the chunker module, TypeError from constructor args) is wrapped as ValueError('Error loading chunker for <DataType>: <cause>'). The root cause is in the chained exception.

Source

Thrown at lib/crewai-tools/src/crewai_tools/rag/data_types.py:62

            DataType.DIRECTORY: ("text_chunker", "TextChunker"),
            DataType.YOUTUBE_VIDEO: ("text_chunker", "TextChunker"),
            DataType.YOUTUBE_CHANNEL: ("text_chunker", "TextChunker"),
            DataType.GITHUB: ("text_chunker", "TextChunker"),
            DataType.DOCS_SITE: ("text_chunker", "TextChunker"),
            DataType.MYSQL: ("text_chunker", "TextChunker"),
            DataType.POSTGRES: ("text_chunker", "TextChunker"),
        }

        if self not in chunkers:
            raise ValueError(f"No chunker defined for {self}")
        module_name, class_name = chunkers[self]
        module_path = f"crewai_tools.rag.chunkers.{module_name}"

        try:
            module = import_module(module_path)
            return cast(BaseChunker, getattr(module, class_name)())
        except Exception as e:
            raise ValueError(f"Error loading chunker for {self}: {e}") from e

    def get_loader(self) -> BaseLoader:
        loaders = {
            DataType.PDF_FILE: ("pdf_loader", "PDFLoader"),
            DataType.TEXT_FILE: ("text_loader", "TextFileLoader"),
            DataType.TEXT: ("text_loader", "TextLoader"),
            DataType.XML: ("xml_loader", "XMLLoader"),
            DataType.WEBSITE: ("webpage_loader", "WebPageLoader"),
            DataType.MDX: ("mdx_loader", "MDXLoader"),
            DataType.JSON: ("json_loader", "JSONLoader"),
            DataType.DOCX: ("docx_loader", "DOCXLoader"),
            DataType.CSV: ("csv_loader", "CSVLoader"),
            DataType.DIRECTORY: ("directory_loader", "DirectoryLoader"),
            DataType.YOUTUBE_VIDEO: ("youtube_video_loader", "YoutubeVideoLoader"),
            DataType.YOUTUBE_CHANNEL: (
                "youtube_channel_loader",
                "YoutubeChannelLoader",
            ),

View on GitHub (pinned to 754d7323be)

Solutions

1. Inspect e.__cause__ to see the underlying import/construction error
2. Install the missing optional dependency it names (e.g. uv add <package>)
3. Reinstall crewai-tools cleanly if modules are missing: uv sync / pip install --force-reinstall crewai-tools
4. Pin a known-good crewai-tools version

Example fix

# before
try:
    chunker = dtype.get_chunker()
except ValueError as e:
    raise
# after
try:
    chunker = dtype.get_chunker()
except ValueError as e:
    logger.error('chunker load failed: %s', e.__cause__)
    # e.g. cause says ModuleNotFoundError: install the named package
    raise

try:
    from crewai_tools.rag.chunkers.text_chunker import TextChunker  # noqa: F401
except ImportError as e:
    raise SystemExit(f'missing chunker dependency: {e}')

try:
    chunker = dtype.get_chunker()
except ValueError as e:
    cause = str(e.__cause__ or '')
    if 'No module named' in cause:
        raise SystemExit(f'install missing dependency: {cause}') from e
    raise

When it happens

Trigger: A chunker module importing an optional heavy dependency that is not installed (e.g. a website/youtube chunker needing extra packages); a broken install where a chunker submodule is missing; constructor raising because of a missing third-party lib.

Common situations: Slim installs of crewai-tools without rag extras; partial/failed pip installs; upgrading crewai-tools where chunker module deps changed.

Related errors

• Error loading loader for {self}: {e}
• pypdf is required for PDF chunking
• Chunk overlap ({chunk_overlap}) cannot be >= chunk size ({ch
• No chunker defined for {self}
• CrewAI embedding providers not available. Make sure crewai i

AI-assisted analysis of crewAIInc/crewAI@754d7323be (2026-08-15). Data as JSON: /api/errors/4380aa511abf1637. Report an issue: GitHub.