What causes "Directory does not exist: {source_ref}" in crewAIInc/crewAI?

Verify the path exists before loading: os.path.isdir(path) or Path(path).resolve() to catch cwd-relative mistakes.

crewAIInc/crewAI · error · FileNotFoundError

Directory does not exist: {source_ref}

Error message

Directory does not exist: {source_ref}

What it means

Raised by DirectoryLoader.load() when the local directory path given to the loader does not exist on disk. The loader explicitly rejects URLs first, then checks os.path.exists(source_ref); a missing path raises FileNotFoundError before any file collection starts. It is a fail-fast guard so callers learn immediately that the source directory is wrong rather than getting an empty result.

Source

Thrown at lib/crewai-tools/src/crewai_tools/rag/loaders/directory_loader.py:29

        """Load and process all files from a directory recursively.

        Args:
            source_content: Directory path or URL to a directory listing
            **kwargs: Additional options:
                - recursive: bool (default True) - Whether to search recursively
                - include_extensions: list - Only include files with these extensions
                - exclude_extensions: list - Exclude files with these extensions
                - max_files: int - Maximum number of files to process
        """
        source_ref = source_content.source_ref

        if source_content.is_url():
            raise ValueError(
                "URL directory loading is not supported. Please provide a local directory path."
            )

        if not os.path.exists(source_ref):
            raise FileNotFoundError(f"Directory does not exist: {source_ref}")

        if not os.path.isdir(source_ref):
            raise ValueError(f"Path is not a directory: {source_ref}")

        return self._process_directory(source_ref, kwargs)

    def _process_directory(self, dir_path: str, kwargs: dict[str, Any]) -> LoaderResult:
        recursive: bool = kwargs.get("recursive", True)
        include_extensions: list[str] | None = kwargs.get("include_extensions", None)
        exclude_extensions: list[str] | None = kwargs.get("exclude_extensions", None)
        max_files: int | None = kwargs.get("max_files", None)

        files = self._find_files(
            dir_path, recursive, include_extensions, exclude_extensions
        )

        if max_files is not None and len(files) > max_files:
            files = files[:max_files]

View on GitHub (pinned to 754d7323be)

Solutions

Verify the path exists before loading: os.path.isdir(path) or Path(path).resolve() to catch cwd-relative mistakes.
If containerized, mount the directory into the container and confirm the mount target matches the path passed to the loader.
Use absolute paths (Path(__file__).parent / 'data') instead of strings that depend on process cwd.
Check for typos, trailing whitespace, and shell-expansion artifacts (~ unexpanded, escaped spaces) in the configured path.

Example fix

# before
loader = DirectoryLoader()
result = loader.load(SourceContent('data/docs'))

# after
from pathlib import Path
p = Path('data/docs').resolve()
if not p.is_dir():
    raise FileNotFoundError(f'docs dir missing: {p}')
result = loader.load(SourceContent(str(p)))

Defensive patterns

Strategy: validation

Validate before calling

import os

def assert_source_dir(path: str) -> None:
    if not os.path.exists(path):
        raise FileNotFoundError(f'Directory does not exist: {path}')
    if not os.path.isdir(path):
        raise NotADirectoryError(f'Not a directory: {path}')

Try / catch

try:\n    result = loader.load(source)\nexcept FileNotFoundError:\n    logger.warning('skipping missing dir: %s', source.source_ref)\n    result = None

Prevention

Resolve paths to absolute form before constructing SourceContent.
In containers, verify mounts with a startup check before any load call.
Never trust raw config strings; expand user/env tokens first.

When it happens

Trigger: Calling DirectoryLoader().load(SourceContent('/path/that/does/not/exist')) or passing a directory string with a typo, a relative path resolved against the wrong working directory, or a path from config/env that is not mounted in the container.

Common situations: Running CrewAI tools inside Docker where the data directory is not volume-mounted; CI jobs where fixtures are generated after the loader runs; relative paths that work locally but break when the process cwd changes; paths built from environment variables that are unset in production.

Related errors

AI-assisted analysis of crewAIInc/crewAI@754d7323be (2026-08-15). Data as JSON: /api/errors/d1d2c52053f8da6e. Report an issue: GitHub.