unclecode/crawl4ai · error · FileNotFoundError

Local file not found: {path}

Error message

Local file not found: {path}

What it means

Raised by the HTTP-mode crawler's _handle_file() when a file:// URL targets a path that does not exist. It uses urlparse(url).path (not the url[7:] strip used by the browser strategy), streams the file in chunks via aiofiles, and decodes as utf-8 with replacement. The existence check runs before any I/O starts.

Source

Thrown at crawl4ai/async_crawler_strategy.py:2582

            try:
                await asyncio.wait_for(self._session.close(), timeout=5.0)
            except asyncio.TimeoutError:
                if self.logger:
                    self.logger.warning(
                        message="Session cleanup timed out",
                        tag="CLEANUP"
                    )
            finally:
                self._session = None

    async def _stream_file(self, path: str) -> AsyncGenerator[memoryview, None]:
        async with aiofiles.open(path, mode='rb') as f:
            while chunk := await f.read(self.chunk_size):
                yield memoryview(chunk)

    async def _handle_file(self, path: str) -> AsyncCrawlResponse:
        if not os.path.exists(path):
            raise FileNotFoundError(f"Local file not found: {path}")
            
        chunks = []
        async for chunk in self._stream_file(path):
            chunks.append(chunk.tobytes().decode('utf-8', errors='replace'))
            
        return AsyncCrawlResponse(
            html=''.join(chunks),
            response_headers={},
            status_code=200
        )

    async def _handle_raw(self, content: str, base_url: str = None) -> AsyncCrawlResponse:
        return AsyncCrawlResponse(
            html=content,
            response_headers={},
            status_code=200,
            # For raw: URLs, use base_url if provided; don't fall back to the raw content
            redirected_url=base_url

View on GitHub (pinned to 7e80152142)

Solutions

1. Verify existence first with os.path.exists(urllib.parse.unquote(parsed.path)) and build the URL with Path.as_uri().
2. Mount or copy the file into the container/CI environment.
3. Percent-decode the URL path before checking when it may contain encoded characters.
4. Skip and log missing files when crawling a batch of file:// URLs.

Example fix

// before
await crawler.crawler_strategy.crawl("file://data/report.html")

// after
from pathlib import Path
from urllib.parse import unquote, urlparse
path = unquote(urlparse(file_url).path)
if not os.path.exists(path):
    raise FileNotFoundError(path)
await crawler.crawler_strategy.crawl(file_url)

import os
from urllib.parse import urlparse, unquote

def file_url_exists(url: str) -> bool:
    return os.path.isfile(unquote(urlparse(url).path))

try:
    resp = await crawler.crawler_strategy.crawl(file_url)
except FileNotFoundError as e:
    logger.warning(f"skip missing file: {e}")

When it happens

Trigger: Running AsyncWebCrawler/AsyncHTTPCrawler with crawl(url='file:///no/such/file.html'). Note urlparse keeps percent-encoding: a URL with %20 spaces will fail the os.path.exists check unless the path is unquoted; also relative/Windows-style paths.

Common situations: Docker or CI environments where the file is not mounted; URLs built by hand with encoded characters; files removed between queue construction and crawl; sharing fixture paths across machines.

Related errors

• Local file not found: {local_file_path}
• Profile not found: {profile_path}
• Profile not found: {profile_name_or_path}
• Container not found: ${config.container_selector}
• Timeout after {timeout}ms waiting for selector '{wait_for}'

AI-assisted analysis of unclecode/crawl4ai@7e80152142 (2026-08-14). Data as JSON: /api/errors/ce2e1143d9cb23cf. Report an issue: GitHub.