{"record":{"id":"abc43f1bd12999f2","repo":"unclecode/crawl4ai","slug":"unsafe-download-filename-rejected-filename-r","errorCode":null,"errorMessage":"Unsafe download filename rejected: {filename!r}","messagePattern":"Unsafe download filename rejected: (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"crawl4ai/async_crawler_strategy.py","lineNumber":2438,"sourceCode":"def _safe_download_filepath(downloads_path: str, filename: str) -> str:\n    \"\"\"Resolve a download destination confined to ``downloads_path``.\n\n    The filename is derived from attacker-influenced input (a remote\n    Content-Disposition header, or the browser's suggested filename), so it is\n    reduced to a bare basename (dropping absolute paths and ``..`` traversal)\n    and the resolved path is re-checked to live inside the downloads root,\n    rejecting any pre-existing symlink that points outside. Raises ValueError\n    on any escape. The final write must still use ``_nofollow_opener`` (or an\n    equivalent ``O_NOFOLLOW`` / pre-write symlink check) to close the TOCTOU\n    window between this check and the open.\n    \"\"\"\n    safe_name = os.path.basename(filename or \"\")\n    if not safe_name or safe_name in (\".\", \"..\"):\n        safe_name = f\"download_{hashlib.md5((filename or '').encode()).hexdigest()[:10]}\"\n    real_root = os.path.realpath(downloads_path)\n    real_path = os.path.realpath(os.path.join(real_root, safe_name))\n    if os.path.commonpath([real_root, real_path]) != real_root:\n        raise ValueError(f\"Unsafe download filename rejected: {filename!r}\")\n    return real_path\n\n\ndef _nofollow_opener(path, flags):\n    \"\"\"Opener for ``open``/``aiofiles.open`` that refuses to follow a symlink at\n    the final path component, closing the TOCTOU symlink-swap race after a path\n    has been confined by ``_safe_download_filepath``.\"\"\"\n    return os.open(path, flags | os.O_NOFOLLOW)\n\n\nclass HTTPCrawlerError(Exception):\n    \"\"\"Base error class for HTTP crawler specific exceptions\"\"\"\n    pass\n\n\nclass ConnectionTimeoutError(HTTPCrawlerError):\n    \"\"\"Raised when connection timeout occurs\"\"\"\n    pass","sourceCodeStart":2420,"sourceCodeEnd":2456,"githubUrl":"https://github.com/unclecode/crawl4ai/blob/7e801521428ee12509994d39151006f64055ebe3/crawl4ai/async_crawler_strategy.py#L2420-L2456","documentation":"Raised by _safe_download_filepath() when a suggested download filename escapes the downloads root after basename sanitization and realpath resolution. The function takes os.path.basename of the name, joins it to the realpath of downloads_path, and requires the commonpath of root and resolved path to equal the root; any residual escape (e.g. via a pre-existing symlink inside the directory pointing outside) raises ValueError.","triggerScenarios":"A pre-existing symlink named like the suggested filename inside the downloads directory that resolves outside the root (realpath follows it and commonpath check fails). Pure traversal via '../' in the filename is already neutralized by basename(), so the realistic trigger is the symlink-inside-directory case.","commonSituations":"Untrusted archives extracted into the downloads dir creating symlinks; multi-process crawlers sharing a downloads dir with differing roots (one root inside a symlinked path, e.g. /tmp vs /private/tmp on macOS); prior malicious downloads planting symlinks.","solutions":["Remove symlinks inside the downloads directory (find downloads_path -type l -delete) so realpath stays inside the root.","Point downloads_path at a real (non-symlinked) directory, avoiding macOS /tmp -> /private/tmp style mismatches between callers.","Use a per-run or per-process unique downloads directory.","Restrict write permissions on the downloads dir so only the crawler can create entries."],"exampleFix":"// before\n# downloads/data -> /etc/passwd  (symlink planted inside downloads dir)\n# ValueError: Unsafe download filename rejected\n\n// after\nimport os\nfor name in os.listdir(downloads_path):\n    p = os.path.join(downloads_path, name)\n    if os.path.islink(p):\n        os.unlink(p)\nawait crawler.arun(url, config)","handlingStrategy":"validation","validationCode":"import os\n\ndef downloads_dir_clean(path: str) -> bool:\n    rp = os.path.realpath(path)\n    for root, _dirs, files in os.walk(rp):\n        for f in files:\n            if os.path.islink(os.path.join(root, f)):\n                return False\n    return True","typeGuard":null,"tryCatchPattern":"try:\n    await crawler.arun(url, config=cfg)\nexcept ValueError as e:\n    if \"Unsafe download filename\" in str(e):\n        logger.error(f\"possible symlink attack in downloads dir: {e}\")","preventionTips":["Point downloads_path at a real non-symlinked directory","Use per-run downloads directories","Audit the downloads dir for symlinks in scheduled jobs"],"tags":["security","path-traversal","downloads","symlink"],"backgroundTag":null,"analyzedSha":"7e801521428ee12509994d39151006f64055ebe3","analyzedAt":"2026-08-14T20:46:20.673Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}