unclecode/crawl4ai · warning · ValueError

Invalid wait_for parameter: '{wait_for}'. It should be eithe

Error message

Invalid wait_for parameter: '{wait_for}'. It should be either a valid CSS selector, a JavaScript function, or explicitly prefixed with 'js:' or 'css:'.

What it means

ArtifactNotFound raised when a path for the id exists but lstat shows it is not a regular file — i.e. a symlink or device node sits where the artifact should be. The store is written with O_EXCL|O_NOFOLLOW and 0o600, so this state indicates tampering or filesystem corruption, and it is deliberately indistinguishable from a missing artifact.

Source

Thrown at crawl4ai/async_crawler_strategy.py:292

                return await self.csp_compliant_wait(page, wait_for, timeout)
            else:
                # Assume it's a CSS selector first
                try:
                    await page.wait_for_selector(wait_for, timeout=timeout)
                except Error as e:
                    if "Timeout" in str(e):
                        raise TimeoutError(
                            f"Timeout after {timeout}ms waiting for selector '{wait_for}'"
                        )
                    else:
                        # If it's not a timeout error, it might be an invalid selector
                        # Let's try to evaluate it as a JavaScript function as a fallback
                        try:
                            return await self.csp_compliant_wait(
                                page, f"() => {{{wait_for}}}", timeout
                            )
                        except Error:
                            raise ValueError(
                                f"Invalid wait_for parameter: '{wait_for}'. "
                                "It should be either a valid CSS selector, a JavaScript function, "
                                "or explicitly prefixed with 'js:' or 'css:'."
                            )

    async def csp_compliant_wait(
        self, page: Page, user_wait_function: str, timeout: float = 30000
    ):
        """
        Wait for a condition in a CSP-compliant way.

        Args:
            page: Playwright page object
            user_wait_function: JavaScript function as string that returns boolean
            timeout: Maximum time to wait in milliseconds

        Returns:
            bool: True if condition was met, False if timed out

View on GitHub (pinned to 7e80152142)

Solutions

Inspect the artifact directory and remove symlinks/special files — only regular files created by write_artifact belong there
Ensure ARTIFACT_DIR points to a directory the server exclusively owns (not shared with other tooling)
Regenerate the artifact via the crawl API; do not hand-place files into the store

Defensive patterns

Strategy: validation

Validate before calling

# preflight: ensure the store contains only regular files
import os, stat
def store_is_clean(artifact_dir):
    return all(stat.S_ISREG(e.stat(follow_symlinks=False).st_mode)
               for e in os.scandir(artifact_dir))

Try / catch

try:
    path, mime = resolve_artifact(artifact_id)
except ArtifactNotFound:
    audit_store_for_symlinks(ARTIFACT_DIR)  # surface tampering to ops
    raise

Prevention

Give the artifact directory exclusively to the server process
Never hand-place or link files into the store
Alert on this pattern in prod: a non-regular file may indicate tampering

When it happens

Trigger: Someone replaces an artifact file in the store directory with a symlink (or the volume is corrupted); resolve_artifact lstats the candidate path, sees a non-regular mode, and raises ArtifactNotFound.

Common situations: A mounted volume with pre-existing symlinked files; an operator or another process 'organizing' the artifact dir; container image layers introducing links into the store path.

Related errors

AI-assisted analysis of unclecode/crawl4ai@7e80152142 (2026-08-14). Data as JSON: /api/errors/55a32e7d0203cd77. Report an issue: GitHub.