unclecode/crawl4ai · error · ValueError

Error sanitizing input: {str(e)}

Error message

Error sanitizing input: {str(e)}

What it means

ValueError wrapping any unexpected exception from the input-sanitization helper in utils.py: it first strips/re-encodes text as UTF-8 (falling back to ASCII with a printed warning), and only the outer broad `except Exception` produces this message. Because both inner branches handle Unicode errors, reaching the outer raise indicates a non-encoding failure — most commonly passing a non-string (bytes, None mishandled by a caller, or objects with broken __str__).

Source

Thrown at crawl4ai/utils.py:792

    return sanitized_html


def sanitize_input_encode(text: str) -> str:
    """Sanitize input to handle potential encoding issues."""
    try:
        try:
            if not text:
                return ""
            # Attempt to encode and decode as UTF-8 to handle potential encoding issues
            return text.encode("utf-8", errors="ignore").decode("utf-8")
        except UnicodeEncodeError as e:
            print(
                f"Warning: Encoding issue detected. Some characters may be lost. Error: {e}"
            )
            # Fall back to ASCII if UTF-8 fails
            return text.encode("ascii", errors="ignore").decode("ascii")
    except Exception as e:
        raise ValueError(f"Error sanitizing input: {str(e)}") from e


def escape_json_string(s):
    """
    Escapes characters in a string to be JSON safe.

    Parameters:
    s (str): The input string to be escaped.

    Returns:
    str: The escaped string, safe for JSON encoding.
    """
    # Replace problematic backslash first
    s = s.replace("\\", "\\\\")

    # Replace the double quote
    s = s.replace('"', '\\"')

View on GitHub (pinned to 7e80152142)

Solutions

1. Decode bytes to str before the call: text.decode('utf-8', errors='ignore') if isinstance(text, bytes).
2. Ensure the value is a plain str; str(obj) custom objects with raising __str__ should be normalized first.
3. Inspect the wrapped {str(e)} — it names the real underlying exception; fix that root cause.
4. Guard empty inputs before calling rather than relying on the helper's falsy shortcut for odd falsy types.

Example fix

# before
clean = sanitize_input(raw_bytes)  # ValueError: Error sanitizing input: 'bytes' object has no attribute ...

# after
if isinstance(raw, bytes):
    raw = raw.decode("utf-8", errors="ignore")
clean = sanitize_input(raw)

def coerce_text(value) -> str:
    if value is None:
        return ""
    if isinstance(value, bytes):
        return value.decode("utf-8", errors="ignore")
    if not isinstance(value, str):
        return str(value)
    return value

# clean = sanitize_input(coerce_text(raw))

def is_sanitizable_text(value) -> bool:
    return isinstance(value, str) or (isinstance(value, bytes) and value is not None)

try:
    clean = sanitize_input(coerce_text(raw))
except ValueError as e:
    if 'Error sanitizing input' in str(e):
        logger.error(f"Sanitization failed for {type(raw).__name__}: {e}")
        clean = ""

When it happens

Trigger: Calling the sanitize helper with bytes containing invalid sequences under unusual codecs is mostly absorbed; this error realistically fires when text is not a str (e.g. bytes passed where str is expected and encode/decode attribute flow fails), or a custom object's encode raises a non-Unicode error.

Common situations: Feeding raw HTTP bytes or file reads opened in 'rb' mode into text-processing APIs, mixed str/bytes pipelines, or upstream data already mangled by a previous exception handler.

Related errors

• URL must start with 'http://', 'https://', 'file://', or 'ra
• Unsupported URL scheme: {scheme}
• Invalid source '{s}'. Valid sources are: {', '.join(valid_so
• Invalid URL, make sure the URL is a non-empty string
• `domain_or_domains` must be a string or a list of strings.

AI-assisted analysis of unclecode/crawl4ai@7e80152142 (2026-08-14). Data as JSON: /api/errors/9d441c538f6cbdc3. Report an issue: GitHub.