{"record":{"id":"9d441c538f6cbdc3","repo":"unclecode/crawl4ai","slug":"error-sanitizing-input-str-e","errorCode":null,"errorMessage":"Error sanitizing input: {str(e)}","messagePattern":"Error sanitizing input: (.+?)","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"crawl4ai/utils.py","lineNumber":792,"sourceCode":"    return sanitized_html\n\n\ndef sanitize_input_encode(text: str) -> str:\n    \"\"\"Sanitize input to handle potential encoding issues.\"\"\"\n    try:\n        try:\n            if not text:\n                return \"\"\n            # Attempt to encode and decode as UTF-8 to handle potential encoding issues\n            return text.encode(\"utf-8\", errors=\"ignore\").decode(\"utf-8\")\n        except UnicodeEncodeError as e:\n            print(\n                f\"Warning: Encoding issue detected. Some characters may be lost. Error: {e}\"\n            )\n            # Fall back to ASCII if UTF-8 fails\n            return text.encode(\"ascii\", errors=\"ignore\").decode(\"ascii\")\n    except Exception as e:\n        raise ValueError(f\"Error sanitizing input: {str(e)}\") from e\n\n\ndef escape_json_string(s):\n    \"\"\"\n    Escapes characters in a string to be JSON safe.\n\n    Parameters:\n    s (str): The input string to be escaped.\n\n    Returns:\n    str: The escaped string, safe for JSON encoding.\n    \"\"\"\n    # Replace problematic backslash first\n    s = s.replace(\"\\\\\", \"\\\\\\\\\")\n\n    # Replace the double quote\n    s = s.replace('\"', '\\\\\"')\n","sourceCodeStart":774,"sourceCodeEnd":810,"githubUrl":"https://github.com/unclecode/crawl4ai/blob/7e801521428ee12509994d39151006f64055ebe3/crawl4ai/utils.py#L774-L810","documentation":"ValueError wrapping any unexpected exception from the input-sanitization helper in utils.py: it first strips/re-encodes text as UTF-8 (falling back to ASCII with a printed warning), and only the outer broad `except Exception` produces this message. Because both inner branches handle Unicode errors, reaching the outer raise indicates a non-encoding failure — most commonly passing a non-string (bytes, None mishandled by a caller, or objects with broken __str__).","triggerScenarios":"Calling the sanitize helper with bytes containing invalid sequences under unusual codecs is mostly absorbed; this error realistically fires when text is not a str (e.g. bytes passed where str is expected and encode/decode attribute flow fails), or a custom object's encode raises a non-Unicode error.","commonSituations":"Feeding raw HTTP bytes or file reads opened in 'rb' mode into text-processing APIs, mixed str/bytes pipelines, or upstream data already mangled by a previous exception handler.","solutions":["Decode bytes to str before the call: text.decode('utf-8', errors='ignore') if isinstance(text, bytes).","Ensure the value is a plain str; str(obj) custom objects with raising __str__ should be normalized first.","Inspect the wrapped {str(e)} — it names the real underlying exception; fix that root cause.","Guard empty inputs before calling rather than relying on the helper's falsy shortcut for odd falsy types."],"exampleFix":"# before\nclean = sanitize_input(raw_bytes)  # ValueError: Error sanitizing input: 'bytes' object has no attribute ...\n\n# after\nif isinstance(raw, bytes):\n    raw = raw.decode(\"utf-8\", errors=\"ignore\")\nclean = sanitize_input(raw)","handlingStrategy":"type-guard","validationCode":"def coerce_text(value) -> str:\n    if value is None:\n        return \"\"\n    if isinstance(value, bytes):\n        return value.decode(\"utf-8\", errors=\"ignore\")\n    if not isinstance(value, str):\n        return str(value)\n    return value\n\n# clean = sanitize_input(coerce_text(raw))","typeGuard":"def is_sanitizable_text(value) -> bool:\n    return isinstance(value, str) or (isinstance(value, bytes) and value is not None)","tryCatchPattern":"try:\n    clean = sanitize_input(coerce_text(raw))\nexcept ValueError as e:\n    if 'Error sanitizing input' in str(e):\n        logger.error(f\"Sanitization failed for {type(raw).__name__}: {e}\")\n        clean = \"\"","preventionTips":["Normalize bytes→str at system boundaries (file/network) before text utilities.","Read files in text mode ('r', encoding='utf-8', errors='ignore') when they feed sanitization.","The wrapped message names the real error — fix the root cause, don't blanket-catch."],"tags":["encoding","sanitization","input-validation","unicode"],"backgroundTag":null,"analyzedSha":"7e801521428ee12509994d39151006f64055ebe3","analyzedAt":"2026-08-14T20:46:20.673Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}