unclecode/crawl4ai · error · ValueError

LLM did not return valid JSON: {raw}

Error message

LLM did not return valid JSON: {raw}

What it means

Error "LLM did not return valid JSON: {raw}" thrown in unclecode/crawl4ai.

Source

Thrown at crawl4ai/extraction_strategy.py:2818

        # ── LLM call (with retry/backoff)
        resp = perform_completion_with_backoff(
            provider=llm_config.provider,
            prompt_with_variables="\n\n".join([system_msg, user_msg]),
            json_response=True,
            api_token=llm_config.api_token,
            base_url=llm_config.base_url,
            extra_args=kwargs,
        )

        # ── clean & load JSON (fix common escape mistakes *before* json.loads)
        raw = resp.choices[0].message.content
        raw = raw.replace("\x08", "\\b")                     # stray back-space → \b
        raw = re.sub(r'(?<!\\)\\(?![\\u"])', r"\\\\", raw)   # lone \ → \\

        try:
            pattern_dict = json.loads(raw)
        except Exception as exc:
            raise ValueError(f"LLM did not return valid JSON: {raw}") from exc

        # quick sanity-compile
        for lbl, pat in pattern_dict.items():
            try:
                re.compile(pat)
            except re.error as e:
                raise ValueError(f"Invalid regex for '{lbl}': {e}") from None

        return pattern_dict

View on GitHub (pinned to 7e80152142)

Solutions

1. Ensure the LLM prompt requests strict JSON output and set response_format to JSON where supported.
2. Validate and strip markdown fences from the raw response before parsing, or retry with a clearer instruction.

Example fix

config = LLMConfig(provider="openai/gpt-4o"); strategy = LLMExtractionStrategy(llm_config=config, instruction="Return only valid JSON")

When it happens

Trigger: Thrown at crawl4ai/extraction_strategy.py:2818 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of unclecode/crawl4ai@7e80152142 (2026-08-14). Data as JSON: /api/errors/18f64228fb0156cc. Report an issue: GitHub.