What causes "Failed to generate schema: no attempts succeeded" in unclecode/crawl4ai?

Fix the underlying per-attempt failure first — earlier exceptions (96/97/98 shapes) hold the root cause; this error only says all attempts were lost

unclecode/crawl4ai · error · Exception

Failed to generate schema: no attempts succeeded

Error message

Failed to generate schema: no attempts succeeded

What it means

Raised at the very end of generate_schema when the attempt loop finished without ever producing a parsed schema — last_schema is None, so the safety net cannot return anything. It means every attempt failed before validation even began (all attempts hit empty responses, API failures, or unrecoverable JSON errors with the repair loop exhausted).

Source

Thrown at crawl4ai/extraction_strategy.py:1987

            if attempt >= max_attempts - 1:
                return schema

            # Detect repeated schema
            current_json = json.dumps(schema, sort_keys=True)
            is_repeated = current_json == prev_schema_json
            prev_schema_json = current_json

            # Build feedback and extend conversation
            feedback = JsonElementExtractionStrategy._build_feedback_message(
                best_result, schema, attempt + 1, is_repeated
            )
            messages.append({"role": "assistant", "content": raw})
            messages.append({"role": "user", "content": feedback})

        # Should not reach here, but return last schema as safety net
        if last_schema is not None:
            return last_schema
        raise Exception("Failed to generate schema: no attempts succeeded")

class JsonCssExtractionStrategy(JsonElementExtractionStrategy):
    """
    Concrete implementation of `JsonElementExtractionStrategy` using CSS selectors.

    How it works:
    1. Parses HTML content with BeautifulSoup.
    2. Selects elements using CSS selectors defined in the schema.
    3. Extracts field data and applies transformations as defined.

    Attributes:
        schema (Dict[str, Any]): The schema defining the extraction rules.
        verbose (bool): Enables verbose logging for debugging purposes.

    Methods:
        _parse_html(html_content): Parses HTML content into a BeautifulSoup object.
        _get_base_elements(parsed_html, selector): Selects base elements using a CSS selector.
        _get_elements(element, selector): Selects child elements using a CSS selector.

View on GitHub (pinned to 7e80152142)

Solutions

Fix the underlying per-attempt failure first — earlier exceptions (96/97/98 shapes) hold the root cause; this error only says all attempts were lost
Increase max_attempts and keep validate=True so the repair loop has room to work
Test the LLM config in isolation with a JSON-output prompt; if it fails there, fix provider/model/credentials before retrying schema generation
Simplify input: pass a smaller, cleaner html sample so the model has an easier job

Example fix

// before
schema = await JsonElementExtractionStrategy.generate_schema(
    html=huge_html, llm_config=cfg, max_attempts=1)  # no attempts succeeded

// after
# sanity-test the LLM first, then retry with a repair budget
schema = await JsonElementExtractionStrategy.generate_schema(
    html=smaller_html, llm_config=cfg, validate=True, max_attempts=5)

Defensive patterns

Strategy: retry

Validate before calling

# ensure per-attempt LLM health first; this error only means every attempt died
resp = await aperform_completion_with_backoff(
    provider=cfg.provider, api_token=cfg.api_token,
    prompt='Return JSON: {"ok": true}', base_url=cfg.base_url)
import json
json.loads(resp.choices[0].message.content)  # strict-JSON capable model?

Try / catch

try:
    schema = await JsonElementExtractionStrategy.generate_schema(
        html=h, llm_config=cfg, validate=True, max_attempts=5)
except Exception as e:
    if str(e) == "Failed to generate schema: no attempts succeeded":
        log.error("all attempts failed — check earlier per-attempt errors for root cause")
    raise

Prevention

Fix the underlying per-attempt failures (empty responses, parse errors, API errors) — this error is only the aggregate symptom
Give the loop room: validate=True plus a larger max_attempts
Shrink/clean the HTML sample so the model's job is easier

When it happens

Trigger: Repeated failures across all max_attempts with validate=True where JSON parse repair never succeeded (each attempt raised before a usable schema was stored in last_schema); or a config combination where attempts immediately error (bad API key producing the error-98 wrapper each time).

Common situations: Persistent LLM-side breakage: invalid credentials for the whole run, an endpoint that always returns empty content, or a model that never emits parseable JSON within the attempt budget.

Related errors

AI-assisted analysis of unclecode/crawl4ai@7e80152142 (2026-08-14). Data as JSON: /api/errors/ea632d2bc4058c35. Report an issue: GitHub.