unclecode/crawl4ai · error · HTTPException

error_msg

Error message

error_msg

What it means

HTTP 400 from the markdown endpoint (deploy/docker/api.py:341) when filter_type == LLM and validate_llm_provider(config, provider) rejects the provider configuration; detail is the validator's error_msg (e.g. provider not allow-listed or missing required config). Fast-path client validation before any crawling starts, so nothing is charged/crawled.

Source

Thrown at deploy/docker/api.py:341

async def handle_markdown_request(
    url: str,
    filter_type: FilterType,
    query: Optional[str] = None,
    cache: str = "0",
    config: Optional[dict] = None,
    provider: Optional[str] = None,
    temperature: Optional[float] = None,
    base_url: Optional[str] = None
) -> str:
    """Handle markdown generation requests."""
    crawler = None
    try:
        # Validate provider if using LLM filter
        if filter_type == FilterType.LLM:
            is_valid, error_msg = validate_llm_provider(config, provider)
            if not is_valid:
                raise HTTPException(
                    status_code=status.HTTP_400_BAD_REQUEST,
                    detail=error_msg
                )
        decoded_url = unquote(url)
        if not decoded_url.startswith(('http://', 'https://')) and not decoded_url.startswith(("raw:", "raw://")):
            decoded_url = 'https://' + decoded_url
        validate_url_destination(decoded_url)

        if filter_type == FilterType.RAW:
            md_generator = DefaultMarkdownGenerator()
        else:
            # Provider by name only; base_url/api_token are server-derived.
            from llm_broker import resolve_llm
            _llm = resolve_llm(config, provider)
            content_filter = {
                FilterType.FIT: PruningContentFilter(),
                FilterType.BM25: BM25ContentFilter(user_query=query or ""),
                FilterType.LLM: LLMContentFilter(

View on GitHub (pinned to 7e80152142)

Solutions

1. Read error_msg in the 400 detail — it states exactly what validation failed.
2. Use a provider name exactly matching the server's llm config section.
3. Fix server config (add provider with token/base_url) if that provider is intended to be offered.
4. Or switch filter_type to raw to bypass LLM filtering entirely.

Example fix

# before
r = requests.get(f"{base}/md", params={"url": u, "f": "llm", "provider": "custom"})

# after
r = requests.get(f"{base}/md", params={"url": u, "f": "llm", "provider": "openai/gpt-4o-mini"})
if r.status_code == 400:
    print(r.json()["detail"])  # validator's reason

def llm_filter_request_valid(filter_type: str, provider: str | None) -> bool:
    if filter_type != "llm":
        return True
    return provider is not None and provider in ALLOWED_PROVIDERS

from enum import Enum

class FilterType(str, Enum):
    RAW = "raw"
    LLM = "llm"

def is_valid_filter_combo(f: str, provider: str | None) -> bool:
    return (f == FilterType.RAW) or (f == FilterType.LLM and provider in ALLOWED_PROVIDERS)

r = await client.get("/md", params=params)
if r.status_code == 400:
    params["f"] = "raw"  # drop LLM filter, retry without
    r = await client.get("/md", params=params)

When it happens

Trigger: GET/POST /md with filter=llm and a provider name absent from the server's configured LLM providers; provider configured but its required fields (api_token, base_url) missing server-side; case-mismatched provider strings.

Common situations: Same allow-list drift as 150 but on the markdown-generation route; server config trimmed for security and clients unaware of permitted providers.

Related errors

• str(e)
• Container not found: ${config.container_selector}
• markdown_generator must be an instance of MarkdownGeneration
• str(e)
• Invalid status: {status}. Must be one of: all, active, compl

AI-assisted analysis of unclecode/crawl4ai@7e80152142 (2026-08-14). Data as JSON: /api/errors/e0fc6370fb5e2051. Report an issue: GitHub.