unclecode/crawl4ai · warning · ValueError

type must be 'CrawlerRunConfig' or 'BrowserConfig'

Error message

type must be 'CrawlerRunConfig' or 'BrowserConfig'

What it means

A ValueError raised by _config_from_json() in the crawl server when the 'type' field of a submitted {type, params} config payload is anything other than the exact strings 'CrawlerRunConfig' or 'BrowserConfig'. The helper is a hardened config-dump validator: only these two gated, side-effect-free types may be constructed; the untrusted loader additionally rejects power-fields (LLM*, proxy, deep-crawl), drops unknown fields, and clamps quantities.

Source

Thrown at deploy/docker/server.py:477

}


def _config_from_json(data: dict) -> dict:
    """Validate a {type, params} config under the untrusted trust boundary and
    echo the normalized result.

    This endpoint is no longer a gadget-construction oracle: only the gated,
    side-effect-free CrawlerRunConfig/BrowserConfig types may be validated, the
    untrusted gate raises on forbidden power-fields and disallowed nested types
    (LLM*, proxy, deep-crawl - which is what would read env/secrets), drops
    unknown fields, and clamps quantities."""
    config_type = data.get("type")
    if config_type == "CrawlerRunConfig":
        obj = CrawlerRunConfig.load(data, provenance=Provenance.UNTRUSTED)
    elif config_type == "BrowserConfig":
        obj = BrowserConfig.load(data, provenance=Provenance.UNTRUSTED)
    else:
        raise ValueError("type must be 'CrawlerRunConfig' or 'BrowserConfig'")
    return obj.dump()


# ── job router ──────────────────────────────────────────────
app.include_router(init_job_router(redis, config, token_dep))

# ── monitor router ──────────────────────────────────────────
# Do not attach token_dep at router level: it is HTTP Request-only and breaks
# the WebSocket upgrade on /monitor/ws (TypeError: _principal() missing 'request').
# AuthGateMiddleware already authenticates HTTP + WS; destructive monitor
# actions keep their own Depends(require_admin).
from monitor_routes import router as monitor_router
app.include_router(monitor_router)

logger = logging.getLogger(__name__)


# ── central exception handling (no internal detail leaks) ─────────────

View on GitHub (pinned to 7e80152142)

Solutions

1. Set type to exactly 'CrawlerRunConfig' or 'BrowserConfig' (case-sensitive) and put settings under the params field.
2. Do not attempt to configure LLM, proxy, or deep-crawl behavior through this endpoint - the untrusted gate strips/rejects those by design; use server-side configuration channels instead.
3. Validate the payload shape client-side against the two allowed types before submitting.

Example fix

# before
{"type": "crawler_config", "params": {"word_count_threshold": 200}}  # ValueError

# after
{"type": "CrawlerRunConfig", "params": {"word_count_threshold": 200}}

ALLOWED_CONFIG_TYPES = {'CrawlerRunConfig', 'BrowserConfig'}

def valid_config_payload(data: dict) -> bool:
    return (
        isinstance(data, dict)
        and data.get('type') in ALLOWED_CONFIG_TYPES
        and isinstance(data.get('params', {}), dict)
    )

from typing import TypeGuard, Literal, TypedDict

ConfigType = Literal['CrawlerRunConfig', 'BrowserConfig']

class ConfigPayload(TypedDict):
    type: ConfigType
    params: dict

def is_config_payload(v) -> TypeGuard[ConfigPayload]:
    return (
        isinstance(v, dict)
        and v.get('type') in ('CrawlerRunConfig', 'BrowserConfig')
        and isinstance(v.get('params', {}), dict)
    )

try:
    result = client.post('/config/dump', json=payload)
except (ValueError, HTTPError) as e:
    raise ValueError(
        f"config type must be 'CrawlerRunConfig' or 'BrowserConfig'; got {payload.get('type')!r}"
    ) from e

When it happens

Trigger: POSTing to /config/dump or a crawl endpoint's config slot with {"type": "AsyncPlaywrightCrawlerStrategy"}, {"type": "crawler"}, {"type": "LLMConfig"}, a missing 'type' key (None), or wrong casing like 'crawlerrunconfig'.

Common situations: Clients ported from an older API that accepted arbitrary crawl4ai class names; attempts to smuggle in LLM or proxy configuration through the config slot (correctly rejected); copy-paste typos and casing mismatches in hand-written payloads.

Related errors

• URL must start with {schemes}
• Token issuance is disabled: no api_token is configured on th
• [NSTProxy] Invalid protocol: {protocol}
• concurrency must be positive
• timeout must be positive

AI-assisted analysis of unclecode/crawl4ai@7e80152142 (2026-08-14). Data as JSON: /api/errors/3624abffca536155. Report an issue: GitHub.