sgl-project/sglang · error · ValueError

cache_dit_params must be a dict, got {type(raw).__name__}.

Error message

cache_dit_params must be a dict, got {type(raw).__name__}.

What it means

resolve_cache_dit_request_overrides validates the per-request cache_dit_params field (reached via _maybe_enable_cache_dit). After the None check, it requires the value to be a dict; any other type (string, list, number) fails fast so the request is rejected before cache-dit is configured.

Source

Thrown at python/sglang/multimodal_gen/runtime/cache/cache_dit_integration.py:228

CACHE_DIT_REQUEST_SCM_KEYS = frozenset(
    {
        "scm_preset",
        "scm_compute_bins",
        "scm_cache_bins",
        "scm_policy",
    }
)
CACHE_DIT_REQUEST_PARAM_KEYS = (
    CACHE_DIT_REQUEST_KNOB_KEYS | CACHE_DIT_REQUEST_SCM_KEYS | {"secondary"}
)


def resolve_cache_dit_request_overrides(raw: dict | None) -> dict:
    """Validate cache_dit_params and return a copy; unknown keys fail the request."""
    if raw is None:
        return {}
    if not isinstance(raw, dict):
        raise ValueError(f"cache_dit_params must be a dict, got {type(raw).__name__}.")
    unknown = set(raw) - CACHE_DIT_REQUEST_PARAM_KEYS
    if unknown:
        raise ValueError(
            f"Unknown cache_dit_params keys: {sorted(unknown)}. "
            f"Valid keys: {sorted(CACHE_DIT_REQUEST_PARAM_KEYS)}."
        )
    overrides = dict(raw)
    secondary = overrides.get("secondary")
    if secondary is not None:
        if not isinstance(secondary, dict):
            raise ValueError(
                "cache_dit_params['secondary'] must be a dict, got "
                f"{type(secondary).__name__}."
            )
        unknown = set(secondary) - CACHE_DIT_REQUEST_KNOB_KEYS
        if unknown:
            raise ValueError(
                f"Unknown cache_dit_params['secondary'] keys: {sorted(unknown)}. "

View on GitHub (pinned to 0132848349)

Solutions

  1. Send cache_dit_params as a JSON object (e.g. {"secondary": {...}}) or omit it / pass null for defaults.
  2. Fix client serialization that double-encodes the dict as a string.
  3. Add request-schema validation enforcing an object type for this field.

Example fix

// before
request(cache_dit_params='[{"cfg": 1.0}]')  // string
// after
request(cache_dit_params={"secondary": {"cfg": 1.0}})  // object
Defensive patterns

Strategy: type-guard

Validate before calling

if cache_dit_params is not None and not isinstance(cache_dit_params, dict):
    raise TypeError("cache_dit_params must be a dict or None")
resp = client.generate(prompt, cache_dit_params=cache_dit_params)

Type guard

def is_valid_cache_dit_params(v: object) -> bool:
    return v is None or isinstance(v, dict)

Try / catch

try:
    client.generate(prompt, cache_dit_params=params)
except ValueError as e:
    if "must be a dict" in str(e):
        params = json.loads(params) if isinstance(params, str) else {}
        retry(client.generate, prompt, cache_dit_params=params)
    else:
        raise

Prevention

When it happens

Trigger: Sending a generate/image request with cache_dit_params set to a non-dict value, e.g. cache_dit_params="true" or ["steps", 4], which reaches resolve_cache_dit_request_overrides.

Common situations: Client-side JSON double-encoding turning the dict into a string; passing a list of key-value pairs; a request schema defaulting cache_dit_params to a non-dict sentinel.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/ebe2fe0088162937. Report an issue: GitHub.