sgl-project/sglang · error · ValueError

Could not parse attention backend config

Error message

Could not parse attention backend config: {config_str}

What it means

The attention-backend config string (e.g. --attention-backend flash:kv_cache_dtype=fp8,...) is parsed key-by-key into bool/int/float/str. If any part of parsing or value coercion throws, the generic 'Could not parse attention backend config' error wraps it, discarding the underlying cause.

Solutions

1. Simplify the config string to confirm the backend name alone parses, then add one key=value at a time to find the bad token
2. Match the documented 'name:key=value,key=value' grammar exactly, quoting the whole flag in shell
3. Check for stray characters from shell expansion (unquoted colons/commas)

Example fix

# before
--attention-backend 'flash:kv_cache_dtype==fp8'
# after
--attention-backend 'flash:kv_cache_dtype=fp8'

def validate_backend_config(s: str) -> bool:
    try:
        name, _, rest = s.partition(':')
        return bool(name) and all('=' in kv for kv in rest.split(',') if kv)
    except Exception:
        return False

try:
    parsed = ServerArgs._parse_attention_backend_config(cfg)
except ValueError:
    logger.warning('bad attention backend config %r; using default', cfg)
    parsed = {}

When it happens

Trigger: Passing malformed syntax like 'flash:kv_cache_dtype=' or 'flash:flag==true', or an unterminated JSON-ish fragment; any non-numeric, non-bool, non-str token that breaks the coercion ladder.

Common situations: Hand-writing backend override strings; shell quoting mangling colons/commas; version changes in the accepted config grammar.

Related errors

• Attention backend name must be a string
• Component attention backend key must be a string
• Component attention backend key must not be empty
• component_attention_backends must be a dict or a…
• component_attention_backends must use component=backend…

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/417647aaf91035c0. Report an issue: GitHub.

Appendix: source

Thrown at python/sglang/multimodal_gen/runtime/server_args/server_args.py:1724

        # 3. treat as k=v pairs (simple implementation). e.g., "sparsity=0.5,enable_x=true"
        try:
            config = {}
            pairs = config_str.split(",")
            for pair in pairs:
                k, v = pair.split("=", 1)
                k = k.strip()
                v = v.strip()
                if v.lower() == "true":
                    v = True
                elif v.lower() == "false":
                    v = False
                elif v.replace(".", "", 1).isdigit():
                    v = float(v) if "." in v else int(v)
                config[k] = v
            return config
        except Exception:
            raise ValueError(f"Could not parse attention backend config: {config_str}")

    def __post_init__(self):
        # configure logger before use
        configure_logger(server_args=self)

        component_paths: dict[str, str] = {}
        component_weights_paths = dict(self.component_weights_paths)
        for component, path in self.component_paths.items():
            supports_weight_file_override = (
                is_dit_component_name(component)
                or is_text_encoder_component_name(component)
                or is_image_encoder_component_name(component)
                or is_vae_component_name(component)
            )
            if (
                not supports_weight_file_override
                or not is_explicit_weight_file_reference(path)
            ):

View on GitHub (pinned to 0132848349)