What causes "kv_cache_quant_config must be QVGKVQuantArgs or a dict" in sgl-project/sglang?

Pass a plain dict of QVGKVQuantArgs fields and let from_dict construct/validate it.

sgl-project/sglang · error · TypeError

kv_cache_quant_config must be QVGKVQuantArgs or a dict

Error message

kv_cache_quant_config must be QVGKVQuantArgs or a dict

What it means

TypeError from ServerArgs.from_dict when the kv_cache_quant_config field is present but is neither a QVGKVQuantArgs instance nor a dict. The constructor accepts an object, a dict of kwargs, or the internal path via QVGKVQuantArgs.from_dict; anything else (string, list, None-with-flag, etc.) is rejected.

Source

Thrown at python/sglang/multimodal_gen/runtime/server_args/server_args.py:3116

            if attr == "_explicit_arg_names":
                continue
            elif attr == "pipeline_config":
                pipeline_config = PipelineConfig.from_kwargs(kwargs)
                logger.debug(f"Using PipelineConfig: {type(pipeline_config)}")
                server_args_kwargs["pipeline_config"] = pipeline_config
            elif attr == "nunchaku_config":
                nunchaku_config = NunchakuSVDQuantArgs.from_dict(kwargs)
                server_args_kwargs["nunchaku_config"] = nunchaku_config
            elif attr == "kv_cache_quant_config":
                kv_quant_config = kwargs.get("kv_cache_quant_config")
                if kv_quant_config is None:
                    kv_quant_config = QVGKVQuantArgs.from_dict(kwargs)
                elif isinstance(kv_quant_config, dict):
                    kv_quant_config = QVGKVQuantArgs(**kv_quant_config).validate()
                elif isinstance(kv_quant_config, QVGKVQuantArgs):
                    kv_quant_config.validate()
                else:
                    raise TypeError(
                        "kv_cache_quant_config must be QVGKVQuantArgs or a dict"
                    )
                server_args_kwargs["kv_cache_quant_config"] = kv_quant_config
            elif attr in kwargs:
                server_args_kwargs[attr] = kwargs[attr]

        return cls(**server_args_kwargs)

    @staticmethod
    def _reject_retired_args(kwargs: dict[str, Any]) -> None:
        retired_args = {
            "decoder_tp": "decoder_sp for decoder/VAE parallel decode",
            "warmup": "warmup_mode=request or warmup_mode=off",
            "server_warmup": "warmup_mode=server or warmup_mode=off",
        }
        removed = [name for name in retired_args if name in kwargs]
        if removed:
            replacements = "; ".join(

View on GitHub (pinned to 0132848349)

Solutions

Pass a plain dict of QVGKVQuantArgs fields and let from_dict construct/validate it.
If you have the structured object already, ensure it is actually QVGKVQuantArgs (not a look-alike dataclass from another module version).
If the value comes from JSON as a string, json.loads it first or restructure the config as a mapping.

Example fix

# before
{"kv_cache_quant_config": "{\"method\": \"awq\"}"}
# after
{"kv_cache_quant_config": {"method": "awq"}}

Defensive patterns

Strategy: type-guard

Validate before calling

from sglang.multimodal_gen.runtime.server_args.server_args import QVGKVQuantArgs
cfg = raw.get("kv_cache_quant_config")
assert cfg is None or isinstance(cfg, (dict, QVGKVQuantArgs)), type(cfg)

Type guard

def is_kv_quant_config(v) -> bool:
    from sglang.multimodal_gen.runtime.server_args.server_args import QVGKVQuantArgs
    return v is None or isinstance(v, (dict, QVGKVQuantArgs))

Try / catch

try:
    args = ServerArgs.from_dict(d)
except TypeError as e:
    if "kv_cache_quant_config" in str(e):
        d["kv_cache_quant_config"] = dict(d["kv_cache_quant_config"])
        args = ServerArgs.from_dict(d)
    else:
        raise

Prevention

Keep quant config as a plain dict in files.
Validate loaded JSON types before passing to from_dict.
json.loads strings before nesting them.

When it happens

Trigger: Passing kv_cache_quant_config as a JSON string from a config file, a list of settings, or an already-validated object of the wrong type; also passing a nested dict under the wrong key so it lands as a foreign type.

Common situations: YAML/JSON config files where quantization settings were written as a string or list; programmatic ServerArgs(**kwargs) built by generic config loaders that keep raw JSON types; version changes that switched the field from string id to structured args.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/9f25f6e3dea7b44a. Report an issue: GitHub.