sgl-project/sglang · error · ValueError

MP mode requires --lmcache-config-file (the YAML supplies mp

Error message

MP mode requires --lmcache-config-file (the YAML supplies mp_host / mp_port).

What it means

LMCacheRadixCache defaults to MP mode (multi-process, using LMCacheMPConnector) except on XPU. MP mode requires the CLI --lmcache-config-file because the YAML it points to supplies mp_host/mp_port for the connector's endpoint; without cli_lmc_cfg, __init__ raises ValueError.

Source

Thrown at python/sglang/srt/mem_cache/storage/lmcache/lmc_radix_cache.py:144

            ),
            v_pool=getattr(
                kvcache,
                "v_buffer",
                getattr(self.token_to_kv_pool_allocator._kvcache, "v_buffer"),
            ),
            tp_group=tp_group.device_group if tp_group is not None else None,
        )

        self.load_stream = create_device_stream(self.device)
        self.store_stream = create_device_stream(self.device)

        # MP (multi-process) is the default. XPU defaults to IP (in-process
        # layerwise) because the MP connector shares the KV cache via CUDA IPC
        # (``Tensor._share_cuda_``), which is unavailable on XPU.
        self._mode = LMCacheMode.IP if self.device.type == "xpu" else LMCacheMode.MP
        if self._mode is LMCacheMode.MP:
            if not cli_lmc_cfg:
                raise ValueError(
                    "MP mode requires --lmcache-config-file (the YAML "
                    "supplies mp_host / mp_port)."
                )
            lm_cfg = lmcache_get_config(cli_lmc_cfg)
            self.lmcache_connector = LMCacheMPConnector(
                page_size=params.page_size,
                host=lm_cfg.mp_host,
                port=lm_cfg.mp_port,
                **connector_kwargs,
            )
        elif self._mode is LMCacheMode.IP:
            self.lmcache_connector = LMCacheLayerwiseConnector(
                config_file=cli_lmc_cfg, **connector_kwargs
            )
            # Per-layer hook
            self.layer_done_executor = LayerTransferCounter(
                num_layers=(
                    model_config.num_hidden_layers if model_config is not None else 0

View on GitHub (pinned to 0132848349)

Solutions

Pass --lmcache-config-file /path/to/lmcache.yaml when launching the server, with mp_host/mp_port set in the YAML
If you intended in-process layerwise mode, use an XPU device or explicitly configure the IP mode path supported by your version
Update launch scripts/checklists — MP is now the default on non-XPU devices

Example fix

# before
python -m sglang.launch_server --model ... --enable-lmcache

# after
python -m sglang.launch_server --model ... --enable-lmcache \
  --lmcache-config-file /etc/sglang/lmcache.yaml  # contains mp_host/mp_port

Defensive patterns

Strategy: validation

Validate before calling

if device.type != 'xpu':
    assert cli_lmc_cfg, 'MP mode needs --lmcache-config-file with mp_host/mp_port'
cache = LMCacheRadixCache(params, device, cli_lmc_cfg=cli_lmc_cfg, ...)

Try / catch

try:
    cache = LMCacheRadixCache(params, device)
except ValueError as e:
    if 'MP mode requires --lmcache-config-file' in str(e):
        raise SystemExit('add --lmcache-config-file lmcache.yaml (with mp_host/mp_port)')
    raise

Prevention

Always pass --lmcache-config-file when enabling LMCache on GPU (non-XPU) after upgrades
Keep mp_host/mp_port filled in the YAML; validate it in a preflight script

When it happens

Trigger: Constructing LMCacheRadixCache on a CUDA device without passing the lmcache config file (cli_lmc_cfg is None) — e.g. enabling LMCache via server args/env without --lmcache-config-file.

Common situations: Switching from the old IP/layerwise integration (no config file needed) to the new default MP mode after an upgrade; config file flag forgotten in launch scripts; XPU deployment changed to GPU without adding the YAML.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/388d5676a347e1fb. Report an issue: GitHub.