sgl-project/sglang · error · ValueError

Invalid hicache storage backend extra config JSON: {e}

Error message

Invalid hicache storage backend extra config JSON: {e}

What it means

When a hicache_storage_backend_extra_config string is set (memory config), the decode offload manager parses it with json.loads and re-raises JSONDecodeError as ValueError. The extra config must be a valid JSON object string (it is passed to the storage backend, e.g. Mooncake/IoUring options).

Source

Thrown at python/sglang/srt/disaggregation/decode_kvcache_offload_manager.py:78

        if not isinstance(kv_cache, (MHATokenToKVPool, MLATokenToKVPool)):
            raise ValueError("Unsupported KV cache type for decode offload")
        self.decode_host_mem_pool = build_kv_host_pool(
            kv_pool=kv_cache,
            page_size=self.page_size,
            use_mla=isinstance(kv_cache, MLATokenToKVPool),
        )

        self.tp_group = tp_group
        self.tp_world_size = torch.distributed.get_world_size(group=self.tp_group)

        hicache_storage_backend_extra_config = {}
        if get_memory().hicache_storage_backend_extra_config:
            try:
                hicache_storage_backend_extra_config = json.loads(
                    get_memory().hicache_storage_backend_extra_config
                )
            except json.JSONDecodeError as e:
                raise ValueError(
                    f"Invalid hicache storage backend extra config JSON: {e}"
                )

        self.cache_controller = HiCacheController(
            token_to_kv_pool_allocator=self.token_to_kv_pool_allocator,
            mem_pool_host=self.decode_host_mem_pool,
            page_size=self.page_size,
            tp_group=tp_group,
            io_backend=get_memory().hicache_io_backend,
            load_cache_event=threading.Event(),
            storage_backend=get_memory().hicache_storage_backend,
            model_name=get_serving().served_model_name,
            storage_backend_extra_config=hicache_storage_backend_extra_config,
        )

        self.ongoing_offload = {}
        self.ongoing_backup = {}
        self.offloaded_state = {}

View on GitHub (pinned to 0132848349)

Solutions

1. Validate the string with any JSON linter, then pass it as a compact JSON object: --hicache-storage-backend-extra-config '{"k":"v"}'
2. Use double quotes inside and single-quote (or proper escaping) the whole argument in shell
3. Remove the flag if you don't need backend-specific extra config

Example fix

# before
--hicache-storage-backend-extra-config "{'k': 'v',}"  # invalid JSON
# after
--hicache-storage-backend-extra-config '{"k": "v"}'

import json
json.loads(hicache_storage_backend_extra_config_str)  # raises before server start if invalid

def is_valid_extra_config(s: str) -> bool:
    try:
        json.loads(s); return True
    except json.JSONDecodeError:
        return False

try:
    cfg = json.loads(extra_config_str)
except json.JSONDecodeError as e:
    raise ValueError(f'Invalid hicache storage backend extra config JSON: {e}') from e

When it happens

Trigger: Passing --hicache-storage-backend-extra-config with malformed JSON — trailing commas, single quotes, unquoted keys, or a plain k=v string instead of JSON.

Common situations: Hand-writing the flag on the command line with shell quoting issues; pasting a Python-dict-style string instead of JSON; truncation by a config templating system.

Related errors

• /v1/models ${response.status}
• Unsupported msgpack byte ${b}
• This browser does not support gzip stream decoding
• MiniCPM SALA does not support hierarchical cache
• Ascend PD transfer does not support HiSparse destination dev

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/dde5acd6c620ff4e. Report an issue: GitHub.