sgl-project/sglang · error · ValueError

MLA model is not supported without global metadata server, p

Error message

MLA model is not supported without global metadata server, please refer to https://github.com/sgl-project/sglang/blob/main/python/sglang/srt/mem_cache/storage/hf3fs/doc

What it means

from_env_config falls back to a default single-node configuration when the env var (HiCacheHF3FS.default_env_var) is unset — but MLA models need a global metadata server to coordinate the shared compressed KV layout, so the fallback is rejected with ValueError pointing at the multinode deployment doc.

Source

Thrown at python/sglang/srt/mem_cache/storage/hf3fs/storage_hf3fs.py:311

            )

            if storage_config.extra_config is not None:
                use_mock_client = storage_config.extra_config.get(
                    "use_mock_hf3fs_client", False
                )
        else:
            rank, is_mla_model, is_page_first_layout = (
                0,
                False,
                False,
            )

        mla_unsupported_msg = f"MLA model is not supported without global metadata server, please refer to https://github.com/sgl-project/sglang/blob/main/python/sglang/srt/mem_cache/storage/hf3fs/docs/deploy_sglang_3fs_multinode.md"

        config_path = os.getenv(HiCacheHF3FS.default_env_var)
        if not config_path:
            if is_mla_model:
                raise ValueError(mla_unsupported_msg)

            return HiCacheHF3FS(
                rank=rank,
                file_path=f"/data/hicache.{rank}.bin",
                file_size=1 << 40,
                numjobs=16,
                bytes_per_page=bytes_per_page,
                entries=8,
                client_timeout=5,
                dtype=dtype,
                metadata_client=Hf3fsLocalMetadataClient(),
                is_page_first_layout=is_page_first_layout,
                use_mock_client=use_mock_client,
            )

        try:
            with open(config_path, "r") as f:
                config = json.load(f)

View on GitHub (pinned to 0132848349)

Solutions

Set the env var named by HiCacheHF3FS.default_env_var to a JSON config that includes metadata_server_url (deploy per the referenced multinode doc)
Or run a non-MLA model if you only want the default single-node 3FS layout
Template the env var into your deployment manifests so it survives redeploys

Example fix

# before
# (no env var set) -> ValueError for MLA models

# after
export SGLANG_HF3FS_CONFIG=/etc/sglang/hf3fs.json
# hf3fs.json includes "metadata_server_url": "http://meta-svc:8100"

Defensive patterns

Strategy: validation

Validate before calling

import os
from sglang.srt.mem_cache.storage.hf3fs.storage_hf3fs import HiCacheHF3FS

if is_mla_model:
    assert os.getenv(HiCacheHF3FS.default_env_var), (
        'MLA + HF3FS requires the config env var with metadata_server_url'
    )

Try / catch

try:
    backend = HiCacheHF3FS.from_env_config(rank=rank, is_mla_model=True)
except ValueError as e:
    if 'MLA model is not supported' in str(e):
        raise SystemExit('set the HF3FS config env var with metadata_server_url for MLA')
    raise

Prevention

Export the HF3FS config env var in every launch script/unit that enables 3FS hicache
Add a preflight check combining is_mla_model and the env var to fail before model load

When it happens

Trigger: Running an MLA model (e.g. DeepSeek) with the HF3FS storage backend while the config env var is unset, so from_env_config would build the no-metadata-server default /data/hicache.{rank}.bin layout.

Common situations: Enabling 3FS hicache for an MLA model without providing the JSON config; env var dropped when moving to a new container/systemd unit where the variable wasn't exported.

Understand the failure class

Background: "environment variable is not set" and "Missing keys in environment" errors: what missing required env var messages mean and how to fix them — this error's family across 28 libraries.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/e50dbf699f8dac8a. Report an issue: GitHub.