sgl-project/sglang · error · ValueError

MLA model is not supported without global metadata server…

Error message

MLA model is not supported without global metadata server, please refer to https://github.com/sgl-project/sglang/blob/main/python/sglang/srt/mem_cache/storage/hf3fs/doc

What it means

from_env_config falls back to a default single-node configuration when the env var (HiCacheHF3FS.default_env_var) is unset — but MLA models need a global metadata server to coordinate the shared compressed KV layout, so the fallback is rejected with ValueError pointing at the multinode deployment doc.

Solutions

  1. Set the env var named by HiCacheHF3FS.default_env_var to a JSON config that includes metadata_server_url (deploy per the referenced multinode doc)
  2. Or run a non-MLA model if you only want the default single-node 3FS layout
  3. Template the env var into your deployment manifests so it survives redeploys

Example fix

# before
# (no env var set) -> ValueError for MLA models

# after
export SGLANG_HF3FS_CONFIG=/etc/sglang/hf3fs.json
# hf3fs.json includes "metadata_server_url": "http://meta-svc:8100"
Defensive patterns

Strategy: validation

Validate before calling

import os
from sglang.srt.mem_cache.storage.hf3fs.storage_hf3fs import HiCacheHF3FS

if is_mla_model:
    assert os.getenv(HiCacheHF3FS.default_env_var), (
        'MLA + HF3FS requires the config env var with metadata_server_url'
    )

Try / catch

try:
    backend = HiCacheHF3FS.from_env_config(rank=rank, is_mla_model=True)
except ValueError as e:
    if 'MLA model is not supported' in str(e):
        raise SystemExit('set the HF3FS config env var with metadata_server_url for MLA')
    raise

Prevention

When it happens

Trigger: Running an MLA model (e.g. DeepSeek) with the HF3FS storage backend while the config env var is unset, so from_env_config would build the no-metadata-server default /data/hicache.{rank}.bin layout.

Common situations: Enabling 3FS hicache for an MLA model without providing the JSON config; env var dropped when moving to a new container/systemd unit where the variable wasn't exported.

Understand the failure class

Background: "environment variable is not set" and "Missing keys in environment" errors: what missing required env var messages mean and how to fix them — this error's family across 28 libraries.

Related errors


AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/e50dbf699f8dac8a. Report an issue: GitHub.

Appendix: source

Thrown at python/sglang/srt/mem_cache/storage/hf3fs/storage_hf3fs.py:311

            )

            if storage_config.extra_config is not None:
                use_mock_client = storage_config.extra_config.get(
                    "use_mock_hf3fs_client", False
                )
        else:
            rank, is_mla_model, is_page_first_layout = (
                0,
                False,
                False,
            )

        mla_unsupported_msg = f"MLA model is not supported without global metadata server, please refer to https://github.com/sgl-project/sglang/blob/main/python/sglang/srt/mem_cache/storage/hf3fs/docs/deploy_sglang_3fs_multinode.md"

        config_path = os.getenv(HiCacheHF3FS.default_env_var)
        if not config_path:
            if is_mla_model:
                raise ValueError(mla_unsupported_msg)

            return HiCacheHF3FS(
                rank=rank,
                file_path=f"/data/hicache.{rank}.bin",
                file_size=1 << 40,
                numjobs=16,
                bytes_per_page=bytes_per_page,
                entries=8,
                client_timeout=5,
                dtype=dtype,
                metadata_client=Hf3fsLocalMetadataClient(),
                is_page_first_layout=is_page_first_layout,
                use_mock_client=use_mock_client,
            )

        try:
            with open(config_path, "r") as f:
                config = json.load(f)

View on GitHub (pinned to 0132848349)