{"record":{"id":"e50dbf699f8dac8a","repo":"sgl-project/sglang","slug":"mla-model-is-not-supported-without-global-metadata","errorCode":null,"errorMessage":"MLA model is not supported without global metadata server, please refer to https://github.com/sgl-project/sglang/blob/main/python/sglang/srt/mem_cache/storage/hf3fs/doc","messagePattern":"MLA model is not supported without global metadata server, please refer to https://github\\.com/sgl-project/sglang/blob/main/python/sglang/srt/mem_cache/storage/hf3fs/doc","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/mem_cache/storage/hf3fs/storage_hf3fs.py","lineNumber":311,"sourceCode":"            )\n\n            if storage_config.extra_config is not None:\n                use_mock_client = storage_config.extra_config.get(\n                    \"use_mock_hf3fs_client\", False\n                )\n        else:\n            rank, is_mla_model, is_page_first_layout = (\n                0,\n                False,\n                False,\n            )\n\n        mla_unsupported_msg = f\"MLA model is not supported without global metadata server, please refer to https://github.com/sgl-project/sglang/blob/main/python/sglang/srt/mem_cache/storage/hf3fs/docs/deploy_sglang_3fs_multinode.md\"\n\n        config_path = os.getenv(HiCacheHF3FS.default_env_var)\n        if not config_path:\n            if is_mla_model:\n                raise ValueError(mla_unsupported_msg)\n\n            return HiCacheHF3FS(\n                rank=rank,\n                file_path=f\"/data/hicache.{rank}.bin\",\n                file_size=1 << 40,\n                numjobs=16,\n                bytes_per_page=bytes_per_page,\n                entries=8,\n                client_timeout=5,\n                dtype=dtype,\n                metadata_client=Hf3fsLocalMetadataClient(),\n                is_page_first_layout=is_page_first_layout,\n                use_mock_client=use_mock_client,\n            )\n\n        try:\n            with open(config_path, \"r\") as f:\n                config = json.load(f)","sourceCodeStart":293,"sourceCodeEnd":329,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/mem_cache/storage/hf3fs/storage_hf3fs.py#L293-L329","documentation":"from_env_config falls back to a default single-node configuration when the env var (HiCacheHF3FS.default_env_var) is unset — but MLA models need a global metadata server to coordinate the shared compressed KV layout, so the fallback is rejected with ValueError pointing at the multinode deployment doc.","triggerScenarios":"Running an MLA model (e.g. DeepSeek) with the HF3FS storage backend while the config env var is unset, so from_env_config would build the no-metadata-server default /data/hicache.{rank}.bin layout.","commonSituations":"Enabling 3FS hicache for an MLA model without providing the JSON config; env var dropped when moving to a new container/systemd unit where the variable wasn't exported.","solutions":["Set the env var named by HiCacheHF3FS.default_env_var to a JSON config that includes metadata_server_url (deploy per the referenced multinode doc)","Or run a non-MLA model if you only want the default single-node 3FS layout","Template the env var into your deployment manifests so it survives redeploys"],"exampleFix":"# before\n# (no env var set) -> ValueError for MLA models\n\n# after\nexport SGLANG_HF3FS_CONFIG=/etc/sglang/hf3fs.json\n# hf3fs.json includes \"metadata_server_url\": \"http://meta-svc:8100\"","handlingStrategy":"validation","validationCode":"import os\nfrom sglang.srt.mem_cache.storage.hf3fs.storage_hf3fs import HiCacheHF3FS\n\nif is_mla_model:\n    assert os.getenv(HiCacheHF3FS.default_env_var), (\n        'MLA + HF3FS requires the config env var with metadata_server_url'\n    )","typeGuard":null,"tryCatchPattern":"try:\n    backend = HiCacheHF3FS.from_env_config(rank=rank, is_mla_model=True)\nexcept ValueError as e:\n    if 'MLA model is not supported' in str(e):\n        raise SystemExit('set the HF3FS config env var with metadata_server_url for MLA')\n    raise","preventionTips":["Export the HF3FS config env var in every launch script/unit that enables 3FS hicache","Add a preflight check combining is_mla_model and the env var to fail before model load"],"tags":["hf3fs","mla","missing-config","env-var"],"backgroundTag":"missing-env-var","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}