{"record":{"id":"b7834a20d2bef6c2","repo":"sgl-project/sglang","slug":"rank-rank-namespace-namespace-not-initialize","errorCode":null,"errorMessage":"Rank {rank} namespace '{namespace}' not initialized. Please call /{rank}/initialize first.","messagePattern":"Rank (.+?) namespace '(.+?)' not initialized\\. Please call /(.+?)/initialize first\\.","errorType":"http","errorClass":"HTTPException","httpStatus":404,"severity":"error","filePath":"python/sglang/srt/mem_cache/storage/hf3fs/mini_3fs_metadata_server.py","lineNumber":224,"sourceCode":"        self.app.post(\"/{rank}/initialize\")(self.initialize)\n        self.app.post(\"/{rank}/exists\")(self.exists)\n        self.app.post(\"/{rank}/reserve_and_allocate_page_indices\")(\n            self.reserve_and_allocate_page_indices\n        )\n        self.app.post(\"/{rank}/confirm_write\")(self.confirm_write)\n        self.app.post(\"/{rank}/delete_keys\")(self.delete_keys)\n        self.app.post(\"/{rank}/clear\")(self.clear)\n        self.app.post(\"/{rank}/get_page_indices\")(self.get_page_indices)\n\n    def _rank_key(self, rank: int, namespace: str) -> str:\n        \"\"\"Generate the composite key for rank+namespace.\"\"\"\n        return f\"{rank}:{namespace}\"\n\n    def get_rank_metadata(self, rank: int, namespace: str = \"kv\") -> RankMetadata:\n        \"\"\"Get rank metadata with proper error handling.\"\"\"\n        key = self._rank_key(rank, namespace)\n        if key not in self.state.ranks:\n            raise HTTPException(\n                status_code=404,\n                detail=f\"Rank {rank} namespace '{namespace}' not initialized. Please call /{rank}/initialize first.\",\n            )\n        return self.state.ranks[key]\n\n    async def _read_json(self, request: Request) -> dict:\n        \"\"\"Parse request JSON using orjson if available.\"\"\"\n        body = await request.body()\n        return orjson.loads(body)\n\n    def _json_response(self, content: dict):\n        \"\"\"Return ORJSONResponse when available to bypass jsonable_encoder.\"\"\"\n        return ORJSONResponse(content)\n\n    async def initialize(self, rank: int, request: Request):\n        \"\"\"Initialize a rank with specified number of pages.\"\"\"\n        data = await self._read_json(request)\n        num_pages = data[\"num_pages\"]","sourceCodeStart":206,"sourceCodeEnd":242,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/mem_cache/storage/hf3fs/mini_3fs_metadata_server.py#L206-L242","documentation":"This is the FastAPI metadata server's handler: get_rank_metadata looks up state.ranks['{rank}:{namespace}'] and returns HTTP 404 when the rank/namespace pair was never initialized via POST /{rank}/initialize. It exists to make out-of-order startup fail loudly instead of silently allocating wrong pages.","triggerScenarios":"A client (exists, reserve_and_allocate_page_indices, confirm_write, delete_keys, clear, get_page_indices) hitting the server before that rank called initialize; or using a namespace (e.g. 'mla' vs 'kv') that was never initialized for that rank.","commonSituations":"Rank process restart that re-connects without re-initializing; mismatched namespace names between config and client; metadata server restarted (in-memory state lost) while ranks kept running.","solutions":["Call POST /{rank}/initialize with num_pages and namespace before any other endpoint for that rank","Verify the namespace string matches exactly what initialize used (PoolName.KV vs others, case-sensitive key '{rank}:{namespace}')","After a metadata-server restart, restart all ranks or re-run initialize — state is in-memory only"],"exampleFix":"# before\nmeta.reserve_and_allocate_page_indices(rank=0, ...)\n\n# after\nmeta.initialize(rank=0, num_pages=N, namespace=PoolName.KV)\nmeta.reserve_and_allocate_page_indices(rank=0, ...)","handlingStrategy":"validation","validationCode":"resp = requests.get(f'{server}/{rank}/initialized?namespace={ns}')\nif resp.status_code == 404:\n    requests.post(f'{server}/{rank}/initialize',\n                  json={'num_pages': N, 'namespace': ns})","typeGuard":null,"tryCatchPattern":"try:\n    meta.get_page_indices(rank, keys)\nexcept requests.HTTPError as e:\n    if e.response.status_code == 404 and 'not initialized' in e.response.text:\n        initialize_rank(server, rank, ns)\n        meta.get_page_indices(rank, keys)\n    else:\n        raise","preventionTips":["Always run initialize as the first call per (rank, namespace) in startup scripts","Treat metadata-server restarts as full-stack restarts; the state is in-memory"],"tags":["hf3fs","metadata-server","http-404","initialization-order"],"backgroundTag":"service-not-initialized","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}