{"record":{"id":"6143500810124f62","repo":"vllm-project/vllm","slug":"key-existence-check-failed-str-e","errorCode":null,"errorMessage":"Key existence check failed: {str(e)}","messagePattern":"Key existence check failed: (.+?)","errorType":"http","errorClass":"HTTPException","httpStatus":500,"severity":"error","filePath":"vllm/distributed/kv_transfer/kv_connector/v1/hf3fs/hf3fs_metadata_server.py","lineNumber":356,"sourceCode":"        except Exception as e:\n            logger.error(\"Confirm write for keys failed: %s\", e)\n            raise HTTPException(\n                status_code=500, detail=f\"Confirmation failed: {str(e)}\"\n            ) from e\n\n    async def batch_key_exists(self, request: Request):\n        \"\"\"Check if multiple keys exist in metadata.\"\"\"\n        data = await self._read_json(request)\n        keys = data.get(\"keys\", [])\n\n        if not isinstance(keys, list):\n            raise HTTPException(status_code=400, detail=\"Invalid keys format\")\n\n        try:\n            exists_results = self.state.batch_key_exists(keys)\n            return self._json_response({\"exists\": exists_results})\n        except Exception as e:\n            raise HTTPException(\n                status_code=500, detail=f\"Key existence check failed: {str(e)}\"\n            ) from e\n\n    async def get_key_locations(self, request: Request):\n        \"\"\"Get page indices for keys on a specific rank.\"\"\"\n        data = await self._read_json(request)\n        rank = data.get(\"rank\")\n        keys = data.get(\"keys\", [])\n\n        # Validate input format\n        if rank is None or not isinstance(keys, list):\n            raise HTTPException(\n                status_code=400, detail=\"Invalid request format: need 'rank' and 'keys'\"\n            )\n\n        try:\n            # Get key locations\n            locations = self.state.get_key_locations(rank, keys)","sourceCodeStart":338,"sourceCodeEnd":374,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/vllm/distributed/kv_transfer/kv_connector/v1/hf3fs/hf3fs_metadata_server.py#L338-L374","documentation":"HTTP 500 raised when HF3FSMetadataServerState.batch_key_exists(keys) throws while looking up keys in the metadata store. The format check has already passed, so the failure is inside state access — typically unhashable/unserializable key values or internal dict corruption. The original exception is chained via 'from e', and the response detail embeds its str().","triggerScenarios":"Passing a list containing non-string/non-hashable elements such as nested lists or dicts as keys; concurrent mutation of the key map while it is being read; corrupted in-memory state after a partial crash.","commonSituations":"Client builds keys from unhashable data (e.g. lists instead of tuples/strings); mixed-version clients that send structured key objects where plain strings are expected.","solutions":["Inspect the '{str(e)}' part of the 500 detail — it is the underlying state-layer exception.","Ensure all keys are plain hashable values (strings or string-serialized tuples) before sending.","If the state layer is corrupted, restart the metadata server to rebuild state."],"exampleFix":"// before\nkeys = [[\"sess\", 1], [\"sess\", 2]]\n// after\nkeys = [\"sess|1\", \"sess|2\"]","handlingStrategy":"type-guard","validationCode":"keys = [str(k) for k in keys]  # guarantee hashable string keys before POST","typeGuard":"def all_hashable_keys(keys) -> bool:\n    return all(isinstance(k, (str, int)) for k in keys)","tryCatchPattern":"Catch the 500 and surface the embedded str(e) detail; treat as non-retryable state errors and alert — the metadata server state needs attention.","preventionTips":["Serialize composite keys to a single string format ('|'.join) client-side","Never send dicts/lists as key elements"],"tags":["hf3fs","metadata-server","http-500","kv-transfer"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}