vllm-project/vllm · error · HTTPException

Failed to get key locations: {str(e)}

Error message

Failed to get key locations: {str(e)}

What it means

HTTP 500 raised when HF3FSMetadataServerState.get_key_locations(rank, keys) raises after passing format validation. The response detail embeds the original exception text and it is chained, so the root cause (usually a key not registered for that rank or unhashable key values) is visible in the 500 body. This is a server-side lookup failure, not a payload-shape problem.

Source

Thrown at vllm/distributed/kv_transfer/kv_connector/v1/hf3fs/hf3fs_metadata_server.py:377

    async def get_key_locations(self, request: Request):
        """Get page indices for keys on a specific rank."""
        data = await self._read_json(request)
        rank = data.get("rank")
        keys = data.get("keys", [])

        # Validate input format
        if rank is None or not isinstance(keys, list):
            raise HTTPException(
                status_code=400, detail="Invalid request format: need 'rank' and 'keys'"
            )

        try:
            # Get key locations
            locations = self.state.get_key_locations(rank, keys)
            return self._json_response({"locations": locations})
        except Exception as e:
            raise HTTPException(
                status_code=500, detail=f"Failed to get key locations: {str(e)}"
            ) from e

    async def clear(self, request: Request):
        """Clear the metadata server."""
        self.state.clear()
        return Response(status_code=204)

    def run(self, host: str = "0.0.0.0", port: int = 18000):
        """Run the metadata server."""
        import uvicorn

        logger.info("Starting improved metadata server on http://%s:%s", host, port)
        uvicorn.run(self.app, host=host, port=port)


# --- Client implementation ---
class Hf3fsMetadataInterface(ABC):

View on GitHub (pinned to c794754062)

Solutions

  1. Read the '{str(e)}' detail in the 500 response — it carries the underlying state exception.
  2. Call batch_key_exists first to filter keys that are not yet present, avoiding lookups of unknown keys.
  3. Ensure the target rank completed initialize() and allocate_pages_for_keys() before querying locations.
  4. If the race is systematic, check that lookup requests are issued after allocation confirmations.

Example fix

// before
locations = client.get_key_locations(rank, keys)
// after
existing = client.batch_key_exists(keys)["exists"]
locations = client.get_key_locations(rank, [k for k, ok in zip(keys, existing) if ok])
Defensive patterns

Strategy: validation

Validate before calling

exists = client.batch_key_exists(keys)["exists"]
locations = client.get_key_locations(rank, [k for k, ok in zip(keys, exists) if ok])

Try / catch

Catch HTTP 500; read the embedded root-cause detail; filter out unknown keys and retry once — persistent failure means server state problems.

Prevention

When it happens

Trigger: Querying locations for keys that were never allocated on the given rank; passing non-hashable key entries; rank not initialized via the rank/{rank}/initialize route before the lookup.

Common situations: Prefill and decode workers disagreeing on which keys exist (timing race where lookup precedes allocation); querying a rank after it was cleared; client-side key-format drift.

Related errors

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/d4676b1a01a09a71. Report an issue: GitHub.