vllm-project/vllm · error · HTTPException

Invalid request format: need 'rank' and 'confirmations'

Error message

Invalid request format: need 'rank' and 'confirmations'

What it means

Raised as an HTTP 400 by the HF3FS metadata server's confirm_write_for_keys endpoint when the POST body either omits 'rank' (or it is null) or has a 'confirmations' field that is not a JSON list. It is a request-validation error: the server refuses to process the confirmation because it cannot identify the rank or iterate the confirmations. The client of this API is the HF3FS KV connector worker, so seeing it means the connector sent a malformed payload.

Source

Thrown at vllm/distributed/kv_transfer/kv_connector/v1/hf3fs/hf3fs_metadata_server.py:328

            # Convert results to response format
            response = {"rank": rank, "results": list(results.items())}
            return self._json_response(response)
        except Exception as e:
            raise HTTPException(
                status_code=500, detail=f"Allocation failed: {str(e)}"
            ) from e

    async def confirm_write_for_keys(self, request: Request):
        """Confirm write operations for keys."""
        data = await self._read_json(request)
        rank = data.get("rank")
        confirmations = data.get("confirmations", [])
        pages_to_release = data.get("pages_to_release", [])

        # Validate input format
        if rank is None or not isinstance(confirmations, list):
            raise HTTPException(
                status_code=400,
                detail="Invalid request format: need 'rank' and 'confirmations'",
            )

        try:
            self.state.confirm_write_for_keys(rank, confirmations, pages_to_release)

            return Response(status_code=204)

        except Exception as e:
            logger.error("Confirm write for keys failed: %s", e)
            raise HTTPException(
                status_code=500, detail=f"Confirmation failed: {str(e)}"
            ) from e

    async def batch_key_exists(self, request: Request):
        """Check if multiple keys exist in metadata."""
        data = await self._read_json(request)

View on GitHub (pinned to c794754062)

Solutions

1. Ensure the request body contains a non-null integer 'rank' and a list 'confirmations' (empty list is accepted).
2. Check the connector client version matches the metadata server version so the payload schema agrees.
3. When testing manually, replay the exact payload shape used by HF3FSMetadataClient (rank, confirmations, pages_to_release).

Example fix

// before
await client.post("/confirm_write_for_keys", json={"confirmations": cs})
// after
await client.post("/confirm_write_for_keys", json={"rank": rank, "confirmations": cs, "pages_to_release": []})

def confirm_payload_valid(rank, confirmations):
    return rank is not None and isinstance(confirmations, list)

Catch HTTPException/HTTPStatusError client-side; on status 400, log the request body and fix the client payload — retrying unchanged will fail again.

When it happens

Trigger: POST to the confirm_write_for_keys route with body {'confirmations': [...] } (rank missing/null), or with 'confirmations' as a string/dict/number instead of a list; also triggered by sending 'pages_to_release' alone without 'rank'.

Common situations: Hand-rolled test scripts against the metadata server that forget the rank field; version skew between the metadata server and the connector client where the wire format changed; JSON serialization bugs that wrap confirmations in an object.

Related errors

• Invalid initialization parameters
• Invalid request format: need 'rank' and 'keys'
• Invalid keys format
• Rank {rank} not initialized
• Allocation failed: {str(e)}

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/377b6a00ac16d29f. Report an issue: GitHub.