vllm-project/vllm · error · HTTPException

Engine ID mismatch for dp_rank={payload.dp_rank}: expected {

Error message

Engine ID mismatch for dp_rank={payload.dp_rank}: expected {dp_entry.engine_id}, got {payload.engine_id}

What it means

HTTP 400 from the Mooncake bootstrap server's register_worker endpoint: a worker registered under a dp_rank that already has an entry, but with a different engine_id. The bootstrap server keys live workers by (dp_rank, tp_rank, pp_rank) and assumes one engine per dp_rank; a different engine_id means a second, distinct vLLM engine claims the same data-parallel rank — usually a stale registration from a previous engine instance that the server never forgot.

Source

Thrown at vllm/distributed/kv_transfer/kv_connector/v1/mooncake/mooncake_utils.py:90

    def shutdown(self):
        if self.server_thread is None or self.server is None or not self.server.started:
            return

        self.server.should_exit = True
        self.server_thread.join()
        logger.info("Mooncake Bootstrap Server stopped.")

    async def register_worker(self, payload: RegisterWorkerPayload):
        """Handles registration of a prefiller worker."""
        if payload.dp_rank not in self.workers:
            self.workers[payload.dp_rank] = EngineEntry(
                engine_id=payload.engine_id,
                worker_addr={},
            )

        dp_entry = self.workers[payload.dp_rank]
        if dp_entry.engine_id != payload.engine_id:
            raise HTTPException(
                status_code=400,
                detail=(
                    f"Engine ID mismatch for dp_rank={payload.dp_rank}: "
                    f"expected {dp_entry.engine_id}, got {payload.engine_id}"
                ),
            )
        if payload.tp_rank not in dp_entry.worker_addr:
            dp_entry.worker_addr[payload.tp_rank] = {}

        tp_entry = dp_entry.worker_addr[payload.tp_rank]
        if payload.pp_rank in tp_entry:
            raise HTTPException(
                status_code=400,
                detail=(
                    f"Worker with dp_rank={payload.dp_rank}, "
                    f"tp_rank={payload.tp_rank}, pp_rank={payload.pp_rank} "
                    f"is already registered at "
                    f"{tp_entry[payload.pp_rank]}, "

View on GitHub (pinned to c794754062)

Solutions

Restart the Mooncake bootstrap server (or clear its worker registry) so stale engine entries are dropped, then let workers re-register.
Ensure each vLLM deployment pointing at a shared bootstrap server uses a distinct dp_rank range.
Avoid running two engines concurrently with the same dp_rank against one bootstrap server.

Defensive patterns

Strategy: retry

Try / catch

Catch the HTTP 400 from register_worker; on 'Engine ID mismatch', restart/clear the bootstrap server registry (or wait for lease expiry), then retry worker registration.

Prevention

Give each vLLM deployment a distinct dp_rank range when sharing a bootstrap server
Restart the Mooncake bootstrap server alongside engine restarts
Never run two engines with the same dp_rank against one bootstrap server

When it happens

Trigger: A vLLM engine restarts (new engine_id) and re-registers while the bootstrap server still holds the old engine's entry; two separate vLLM clusters pointed at the same bootstrap server with overlapping dp_ranks; misconfigured dp_rank causing collisions between independent deployments.

Common situations: Recovering from a crash by restarting only the engine but not the Mooncake bootstrap server; sharing one bootstrap server between environments (staging/prod) without offsetting dp ranks; duplicate-launch of an engine.

Related errors

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/8636a3fe9b5ca1ec. Report an issue: GitHub.