vllm-project/vllm · error · ValueError

{type(self).__name__} received pp_rank > 0 handshake metadat

Error message

{type(self).__name__} received pp_rank > 0 handshake metadata but does not support PP-disaggregated KV transfer.

What it means

The default set_xfer_handshake_metadata_pp_aware assumes pipeline-parallel rank 0 only: if the incoming metadata dict contains any (pp_rank, tp_rank) key with pp_rank != 0, the connector clearly does not implement PP-disaggregated KV transfer, and the base class rejects it. Connectors that do support PP-disaggregation must override this method to consume all PP producer shards.

Source

Thrown at vllm/distributed/kv_transfer/kv_connector/v1/base.py:694

    ) -> None:
        """
        Set the KV connector handshake metadata for this connector.

        Args:
            metadata (KVConnectorHandshakeMetadata): the handshake metadata to set.
        """
        return None

    def set_xfer_handshake_metadata_pp_aware(
        self, metadata: dict[tuple[int, int], KVConnectorHandshakeMetadata]
    ) -> None:
        """
        Set handshake metadata keyed by (pp_rank, tp_rank).
        - Default implementation assumes pp_rank is always 0
        - PP-aware connectors override this to consume all PP producer shards.
        """
        if any(pp_rank != 0 for pp_rank, _ in metadata):
            raise ValueError(
                f"{type(self).__name__} received pp_rank > 0 handshake metadata "
                "but does not support PP-disaggregated KV transfer."
            )
        self.set_xfer_handshake_metadata(
            {tp_rank: meta for (_, tp_rank), meta in metadata.items()}
        )

    @classmethod
    def build_prom_metrics(
        cls,
        vllm_config: "VllmConfig",
        metric_types: dict[type["PromMetric"], type["PromMetricT"]],
        labelnames: list[str],
        per_engine_labelvalues: dict[int, list[object]],
    ) -> "KVConnectorPromMetrics | None":
        """
        Create a KVConnectorPromMetrics subclass which should register
        per-connector Prometheus metrics and implement observe() to

View on GitHub (pinned to c794754062)

Solutions

1. Run with pipeline_parallel_size=1 for this connector
2. Switch to a connector that implements set_xfer_handshake_metadata_pp_aware (PP-disaggregated KV transfer)
3. For custom connectors, override set_xfer_handshake_metadata_pp_aware to handle pp_rank > 0 shards

Example fix

# before (base class default only):
# raise ValueError('received pp_rank > 0 handshake metadata ...')

# after (custom connector adds PP support)
def set_xfer_handshake_metadata_pp_aware(self, metadata):
    for (pp_rank, tp_rank), meta in metadata.items():
        self._shards[(pp_rank, tp_rank)] = meta

if vllm_config.parallel_config.pipeline_parallel_size > 1:
    assert type(connector).set_xfer_handshake_metadata_pp_aware is not KVConnectorBase_V1.set_xfer_handshake_metadata_pp_aware, (
        "Connector lacks PP-disaggregated KV transfer support"
    )

def supports_pp_kv_transfer(connector) -> bool:
    return (
        type(connector).set_xfer_handshake_metadata_pp_aware
        is not KVConnectorBase_V1.set_xfer_handshake_metadata_pp_aware
    )

When it happens

Trigger: Running with pipeline parallelism (pp > 1) plus a KV connector that only implements set_xfer_handshake_metadata (the tp-only API); the scheduler hands handshake metadata keyed by (pp_rank, tp_rank) including pp_rank > 0, hitting the default implementation.

Common situations: Enabling PP on a PD setup with a connector that never added PP support; upgrading a setup to pp>1 with an older/custom connector.

Understand the failure class

• SSL/TLS and certificate errors — how TLS handshakes and certificate validation fail.

Related errors

• HTTP request failed: {0}
• JSON error: {0}
• Tokenizer error: {0}
• tokenize endpoint unavailable: {0}
• managed frontend engine count ({engine_count}) must equal da

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/0a6c72d63cdb2574. Report an issue: GitHub.