sgl-project/sglang · critical · InternalError

No embedding available for Mooncake GPU-direct transfer: {re

Error message

No embedding available for Mooncake GPU-direct transfer: {req_id}

What it means

On the mooncake GPU-direct transfer backend, _send requires a staged embedding tensor for the request; encoding is synchronous so the embedding must exist before /encode returned. If embedding is None the send cannot proceed and an InternalError is raised with the request id.

Source

Thrown at python/sglang/srt/disaggregation/encoder/server.py:1538

                embedding_dim,
            )
            item_offset = item_end

    async def _send(
        self,
        embedding: torch.Tensor,
        mm_data: EmbeddingData,
        session_id=None,
        buffer_address=None,
        prefill_host=None,
        embedding_port=None,
        url=None,
    ):
        if get_disagg().encoder_transfer_backend == "mooncake":
            # Encode is synchronous, so mm_data was staged before /encode returned.
            req_id = mm_data.req_id
            if embedding is None:
                raise InternalError(
                    f"No embedding available for Mooncake GPU-direct transfer: {req_id}"
                )

            expected_nbytes = mm_data.shape[0] * mm_data.shape[1] * self._element_size
            assert embedding.nbytes == expected_nbytes, (
                f"Embedding size mismatch for {req_id}: "
                f"actual={embedding.nbytes}, expected={expected_nbytes} "
                f"(shape={mm_data.shape}, element_size={self._element_size})"
            )

            # Fall back to a per-send registration only if the shared one failed.
            mr_already_registered = mm_data._mr_ptr == embedding.data_ptr()
            if not mr_already_registered:
                self.engine.register(embedding.data_ptr(), embedding.nbytes)
            _t_xfer_start = time.monotonic()
            xfer_ret = await asyncio.to_thread(
                self.engine.transfer_sync,
                session_id,

View on GitHub (pinned to 0132848349)

Solutions

1. Ensure send() is only invoked on rank 0 after batch_encode returned a real embedding
2. Check encoder logs for a preceding encode failure that left embedding as None
3. Verify _stage_embeddings slice math produced tensors not None
4. If reproducible, file a bug — this is an internal invariant, not a config error

if get_disagg().encoder_transfer_backend == "mooncake" and embedding is None:
    raise RuntimeError(f"encode produced no embedding for {mm_data.req_id}; refusing to send")

def has_embedding(e) -> bool: return e is not None and e.nbytes > 0

catch InternalError on send; mark request failed and re-encode once before dropping

When it happens

Trigger: Calling send() with embedding=None while encoder_transfer_backend == 'mooncake'; i.e. rank!=0 staged None, encoding failed silently, or send was called before _stage_embeddings ran.

Common situations: A bug in the encoder server staging path, calling send from a non-zero rank, or a race where send happens before the encode result is published.

Related errors

• Rank 0 produced no embedding for {ctx.req_id}
• DSV4 HiSparse direct PD transfer currently requires the Moon
• mooncake encoder_transfer_backend requires HTTP encoders; us
• encode metadata not ready
• Encoder produced {mm_embedding.shape[0]} tokens, but preproc

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/d573c37875a3a3e1. Report an issue: GitHub.