What causes "No embedding available for request: {state.req_id}" in sgl-project/sglang?

Inspect encoder logs for the encode-task exception that set embedding_ready without data — the root cause happened earlier

sgl-project/sglang · error · InternalError

No embedding available for request: {state.req_id}

Error message

No embedding available for request: {state.req_id}

What it means

After awaiting state.embedding_ready, the embedding_data field is still None, meaning the ready event was set by a completion/failure path that did not attach embedding data. The server raises InternalError because it cannot return an EmbeddingData for the request.

Source

Thrown at python/sglang/srt/disaggregation/encoder/server.py:669

        metadata = state.embedding_data
        if (
            metadata is not None
            and metadata.embedding is None
            and mm_data.embedding is not None
            and (metadata.shape != mm_data.shape or metadata.dtype != mm_data.dtype)
        ):
            raise InternalError(
                f"Embedding metadata mismatch for {mm_data.req_id}: "
                f"expected={metadata.shape}/{metadata.dtype}, "
                f"actual={mm_data.shape}/{mm_data.dtype}"
            )
        state.embedding_data = mm_data
        state.embedding_ready.set()

    async def _wait_for_embedding(self, state: ReqState) -> EmbeddingData:
        await state.embedding_ready.wait()
        if state.embedding_data is None:
            raise InternalError(f"No embedding available for request: {state.req_id}")
        return state.embedding_data

    async def send_to_destination(
        self, state: ReqState, destination: SendDestination
    ) -> None:
        async with state.lifecycle_condition:
            if (
                self.req_states.get(state.req_id) is not state
                or state.release_requested
            ):
                raise InternalError(f"Encoder request was released: {state.req_id}")
            state.active_sends += 1
        try:
            await self.delivery.send(state, destination)
        finally:
            async with state.lifecycle_condition:
                state.active_sends -= 1
                state.lifecycle_condition.notify_all()

View on GitHub (pinned to 0132848349)

Solutions

Inspect encoder logs for the encode-task exception that set embedding_ready without data — the root cause happened earlier
Ensure encode failures store an error on ReqState and callers check it before send
Re-run the request with a fresh req_id after cleaning up state via release_request
If reproducible, attach a debugger around the encode task to see which path sets embedding_ready with embedding_data=None

Defensive patterns

Strategy: try-catch

Try / catch

try:
    emb = await encoder._wait_for_embedding(state)
except InternalError:
    log.error('encode produced no embedding for %s', state.req_id)
    await encoder.release_request(state.req_id)
    raise

Prevention

Surface encode-task exceptions through a state.error field
Always pair embedding_ready.set() with either data or an explicit error
Cancel dependent sends when an encode task fails

When it happens

Trigger: The encode task for a req_id fails or is cancelled and sets embedding_ready without populating state.embedding_data; then _wait_for_embedding (called from send) wakes up and finds None.

Common situations: Encoder worker exception or GPU error marking the request ready-to-fail without an error path that surfaces the cause; a release/cancellation path setting the event; race between error handling and the send path; failed encode task leaving the state half-initialized.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/c5432cea87a2e9b5. Report an issue: GitHub.