sgl-project/sglang · error · MMError

no staged embedding for /send req_id={req_id} (already relea

Error message

no staged embedding for /send req_id={req_id} (already released)

What it means

The decoder called the encoder's /send endpoint for a req_id whose staged embedding has already been released, so enc.send(...) returned False. The encoder deliberately returns an error envelope instead of 200 so the decoder fails fast rather than waiting forever for a ZMQ ack that will never arrive.

Source

Thrown at python/sglang/srt/disaggregation/encoder/runtime.py:1295

            try:
                content = await server_module.meta_registry.wait(request["req_id"])
            except asyncio.TimeoutError as e:
                raise MMError(
                    "encode metadata not ready", code=HTTPStatus.GATEWAY_TIMEOUT
                ) from e
        elif dp_type == "send":
            req_id = request["req_id"]
            sent = await enc.send(
                req_id=req_id,
                prefill_host=request["prefill_host"],
                embedding_port=request["embedding_port"],
                session_id=request["session_id"],
                buffer_address=request["buffer_address"],
            )
            if not sent:
                # Error envelope, not 200 + phantom count: the decoder must
                # fail fast instead of waiting for a ZMQ ack that never comes.
                raise MMError(
                    f"no staged embedding for /send req_id={req_id} "
                    f"(already released)"
                )
            # Releasing on the first /send breaks decoder TP > 1. No count means
            # a pre-refcount decoder: stay eager rather than pin until the sweep.
            receive_count = request.get("receive_count")
            if receive_count:
                await server_module.meta_registry.note_send_done(req_id, receive_count)
            else:
                await enc.release_request(req_id)
            content = None
        else:
            content = await execute_encode_pipeline(enc, sched, request)

        logger.info(
            f"MM-Encoder [dp_rank={dp_rank}] {dp_type} done: "
            f"req_id={request.get('req_id', '?')}, "
            f"modality={request.get('modality', 'image')}, "

View on GitHub (pinned to 0132848349)

Solutions

1. Make the decoder send exactly once per req_id (deduplicate retries across TP ranks)
2. Ensure all decoder TP ranks register receive_count correctly before the first /send so release is refcounted, not eager
3. Regenerate a fresh req_id for retried requests instead of reusing the old one
4. Check for duplicate delivery of the /send HTTP request (proxy retries, at-least-once middleware)

if encoder_client.already_sent(req_id):
    skip_send(req_id)  # idempotency check before /send

try:
    await encoder.send(req_id)
except MMError as e:
    if 'already released' in str(e):
        treat_as_duplicate(req_id); return  # not fatal for this rank
    raise

When it happens

Trigger: Duplicate /send calls for the same req_id after the embedding was released (e.g. decoder TP ranks retrying, or a race where one rank's send triggers eager release and a later send arrives); sending after an explicit release_request; using a pre-refcount decoder that re-sends.

Common situations: Decoder TP > 1 with mismatched receive_count accounting; client retry logic re-issuing /send; a stale req_id reused across requests after the sweep released it.

Related errors

• Encoder request was released: {state.req_id}
• No embedding available for request: {state.req_id}
• Cannot transition {request_id} from terminal state {old_stat
• Subclass {self.__class__.__name__} must define _supported_at
• You must specify exactly one of input_ids or inputs_embeds

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/063d2eac5107dda5. Report an issue: GitHub.