sgl-project/sglang · error · MMError

encode metadata not ready

Error message

encode metadata not ready

What it means

The DP worker waited on meta_registry.wait(req_id) for the encoder-side metadata (token counts / hashes produced by preprocessing) to appear, but the wait timed out (asyncio.TimeoutError) and is converted to MMError with HTTP 504 GATEWAY_TIMEOUT. It means preprocessing metadata for this request never arrived at the encoder within the deadline.

Source

Thrown at python/sglang/srt/disaggregation/encoder/runtime.py:1280

) -> None:
    t0 = time.time()
    try:
        if dp_type in ("start_profile", "stop_profile"):
            content = await _dp_worker_handle_profile(enc, dp_rank, dp_type, request)
        elif dp_type == "health_encode":
            content = await _dp_worker_health_encode(enc)
        elif dp_type == "register_destinations":
            await enc.register_embedding_destinations(
                request["req_id"],
                request["receive_count"],
                [request["receive_url"]],
            )
            content = None
        elif dp_type == "wait_metadata":
            try:
                content = await server_module.meta_registry.wait(request["req_id"])
            except asyncio.TimeoutError as e:
                raise MMError(
                    "encode metadata not ready", code=HTTPStatus.GATEWAY_TIMEOUT
                ) from e
        elif dp_type == "send":
            req_id = request["req_id"]
            sent = await enc.send(
                req_id=req_id,
                prefill_host=request["prefill_host"],
                embedding_port=request["embedding_port"],
                session_id=request["session_id"],
                buffer_address=request["buffer_address"],
            )
            if not sent:
                # Error envelope, not 200 + phantom count: the decoder must
                # fail fast instead of waiting for a ZMQ ack that never comes.
                raise MMError(
                    f"no staged embedding for /send req_id={req_id} "
                    f"(already released)"
                )

View on GitHub (pinned to 0132848349)

Solutions

1. Retry the request — transient scheduling lag often resolves it
2. Check that the preprocessor (scheduler/prefill side) successfully published metadata for this req_id
3. Increase the meta registry wait timeout if it is configurable
4. Reduce encoder concurrency or check for a stuck preprocessor worker

if not meta_registry.has(req_id):
    await asyncio.sleep(backoff)  # allow publication to land before waiting

for attempt in range(3):
    try:
        return await worker.handle({'dp_type': 'wait_metadata', 'req_id': req_id})
    except MMError as e:
        if e.code != 504 or attempt == 2: raise
        await asyncio.sleep(2 ** attempt)

When it happens

Trigger: Sending a request with dp_type='wait_metadata' whose req_id was never registered by the preprocessor, or where registration is delayed beyond the registry's timeout — e.g. preprocess and encode phases racing, or the metadata message lost/dropped between components.

Common situations: Preprocessor service slow or crashed before publishing metadata; req_id mismatch between the prefill and encode paths; network/ZMQ delivery of metadata delayed under load; scaling up concurrency so metadata publication lags.

Related errors

• Encoder produced {mm_embedding.shape[0]} tokens, but preproc
• No embedding available for Mooncake GPU-direct transfer: {re
• Rank 0 produced no embedding for {ctx.req_id}
• Waiting for main node timeout!
• DeepGEMM Kernels compilation timeout.\n\nFeel free and pleas

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/12c38bb1afed0ec1. Report an issue: GitHub.