sgl-project/sglang · error · ValueError

Duplicate request ID detected: {rid}

Error message

Duplicate request ID detected: {rid}

What it means

The tokenizer manager tracks every in-flight request by request ID (rid_to_state) and rejects any rid that already has live state. A second request reusing an active rid (not yet finished/aborted) triggers this error.

Source

Thrown at python/sglang/srt/managers/tokenizer_manager.py:3427

        if not hasattr(obj, "is_single") or obj.is_single:
            items = [(obj.rid, obj, getattr(obj, "bootstrap_room", None))]
        else:
            items = [
                (
                    obj.rid[i],
                    obj[i],
                    (
                        obj.bootstrap_room[i]
                        if hasattr(obj, "bootstrap_room") and obj.bootstrap_room
                        else None
                    ),
                )
                for i in range(len(obj.rid))
            ]

        for rid, sub_obj, bootstrap_room in items:
            if rid in self.rid_to_state:
                raise ValueError(f"Duplicate request ID detected: {rid}")
            time_stats = APIServerReqTimeStats(disagg_mode=self.disaggregation_mode)
            state = ReqState([], False, asyncio.Event(), sub_obj, time_stats)
            self.rid_to_state[rid] = state
            if self.enable_trace:
                time_stats.init_trace_ctx(rid, bootstrap_room, external_trace_header)
            time_stats.set_created_time(created_time)

    def _discard_pending_req_states(self, obj):
        """Drop rid_to_state entries created by _init_req_state for *obj*.

        Safe to call after a partial/failed dispatch: only entries still present
        are removed, and the scheduler-response path looks up state with
        ``.get(...)`` so a later output for a discarded rid is ignored, not fatal.
        """
        if not hasattr(obj, "is_single") or obj.is_single:
            rids = [obj.rid]
        else:
            rids = obj.rid

View on GitHub (pinned to 0132848349)

Solutions

1. Let the client generate a fresh rid (e.g. uuid4) per request and per retry
2. For batches, verify rid uniqueness within the array before sending
3. If a rid is stuck, abort the original request or wait for finish before resubmitting (see test_abort_allallows_resubmit_same_rid: abort clears the state)

Example fix

# before
rid = "fixed-id"  # reused across retries
response = client.generate(prompt, rid=rid)

# after
import uuid
rid = str(uuid.uuid4())  # fresh per attempt
response = client.generate(prompt, rid=rid)

import uuid
rid = rid or str(uuid.uuid4())
if rid in in_flight_rids_local:
    rid = str(uuid.uuid4())

def unique_rids(reqs: list[dict]) -> bool:
    ids = [r.get("rid") or str(uuid.uuid4()) for r in reqs]
    return len(ids) == len(set(ids))

try:
    out = engine.generate(prompt, rid=rid)
except ValueError as e:
    if "Duplicate request ID" in str(e):
        out = engine.generate(prompt, rid=str(uuid.uuid4()))
    else:
        raise

When it happens

Trigger: Sending /generate or batch requests where rid duplicates an ID whose state hasn't been cleaned up yet; common with retry logic that reuses generated IDs or client-side fixed IDs, and in batch arrays containing repeated rids.

Common situations: Retries after timeout while original request still processing; batch requests containing the same rid twice; client ID generators with collisions (timestamp seeds, non-unique uuid variants); tests reusing rid constants.

Related errors

• Failed to generate image: {str(e)}
• Network error after {consecutive_errors} consecutive failure
• Krea-2 sequence parallelism does not support ragged/padded m
• Pipeline '{pipeline_name}' is already registered; pass overw
• Model path '{model_path}' is already registered

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/e310e98a635c846e. Report an issue: GitHub.