sgl-project/sglang · error · ValueError

Unknown req_type: {req_type!r} (expected 'generate' or 'embe

Error message

Unknown req_type: {req_type!r} (expected 'generate' or 'embed')

What it means

Raised by submit_request in sglang's gRPC bridge when the request dict carries a req_type that is neither 'generate' nor 'embed'. The bridge dispatches solely on those two literals, so any other value (typo, null, or a new request kind) is rejected before an input object is constructed.

Source

Thrown at python/sglang/srt/entrypoints/grpc_bridge.py:294

            _GrpcRequest(is_disconnected_fn=is_disconnected_fn)
            if is_disconnected_fn is not None
            else None
        )
        if req_type == "generate":
            from sglang.srt.managers.io_struct import GenerateReqInput

            obj = GenerateReqInput(**req_dict)
            stream = req_dict.get("stream", False)
            self._submit_on_tm_loop(
                self._run_generate(obj, chunk_callback, stream, mock_request)
            )
        elif req_type == "embed":
            from sglang.srt.managers.io_struct import EmbeddingReqInput

            obj = EmbeddingReqInput(**req_dict)
            self._submit_on_tm_loop(self._run_embed(obj, chunk_callback, mock_request))
        else:
            raise ValueError(
                f"Unknown req_type: {req_type!r} (expected 'generate' or 'embed')"
            )

    async def _run_generate(self, obj, chunk_callback, stream: bool, request):
        ready_event = None
        gen = None
        try:
            ready_event = self._install_on_ready(chunk_callback)
            gen = self.tokenizer_manager.generate_request(obj, request=request)
            if stream:
                completed_choices = set()
                # generate_request does not normalize obj until iteration begins.
                sampling_params = obj.sampling_params or {}
                expected_choices = max(1, int(sampling_params.get("n", 1)))
                async for chunk in gen:
                    choice_finished = (
                        chunk.get("meta_info", {}).get("finish_reason") is not None
                    )

View on GitHub (pinned to 0132848349)

Solutions

1. Set req_type to exactly 'generate' or 'embed' in the request payload before calling submit_request
2. Validate/normalize req_type on the client before dispatch: map 'chat'/'completion'-style intents to 'generate'
3. If you need a new request type, patch the dispatch chain in grpc_bridge.py to handle it upstream rather than passing it through

Example fix

# before
await bridge.submit_request({"req_type": "chat", "text": "hi"})
# after
await bridge.submit_request({"req_type": "generate", "text": "hi"})

req = {"req_type": "generate", "text": "hi"}
assert req.get("req_type") in {"generate", "embed"}, f"bad req_type: {req.get('req_type')!r}"

def is_valid_req_type(req_dict: dict) -> bool:
    return req_dict.get("req_type") in ("generate", "embed")

try:
    await bridge.submit_request(req_dict)
except ValueError as e:
    if "Unknown req_type" in str(e):
        raise ValueError("req_type must be 'generate' or 'embed'") from e
    raise

When it happens

Trigger: Calling submit_request with req_dict['req_type'] set to anything except 'generate' or 'embed' — e.g. 'chat', 'completion', None, or a misspelled key/value — from a gRPC client or a custom servicer adapter.

Common situations: Client/server version skew where a newer client sends a new req_type; hand-written request dicts with typos; JSON deserialization defaulting req_type to null when the field was omitted.

Related errors

• gRPC mode requires the smg-grpc-servicer package. If not ins
• --enable-metrics requires smg-grpc-servicer ≥ 0.5.3 (the ver
• Unknown gemm type: {gemm_type}
• At least one of text, input_ids, or image should be provided
• text and input_ids cannot be provided at the same time

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/c98b44f61820209b. Report an issue: GitHub.