sgl-project/sglang · error · RuntimeError

Invalid dtype: {sampling_params.dtype}

Error message

Invalid dtype: {sampling_params.dtype}

What it means

RuntimeEndpoint backend maps a sampling_params.dtype to an internal regex constraining output: int -> integer regex, float/number -> number, str -> string, bool -> boolean. Any other dtype reaches the else branch and raises RuntimeError('Invalid dtype: ...') before the request is sent.

Source

Thrown at python/sglang/lang/backend/runtime_endpoint.py:150

            sampling_params.stop = []

        dtype_regex = None
        if sampling_params.dtype in ["int", int]:

            dtype_regex = REGEX_INT
            sampling_params.stop.extend([" ", "\n"])
        elif sampling_params.dtype in ["float", float]:

            dtype_regex = REGEX_FLOAT
            sampling_params.stop.extend([" ", "\n"])
        elif sampling_params.dtype in ["str", str]:

            dtype_regex = REGEX_STR
        elif sampling_params.dtype in ["bool", bool]:

            dtype_regex = REGEX_BOOL
        else:
            raise RuntimeError(f"Invalid dtype: {sampling_params.dtype}")

        if dtype_regex is not None and sampling_params.regex is not None:
            warnings.warn(
                f"Both dtype and regex are set. Only dtype will be used. dtype: {sampling_params.dtype}, regex: {sampling_params.regex}"
            )

        sampling_params.regex = dtype_regex

    def generate(
        self,
        s: StreamExecutor,
        sampling_params: SglSamplingParams,
    ):
        self._handle_dtype_to_regex(sampling_params)
        data = {
            "text": s.text_,
            "sampling_params": {
                "skip_special_tokens": global_config.skip_special_tokens_in_output,

View on GitHub (pinned to 0132848349)

Solutions

1. Use one of the recognized dtype names: 'str'/'string', 'int'/'integer', 'float'/'number', 'bool', or None.
2. For structured JSON, use the dedicated json-schema gen support or a backend that implements it, rather than dtype.
3. Verify the value is a string name, not a type object (except bool which is special-cased).

Example fix

# before
s += sgl.gen("out", dtype="json")

# after
s += sgl.gen("out", dtype="str", regex=r'\{.*\}')  # constrain manually

VALID = {None, "str", "string", "int", "integer", "float", "number", "bool"}
assert sampling_params.dtype in VALID, f"Invalid dtype {sampling_params.dtype!r}"

def is_valid_dtype(d) -> bool:
    return d in (None, "str", "string", "int", "integer", "float", "number", "bool", bool)

try:
    run(program)
except RuntimeError as e:
    if "Invalid dtype" in str(e):
        normalize dtype to nearest valid name and rerun
    else:
        raise

When it happens

Trigger: sgl.gen(..., dtype=X) on a program running against a RuntimeEndpoint where X is not in [None,'str','int','integer','float','number','bool'] — e.g. 'list', 'json', a Python type object, or a typo.

Common situations: Expecting JSON schema output via dtype='json' (unsupported on this path); passing Python classes (str/int types instead of their names) — note bool the CLASS is handled but e.g. list is not.

Related errors

• Unknown dtype: {sampling_params.dtype}
• Unknown serve backend {name!r}. Available values: {available
• k_cache can only be None when only_qv=True
• q can only be None when only_qv=True
• q must be torch.float8_e4m3fn, got {q.dtype}

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/d1873bf17fddc97f. Report an issue: GitHub.