sgl-project/sglang · error · ValueError

Unknown dtype: {sampling_params.dtype}

Error message

Unknown dtype: {sampling_params.dtype}

What it means

OpenAI backend generate() dispatches on sampling_params.dtype: None means free text, 'str'/'int'/'float'/'bool' map to constrained completion calls. Any other dtype value falls through to ValueError(f"Unknown dtype: {dtype}") — the requested structured-output type is unsupported.

Source

Thrown at python/sglang/lang/backend/openai.py:220

        elif sampling_params.dtype in [int, "int"]:
            assert (
                not self.is_chat_model
            ), "constrained type not supported on chat model"
            kwargs = sampling_params.to_openai_kwargs()
            kwargs.pop("stop")
            comp = openai_completion(
                client=self.client,
                token_usage=self.token_usage,
                is_chat=self.is_chat_model,
                model=self.model_name,
                prompt=s.text_,
                logit_bias=self.logit_bias_int,
                stop=[" "],
                **kwargs,
            )
            # Leave as a list if that's what is returned.
        else:
            raise ValueError(f"Unknown dtype: {sampling_params.dtype}")

        return comp, {}

    def spec_fill(self, value: str):
        assert self.is_chat_model
        self.spec_format.append({"text": value, "stop": None, "name": None})

    def spec_pattern_match(self, comp):
        for i, term in enumerate(self.spec_format):
            text = term["text"]
            if text != "":
                if comp.startswith(text):
                    comp = comp[len(text) :]
                else:
                    return False
            else:
                pos = comp.find(term["stop"])
                if pos != -1:

View on GitHub (pinned to 0132848349)

Solutions

1. Use one of the supported dtype strings: 'str', 'int', 'float', 'bool', or omit dtype for free text.
2. For structured/JSON output switch to a backend that supports regex/schema (RuntimeEndpoint against a local sglang server with json schema enforcement).
3. Check for typos: it's 'int' not 'integer', 'float' not 'double'.

Example fix

# before
s += sgl.gen("answer", dtype="integer")

# after
s += sgl.gen("answer", dtype="int")

ALLOWED_DTYPES = {None, "str", "int", "float", "bool"}
assert sampling_params_dtype in ALLOWED_DTYPES, f"dtype must be one of {ALLOWED_DTYPES}"

def is_supported_dtype(d) -> bool:
    return d in (None, "str", "int", "float", "bool")

try:
    run(program)
except ValueError as e:
    if "Unknown dtype" in str(e):
        fix dtype to nearest supported ('integer'->'int') and rerun
    else:
        raise

When it happens

Trigger: sgl.gen(..., dtype=some_unsupported_value) with the OpenAI backend — e.g. dtype='json', dtype=list, dtype='number', or a typo like 'integer' instead of 'int'.

Common situations: Expecting JSON-mode or arbitrary schema output (OpenAI backend doesn't implement it here); mismatch between dtype names supported by the RuntimeEndpoint backend vs the OpenAI backend; passing Python types instead of the string names.

Related errors

• Invalid dtype: {sampling_params.dtype}
• This use case is not supported if api speculative execution
• This use case is not supported. For OpenAI chat models, sgl.
• select/choices is not supported for chat models. Please try
• Unknown serve backend {name!r}. Available values: {available

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/35c2c4236fd58e17. Report an issue: GitHub.