Regular expression is not supported in the LiteLLM backend.

Error message

Regular expression is not supported in the LiteLLM backend.

What it means

to_litellm_kwargs warns that the LiteLLM passthrough does not carry a regex constraint; whatever underlying provider LiteLLM routes to will generate without it.

Source

Thrown at python/sglang/lang/ir.py:111

    def to_anthropic_kwargs(self):
        # Anthropic does not support frequency_penalty or presence_penalty, so we drop it here
        if self.regex is not None:
            warnings.warn(
                "Regular expression is not supported in the Anthropic backend."
            )
        return {
            "max_tokens": self.max_new_tokens,
            "stop_sequences": (
                self.stop if isinstance(self.stop, (list, tuple)) else [self.stop]
            ),
            "temperature": self.temperature,
            "top_p": self.top_p,
            "top_k": self.top_k,
        }

    def to_litellm_kwargs(self):
        if self.regex is not None:
            warnings.warn("Regular expression is not supported in the LiteLLM backend.")
        return {
            "max_tokens": self.max_new_tokens,
            "stop": self.stop or None,
            "temperature": self.temperature,
            "top_p": self.top_p,
            "frequency_penalty": self.frequency_penalty,
            "presence_penalty": self.presence_penalty,
        }

    def to_srt_kwargs(self):
        return {
            "max_new_tokens": self.max_new_tokens,
            "min_new_tokens": self.min_new_tokens,
            "n": self.n,
            "stop": self.stop,
            "stop_token_ids": self.stop_token_ids,
            "stop_regex": self.stop_regex,
            "temperature": self.temperature,

View on GitHub (pinned to 0132848349)

Solutions

Use provider-native structured output options through LiteLLM (response_format/json_schema)
Use the sglang runtime endpoint backend when regex enforcement is mandatory
Validate output against the regex client-side and re-sample on failure

Example fix

# before
g = sgl.gen("ans", regex=r"[A-Z]{2}\\d{4}")
# after
import re
ans = backend.generate(...).text
if not re.fullmatch(r"[A-Z]{2}\\d{4}", ans): retry()

Defensive patterns

Strategy: validation

Validate before calling

if backend_name == "litellm" and ir.regex is not None:
    ir.regex = None  # pass response_format/json_schema through litellm instead

Prevention

Treat litellm as passthrough; assume no sglang-side constraints
Add output validators independent of backend

When it happens

Trigger: Running a regex-constrained generation through the LiteLLM backend (generate/generate_stream via litellm client).

Common situations: Using LiteLLM as a universal gateway for many providers while assuming sglang constraints still apply; provider-agnostic eval scripts silently losing constraints.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/0aa9954a59643854. Report an issue: GitHub.