What causes "thinking.budget_tokens must be >= 1024 (got {})" in sgl-project/sglang?

Set thinking.budget_tokens to at least 1024

sgl-project/sglang · error · ValueError

thinking.budget_tokens must be >= 1024 (got {})

Error message

thinking.budget_tokens must be >= 1024 (got {})

What it means

Raised by the Anthropic-compatible endpoint's pydantic field validator when a request sets thinking.type='enabled' with a budget_tokens value below 1024. SGLang mirrors the Anthropic API contract that enabled extended thinking requires a budget of at least 1024 tokens. It is a request-validation error surfaced as a 400 before any inference occurs.

Source

Thrown at python/sglang/srt/entrypoints/anthropic/protocol.py:291

    budget is not enforced. ``display="omitted"`` is accepted but
    similarly cannot suppress reasoning mid-stream and is logged.
    """

    type: Literal["enabled", "disabled", "adaptive"]
    budget_tokens: Optional[int] = None
    display: Optional[Literal["summarized", "omitted"]] = None

    @model_validator(mode="after")
    def _validate_thinking_shape(self):
        # Cross-field rules mirror the SDK's three discriminated variants.
        if self.type == "enabled":
            if self.budget_tokens is None:
                raise ValueError(
                    "thinking.budget_tokens is required when "
                    "thinking.type is 'enabled'"
                )
            if self.budget_tokens < 1024:
                raise ValueError(
                    "thinking.budget_tokens must be >= 1024 "
                    "(got {})".format(self.budget_tokens)
                )
        elif self.type == "disabled":
            if self.budget_tokens is not None:
                raise ValueError(
                    "thinking.budget_tokens is not allowed when "
                    "thinking.type is 'disabled'"
                )
            if self.display is not None:
                raise ValueError(
                    "thinking.display is not allowed when "
                    "thinking.type is 'disabled'"
                )
        elif self.type == "adaptive":
            if self.budget_tokens is not None:
                raise ValueError(
                    "thinking.budget_tokens is not allowed when "

View on GitHub (pinned to 0132848349)

Solutions

Set thinking.budget_tokens to at least 1024
Ensure budget_tokens is less than max_tokens (Anthropic also requires max_tokens > budget_tokens)
If you want no thinking, use thinking.type='disabled' and omit budget_tokens

Example fix

// before
{"thinking": {"type": "enabled", "budget_tokens": 512}, "max_tokens": 2048}
// after
{"thinking": {"type": "enabled", "budget_tokens": 1536}, "max_tokens": 4096}

Defensive patterns

Strategy: validation

Validate before calling

def check_thinking(thinking):
    if thinking.get("type") == "enabled":
        b = thinking.get("budget_tokens")
        if b is None or b < 1024:
            raise ValueError("budget_tokens must be >= 1024")

Try / catch

try: resp = client.messages.create(...)
except ValueError as e:
    if 'budget_tokens' in str(e): fix_budget_and_retry()
    else: raise

Prevention

Validate thinking config in one shared builder function
Keep budget_tokens >= 1024 and < max_tokens

When it happens

Trigger: POST /v1/messages with body {"thinking": {"type": "enabled", "budget_tokens": 512}} (or any value < 1024). Only fires when type=='enabled'; budget_tokens must be present and >= 1024.

Common situations: Porting an OpenAI-style reasoning_effort setting to a small token budget, copying a low budget from another provider, or assuming budget_tokens is optional/permissive. Clients that dynamically compute budget as a fraction of max_tokens can produce < 1024 for short requests.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/c175fc41550549ee. Report an issue: GitHub.