unslothai/unsloth · warning · ValueError

Chat template exceeds the {MAX_CHAT_TEMPLATE_BYTES}-byte lim

Error message

Chat template exceeds the {MAX_CHAT_TEMPLATE_BYTES}-byte limit.

What it means

ValueError from the chat_template_override field validator on the inference request model when the custom Jinja2 template exceeds MAX_CHAT_TEMPLATE_BYTES (65,536 bytes, defined in picker/schemas.py). The check is two-stage: a cheap char-count lower bound rejects obviously oversized input before encoding, then an exact UTF-8 byte-length check catches multibyte-heavy templates that pass the char count. Blank templates are normalized to None, so this only fires on real oversized content.

Source

Thrown at studio/backend/models/inference.py:74

    )
    approved_remote_code_fingerprint: Optional[str] = Field(
        None,
        description = "sha256 fingerprint from the remote-code scan, pinning user approval of this exact custom-code version.",
    )
    chat_template_override: Optional[str] = Field(
        None,
        description = "Custom Jinja2 chat template to use instead of the model's default",
    )

    @field_validator("chat_template_override")
    @classmethod
    def normalize_blank_chat_template_override(cls, value: Optional[str]) -> Optional[str]:
        if value is None:
            return None
        # Char count is a lower bound on UTF-8 byte length: reject an oversized
        # template before spending work encoding it.
        if len(value) > MAX_CHAT_TEMPLATE_BYTES:
            raise ValueError(f"Chat template exceeds the {MAX_CHAT_TEMPLATE_BYTES}-byte limit.")
        if value.strip() == "":
            return None
        if len(value.encode("utf-8")) > MAX_CHAT_TEMPLATE_BYTES:
            raise ValueError(f"Chat template exceeds the {MAX_CHAT_TEMPLATE_BYTES}-byte limit.")
        return value

    cache_type_kv: Optional[str] = Field(
        None,
        description = (
            "KV cache data type for both K and V "
            "(e.g. 'f16', 'bf16', 'q8_0', 'q4_0', 'q4_1', 'q5_0', 'q5_1', 'iq4_nl', 'f32')"
        ),
    )
    mlx_kv_bits: Optional[int] = Field(
        None,
        description = (
            "MLX KV cache quantization bit width (8, 6, 5, 4, 3 or 2). MLX takes a bit "
            "width rather than a llama.cpp dtype name, so this is separate from "

View on GitHub (pinned to 203007d190)

Solutions

Trim the template: move large few-shot content into the system/user messages instead of the Jinja template itself.
Use the model's default template (omit chat_template_override) if your additions are only cosmetic.
If the template legitimately needs to be huge, reference it by file through the template-picker upload path, which enforces its own size caps.
Check the size before sending: len(template.encode('utf-8')) <= 65536.

Example fix

# before
template = open("huge_template.jinja").read()   # 90 KB
req = {"chat_template_override": template, ...}
# after
import sys
template = open("huge_template.jinja").read()
assert len(template.encode("utf-8")) <= 65_536, "template too large"
req = {"chat_template_override": template, ...}

Defensive patterns

Strategy: validation

Validate before calling

MAX_CHAT_TEMPLATE_BYTES = 65_536

def chat_template_within_limit(template: str) -> bool:
    if template is None:
        return True
    # mirror the server: char-count lower bound, then exact UTF-8 length
    return len(template) <= MAX_CHAT_TEMPLATE_BYTES and \
           len(template.encode("utf-8")) <= MAX_CHAT_TEMPLATE_BYTES

Type guard

def is_valid_chat_template_override(v: str | None) -> bool:
    return v is None or (v.strip() != "" and len(v.encode("utf-8")) <= 65_536)

Try / catch

try:
    resp = client.post("/v1/inference", json=payload)
except Exception:
    raise
if resp.status_code == 422 and "byte limit" in resp.text:
    raise TemplateTooLarge(len(payload["chat_template_override"].encode("utf-8")))

Prevention

Keep few-shot examples in messages, not in the Jinja template.
Check len(template.encode('utf-8')) <= 65536 client-side before sending — char count alone is insufficient for multibyte text.
Omit chat_template_override entirely when the model's default template suffices.
Note that blank/whitespace templates are normalized to None, not rejected.

When it happens

Trigger: Sending chat_template_override longer than 65,536 bytes — either >65,536 characters (first check) or fewer characters whose UTF-8 encoding exceeds 65,536 bytes because of multibyte characters like '€' or CJK text (second check); e.g. pasting a giant template with an embedded few-shot prompt.

Common situations: Templates that inline large few-shot examples or a full tokenizer chat template plus additions; templates copied from another tool that embeds base64 or verbose macros; non-English templates where byte length greatly exceeds character count.

Related errors

AI-assisted analysis of unslothai/unsloth@203007d190 (2026-08-15). Data as JSON: /api/errors/501ddc60d0a11e56. Report an issue: GitHub.