{"record":{"id":"501ddc60d0a11e56","repo":"unslothai/unsloth","slug":"chat-template-exceeds-the-max-chat-template-bytes","errorCode":null,"errorMessage":"Chat template exceeds the {MAX_CHAT_TEMPLATE_BYTES}-byte limit.","messagePattern":"Chat template exceeds the (.+?)-byte limit\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"warning","filePath":"studio/backend/models/inference.py","lineNumber":74,"sourceCode":"    )\n    approved_remote_code_fingerprint: Optional[str] = Field(\n        None,\n        description = \"sha256 fingerprint from the remote-code scan, pinning user approval of this exact custom-code version.\",\n    )\n    chat_template_override: Optional[str] = Field(\n        None,\n        description = \"Custom Jinja2 chat template to use instead of the model's default\",\n    )\n\n    @field_validator(\"chat_template_override\")\n    @classmethod\n    def normalize_blank_chat_template_override(cls, value: Optional[str]) -> Optional[str]:\n        if value is None:\n            return None\n        # Char count is a lower bound on UTF-8 byte length: reject an oversized\n        # template before spending work encoding it.\n        if len(value) > MAX_CHAT_TEMPLATE_BYTES:\n            raise ValueError(f\"Chat template exceeds the {MAX_CHAT_TEMPLATE_BYTES}-byte limit.\")\n        if value.strip() == \"\":\n            return None\n        if len(value.encode(\"utf-8\")) > MAX_CHAT_TEMPLATE_BYTES:\n            raise ValueError(f\"Chat template exceeds the {MAX_CHAT_TEMPLATE_BYTES}-byte limit.\")\n        return value\n\n    cache_type_kv: Optional[str] = Field(\n        None,\n        description = (\n            \"KV cache data type for both K and V \"\n            \"(e.g. 'f16', 'bf16', 'q8_0', 'q4_0', 'q4_1', 'q5_0', 'q5_1', 'iq4_nl', 'f32')\"\n        ),\n    )\n    mlx_kv_bits: Optional[int] = Field(\n        None,\n        description = (\n            \"MLX KV cache quantization bit width (8, 6, 5, 4, 3 or 2). MLX takes a bit \"\n            \"width rather than a llama.cpp dtype name, so this is separate from \"","sourceCodeStart":56,"sourceCodeEnd":92,"githubUrl":"https://github.com/unslothai/unsloth/blob/203007d19051dcd2ae33876786d117c99f6b0368/studio/backend/models/inference.py#L56-L92","documentation":"ValueError from the chat_template_override field validator on the inference request model when the custom Jinja2 template exceeds MAX_CHAT_TEMPLATE_BYTES (65,536 bytes, defined in picker/schemas.py). The check is two-stage: a cheap char-count lower bound rejects obviously oversized input before encoding, then an exact UTF-8 byte-length check catches multibyte-heavy templates that pass the char count. Blank templates are normalized to None, so this only fires on real oversized content.","triggerScenarios":"Sending chat_template_override longer than 65,536 bytes — either >65,536 characters (first check) or fewer characters whose UTF-8 encoding exceeds 65,536 bytes because of multibyte characters like '€' or CJK text (second check); e.g. pasting a giant template with an embedded few-shot prompt.","commonSituations":"Templates that inline large few-shot examples or a full tokenizer chat template plus additions; templates copied from another tool that embeds base64 or verbose macros; non-English templates where byte length greatly exceeds character count.","solutions":["Trim the template: move large few-shot content into the system/user messages instead of the Jinja template itself.","Use the model's default template (omit chat_template_override) if your additions are only cosmetic.","If the template legitimately needs to be huge, reference it by file through the template-picker upload path, which enforces its own size caps.","Check the size before sending: len(template.encode('utf-8')) <= 65536."],"exampleFix":"# before\ntemplate = open(\"huge_template.jinja\").read()   # 90 KB\nreq = {\"chat_template_override\": template, ...}\n# after\nimport sys\ntemplate = open(\"huge_template.jinja\").read()\nassert len(template.encode(\"utf-8\")) <= 65_536, \"template too large\"\nreq = {\"chat_template_override\": template, ...}","handlingStrategy":"validation","validationCode":"MAX_CHAT_TEMPLATE_BYTES = 65_536\n\ndef chat_template_within_limit(template: str) -> bool:\n    if template is None:\n        return True\n    # mirror the server: char-count lower bound, then exact UTF-8 length\n    return len(template) <= MAX_CHAT_TEMPLATE_BYTES and \\\n           len(template.encode(\"utf-8\")) <= MAX_CHAT_TEMPLATE_BYTES","typeGuard":"def is_valid_chat_template_override(v: str | None) -> bool:\n    return v is None or (v.strip() != \"\" and len(v.encode(\"utf-8\")) <= 65_536)","tryCatchPattern":"try:\n    resp = client.post(\"/v1/inference\", json=payload)\nexcept Exception:\n    raise\nif resp.status_code == 422 and \"byte limit\" in resp.text:\n    raise TemplateTooLarge(len(payload[\"chat_template_override\"].encode(\"utf-8\")))","preventionTips":["Keep few-shot examples in messages, not in the Jinja template.","Check len(template.encode('utf-8')) <= 65536 client-side before sending — char count alone is insufficient for multibyte text.","Omit chat_template_override entirely when the model's default template suffices.","Note that blank/whitespace templates are normalized to None, not rejected."],"tags":["validation","chat-template","inference","http-422","size-limit"],"backgroundTag":null,"analyzedSha":"203007d19051dcd2ae33876786d117c99f6b0368","analyzedAt":"2026-08-15T02:48:39.846Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}