{"record":{"id":"b78a499b488d5dd8","repo":"unslothai/unsloth","slug":"extra-llama-server-args-cannot-contain-unpaired-su","errorCode":null,"errorMessage":"extra llama-server args cannot contain unpaired surrogate characters","messagePattern":"extra llama-server args cannot contain unpaired surrogate characters","errorType":"validation","errorClass":"ValueError","httpStatus":400,"severity":"error","filePath":"studio/backend/core/inference/llama_server_args.py","lineNumber":261,"sourceCode":"    # arity this module knows for certain.\n    pending_two_value = 0\n    two_value_flag = \"\"\n    for raw in args:\n        token = str(raw)\n        if len(out) >= MAX_EXTRA_ARG_TOKENS:\n            raise ValueError(\n                f\"too many extra llama-server args (limit {MAX_EXTRA_ARG_TOKENS} tokens)\"\n            )\n        # A grammar or JSON schema is a legitimately long single token, so the cap\n        # is on the whole list rather than per token.\n        # Strictly, unlike the sizing below: JSON and the browser can both carry an\n        # unpaired surrogate, which survives every check here and then makes\n        # subprocess.Popen raise while it encodes argv, long after the load has begun\n        # switching models. Refused at the boundary, where it is still a 400.\n        try:\n            encoded = token.encode(\"utf-8\")\n        except UnicodeEncodeError as error:\n            raise ValueError(\n                \"extra llama-server args cannot contain unpaired surrogate characters\"\n            ) from error\n        total_bytes += len(encoded)\n        limit = max_extra_args_bytes()\n        if total_bytes > limit:\n            raise ValueError(f\"extra llama-server args are too large (limit {limit} bytes)\")\n        # execve rejects a NUL outright; the rest would reach the child's parser as\n        # invisible characters and be blamed on the flag they are attached to.\n        if _has_control_characters(token):\n            raise ValueError(\"extra llama-server args cannot contain control characters\")\n        flag = _flag_name(token)\n        if flag is not None and flag in _DENYLIST:\n            raise ValueError(\n                f\"llama-server flag '{flag}' is managed by Unsloth Studio \"\n                f\"and cannot be passed as an extra arg\"\n            )\n        if flag is None:\n            # A token belonging to no flag. Today's llama-server answers \"invalid","sourceCodeStart":243,"sourceCodeEnd":279,"githubUrl":"https://github.com/unslothai/unsloth/blob/203007d19051dcd2ae33876786d117c99f6b0368/studio/backend/core/inference/llama_server_args.py#L243-L279","documentation":"ValueError from the per-token walk in llama_server_args.py:261 — a token in the extra args fails token.encode('utf-8') because it contains an unpaired surrogate (e.g. a lone '\\ud83d' from bad JSON or browser input). Such a string survives Python and JSON checks but crashes subprocess.Popen when it encodes argv, long after the model switch has begun. The check refuses it at the boundary, where the client still gets a 400.","triggerScenarios":"Submitting extra args containing a lone surrogate half — typically from decoding JSON with surrogateescape, from JavaScript string manipulation that split an emoji, or from pasting text mangled by a terminal/encoding conversion.","commonSituations":"Frontend splits/slices a UTF-16 string mid-emoji and sends the orphaned half; Python reads config with errors='surrogateescape'; a chat-template override pasted from a broken copy contains U+D800-U+DFFF alone.","solutions":["Fix the producer: encode/decode strings as proper UTF-8 end-to-end; in JS, operate on code points (Array.from) not UTF-16 code units before slicing.","Sanitize before submit: token.encode('utf-8', 'strict') client-side, or re-encode via token.encode('utf-8','surrogatepass').decode('utf-8','replace') to drop orphans.","If the arg legitimately contains the mangled text, replace the emoji/surrogate with a normal character and resubmit.","Locate the surrogate: print [hex(ord(c)) for c in token if 0xD800 <= ord(c) <= 0xDFFF]."],"exampleFix":"# before\nargs = json.loads(raw_body)[\"extra_args\"]  # may carry lone surrogates\nvalidate_extra_args(args)  # ValueError\n\n# after\ndef sanitize(tokens):\n    return [t.encode(\"utf-8\", \"replace\").decode(\"utf-8\") for t in tokens]\nvalidate_extra_args(sanitize(args))","handlingStrategy":"validation","validationCode":"def utf8_safe(tokens):\n    for t in tokens:\n        t.encode(\"utf-8\")  # raises on lone surrogates\n    return tokens","typeGuard":"def is_utf8_encodable(s) -> bool:\n    try:\n        s.encode(\"utf-8\")\n        return True\n    except UnicodeEncodeError:\n        return False","tryCatchPattern":"try:\n    validate_extra_args(args)\nexcept ValueError as e:\n    if \"surrogate\" in str(e):\n        args = [t.encode(\"utf-8\", \"replace\").decode(\"utf-8\") for t in args]\n        validate_extra_args(args)  # retry once with sanitized args\n    else:\n        raise","preventionTips":["Never decode external text with errors='surrogateescape' for values that become argv.","In JS, slice by code points (Array.from(str)) so emojis are never split into surrogate halves.","Run a strict UTF-8 round-trip check on all user-supplied config strings at ingest.","Replace or drop orphan surrogates before persistence, not at launch time."],"tags":["llama-server","unicode","surrogate","validation","encoding"],"backgroundTag":null,"analyzedSha":"203007d19051dcd2ae33876786d117c99f6b0368","analyzedAt":"2026-08-15T02:48:39.846Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}