{"record":{"id":"0b38016c4f1ced2e","repo":"unslothai/unsloth","slug":"too-many-extra-llama-server-args-limit-max-extra","errorCode":null,"errorMessage":"too many extra llama-server args (limit {MAX_EXTRA_ARG_TOKENS} tokens)","messagePattern":"too many extra llama-server args \\(limit (.+?) tokens\\)","errorType":"validation","errorClass":"ValueError","httpStatus":400,"severity":"error","filePath":"studio/backend/core/inference/llama_server_args.py","lineNumber":249,"sourceCode":"def validate_extra_args(args: Optional[Iterable[str]]) -> list[str]:\n    \"\"\"Validate user-supplied llama-server args. Returns a flat list ready to\n    extend the llama-server command; raises ``ValueError`` naming the\n    offending flag on the first managed token.\"\"\"\n    if not args:\n        return []\n    out: list[str] = []\n    total_bytes = 0\n    # How many following tokens the flag just seen may still claim as values. A\n    # switch claims none, so the next bare token has no owner.\n    pending_values = 0\n    # Values still owed to a two-value flag, tracked apart because it is the one\n    # arity this module knows for certain.\n    pending_two_value = 0\n    two_value_flag = \"\"\n    for raw in args:\n        token = str(raw)\n        if len(out) >= MAX_EXTRA_ARG_TOKENS:\n            raise ValueError(\n                f\"too many extra llama-server args (limit {MAX_EXTRA_ARG_TOKENS} tokens)\"\n            )\n        # A grammar or JSON schema is a legitimately long single token, so the cap\n        # is on the whole list rather than per token.\n        # Strictly, unlike the sizing below: JSON and the browser can both carry an\n        # unpaired surrogate, which survives every check here and then makes\n        # subprocess.Popen raise while it encodes argv, long after the load has begun\n        # switching models. Refused at the boundary, where it is still a 400.\n        try:\n            encoded = token.encode(\"utf-8\")\n        except UnicodeEncodeError as error:\n            raise ValueError(\n                \"extra llama-server args cannot contain unpaired surrogate characters\"\n            ) from error\n        total_bytes += len(encoded)\n        limit = max_extra_args_bytes()\n        if total_bytes > limit:\n            raise ValueError(f\"extra llama-server args are too large (limit {limit} bytes)\")","sourceCodeStart":231,"sourceCodeEnd":267,"githubUrl":"https://github.com/unslothai/unsloth/blob/203007d19051dcd2ae33876786d117c99f6b0368/studio/backend/core/inference/llama_server_args.py#L231-L267","documentation":"ValueError from validate_extra_args in llama_server_args.py:249 — the user-supplied 'extra args' list passed through to llama-server is capped at MAX_EXTRA_ARG_TOKENS tokens. The cap is on the whole list (a grammar or JSON schema is a legitimately long single token), so this fires when the list has too many elements, not when one is long. It is refused at the request boundary (a 400) rather than letting a giant argv reach subprocess spawn.","triggerScenarios":"POSTing/Persisting a model config whose extra_args array has more than MAX_EXTRA_ARG_TOKENS entries (e.g. enumerating many --override-kv key=value pairs or a long flag soup).","commonSituations":"Script-generated arg lists (one --override-kv per LoRA / per tensor) blowing past the cap; copy-pasting a whole llama.cpp launch line into the extra-args field; a config UI appending duplicate flags on each save.","solutions":["Trim the extra args list to the essentials — drop duplicate flags and overrides the server defaults anyway.","If many key=value overrides are needed, move them into a single long token (e.g. one grammar/schema blob) or a config file the server reads, rather than many small tokens.","Raise MAX_EXTRA_ARG_TOKENS if you control the deployment and genuinely need more tokens (it is a module constant in llama_server_args.py).","Check the exact limit from the error message and count your tokens (each array element = one token)."],"exampleFix":"# before\nextra_args = [f\"--override-kv\", f\"key{i}=1\" for i in range(500)]  # 1000 tokens\n\n# after\nextra_args = [\"--override-kv\", \"only.needed.key=1\", \"--flash-attn\", \"on\"]","handlingStrategy":"validation","validationCode":"from core.inference.llama_server_args import MAX_EXTRA_ARG_TOKENS\nif len(extra_args) > MAX_EXTRA_ARG_TOKENS:\n    raise HTTPException(400, f\"extra args limited to {MAX_EXTRA_ARG_TOKENS} tokens\")","typeGuard":"def args_within_token_limit(args, limit) -> bool:\n    return len(list(args)) <= limit","tryCatchPattern":"try:\n    validate_extra_args(extra_args)\nexcept ValueError as e:\n    raise HTTPException(400, str(e))","preventionTips":["Count tokens (list elements) before submit; each element is one token.","Consolidate many overrides into fewer tokens or a config file.","Deduplicate flags in stored configs (UIs often append on each save).","Read the limit from the module constant rather than hardcoding."],"tags":["llama-server","validation","argv","configuration"],"backgroundTag":null,"analyzedSha":"203007d19051dcd2ae33876786d117c99f6b0368","analyzedAt":"2026-08-15T02:48:39.846Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}