unslothai/unsloth · error · ValueError

too many extra llama-server args (limit {MAX_EXTRA_ARG_TOKEN

Error message

too many extra llama-server args (limit {MAX_EXTRA_ARG_TOKENS} tokens)

What it means

ValueError from validate_extra_args in llama_server_args.py:249 — the user-supplied 'extra args' list passed through to llama-server is capped at MAX_EXTRA_ARG_TOKENS tokens. The cap is on the whole list (a grammar or JSON schema is a legitimately long single token), so this fires when the list has too many elements, not when one is long. It is refused at the request boundary (a 400) rather than letting a giant argv reach subprocess spawn.

Source

Thrown at studio/backend/core/inference/llama_server_args.py:249

def validate_extra_args(args: Optional[Iterable[str]]) -> list[str]:
    """Validate user-supplied llama-server args. Returns a flat list ready to
    extend the llama-server command; raises ``ValueError`` naming the
    offending flag on the first managed token."""
    if not args:
        return []
    out: list[str] = []
    total_bytes = 0
    # How many following tokens the flag just seen may still claim as values. A
    # switch claims none, so the next bare token has no owner.
    pending_values = 0
    # Values still owed to a two-value flag, tracked apart because it is the one
    # arity this module knows for certain.
    pending_two_value = 0
    two_value_flag = ""
    for raw in args:
        token = str(raw)
        if len(out) >= MAX_EXTRA_ARG_TOKENS:
            raise ValueError(
                f"too many extra llama-server args (limit {MAX_EXTRA_ARG_TOKENS} tokens)"
            )
        # A grammar or JSON schema is a legitimately long single token, so the cap
        # is on the whole list rather than per token.
        # Strictly, unlike the sizing below: JSON and the browser can both carry an
        # unpaired surrogate, which survives every check here and then makes
        # subprocess.Popen raise while it encodes argv, long after the load has begun
        # switching models. Refused at the boundary, where it is still a 400.
        try:
            encoded = token.encode("utf-8")
        except UnicodeEncodeError as error:
            raise ValueError(
                "extra llama-server args cannot contain unpaired surrogate characters"
            ) from error
        total_bytes += len(encoded)
        limit = max_extra_args_bytes()
        if total_bytes > limit:
            raise ValueError(f"extra llama-server args are too large (limit {limit} bytes)")

View on GitHub (pinned to 203007d190)

Solutions

Trim the extra args list to the essentials — drop duplicate flags and overrides the server defaults anyway.
If many key=value overrides are needed, move them into a single long token (e.g. one grammar/schema blob) or a config file the server reads, rather than many small tokens.
Raise MAX_EXTRA_ARG_TOKENS if you control the deployment and genuinely need more tokens (it is a module constant in llama_server_args.py).
Check the exact limit from the error message and count your tokens (each array element = one token).

Example fix

# before
extra_args = [f"--override-kv", f"key{i}=1" for i in range(500)]  # 1000 tokens

# after
extra_args = ["--override-kv", "only.needed.key=1", "--flash-attn", "on"]

Defensive patterns

Strategy: validation

Validate before calling

from core.inference.llama_server_args import MAX_EXTRA_ARG_TOKENS
if len(extra_args) > MAX_EXTRA_ARG_TOKENS:
    raise HTTPException(400, f"extra args limited to {MAX_EXTRA_ARG_TOKENS} tokens")

Type guard

def args_within_token_limit(args, limit) -> bool:
    return len(list(args)) <= limit

Try / catch

try:
    validate_extra_args(extra_args)
except ValueError as e:
    raise HTTPException(400, str(e))

Prevention

Count tokens (list elements) before submit; each element is one token.
Consolidate many overrides into fewer tokens or a config file.
Deduplicate flags in stored configs (UIs often append on each save).
Read the limit from the module constant rather than hardcoding.

When it happens

Trigger: POSTing/Persisting a model config whose extra_args array has more than MAX_EXTRA_ARG_TOKENS entries (e.g. enumerating many --override-kv key=value pairs or a long flag soup).

Common situations: Script-generated arg lists (one --override-kv per LoRA / per tensor) blowing past the cap; copy-pasting a whole llama.cpp launch line into the extra-args field; a config UI appending duplicate flags on each save.

Related errors

AI-assisted analysis of unslothai/unsloth@203007d190 (2026-08-15). Data as JSON: /api/errors/0b38016c4f1ced2e. Report an issue: GitHub.