unslothai/unsloth · error · ValueError

tensor_split entries must be finite and non-negative

Error message

tensor_split entries must be finite and non-negative

What it means

Raised by the tensor_split field_validator when any entry is non-finite (NaN/Inf) or negative. The comment explains why: a degenerate split is silently dropped at launch (stored as None) yet still compared raw in the reload dedupe, so an identical Apply would reload forever; rejecting up front makes the failure visible as a 422.

Source

Thrown at studio/backend/models/inference.py:260

        # a 422. Mirrors ModelOverrideRequest._no_booleans so /load and /settings agree.
        # Kept off the annotation: an Annotated BeforeValidator stops the Field constraints
        # folding into the int core schema, and they leak into OpenAPI as ge/le.
        if isinstance(value, bool):
            raise ValueError("Expected a number, got a boolean.")
        return value

    @field_validator("tensor_split")
    @classmethod
    def _reject_degenerate_tensor_split(cls, value: Optional[List[float]]) -> Optional[List[float]]:
        # A negative / non-finite / all-zero split is silently dropped at launch
        # (stored as None) yet still compared raw in the reload dedupe, so an
        # identical Apply reloads forever. Reject it up front; [] = no split.
        if not value:
            return value
        import math

        if any((not math.isfinite(v)) or v < 0 for v in value):
            raise ValueError("tensor_split entries must be finite and non-negative")
        if sum(value) <= 0:
            raise ValueError("tensor_split must have a positive total")
        return value

    llama_extra_args: Optional[List[str]] = Field(
        None,
        description = (
            "Extra arguments forwarded verbatim to llama-server for GGUF models. "
            "One token per list entry, e.g. ['--top-k', '20', '--seed', '42']. "
            "Unsloth-managed flags (model identity, port, context length, GPU placement, "
            "auth, UI/server mode) are rejected. Ignored for non-GGUF models."
        ),
    )
    force_cancel_active: bool = Field(
        False,
        description = (
            "Stop chats still generating instead of refusing with 409. A load "
            "replaces the llama-server every open conversation decodes on."

View on GitHub (pinned to 203007d190)

Solutions

1. Clamp/validate entries client-side: all finite, all >= 0 (see validationCode).
2. Trace where NaN enters: usually a ratio computed as x/total with total == 0.
3. Serialize with json.dumps(..., allow_nan=False) so bad floats fail loudly on the client, not the server.
4. Use [] for 'no split' rather than [0, 0].

Example fix

# before
split = [vram_a / total_vram for vram_a in gpus]  # total_vram can be 0 -> NaN/inf
payload = {"tensor_split": split}

# after
import math
split = [vram_a / total_vram for vram_a in gpus]
assert total_vram > 0 and all(math.isfinite(v) and v >= 0 for v in split)
payload = {"tensor_split": split}

import math
def valid_tensor_split(split: list[float] | None) -> bool:
    if not split:
        return True
    return all(math.isfinite(v) and v >= 0 for v in split) and sum(split) > 0

When it happens

Trigger: POSTing tensor_split like [-1, 1.0], [0.5, NaN], or [Infinity, 0.25] with /load or /settings. JSON cannot express NaN natively, so this usually arrives via Python json.dumps(allow_nan=True) or a hand-built dict from float('nan').

Common situations: Computing splits from GPU memory ratios that divide by zero; serializing with the non-strict JSON default (Python emits NaN/Infinity literals); normalizing a user's '50/50' input and producing -0.0 or negative remainder; copying tensor-split examples from llama.cpp docs with typo'd signs.

Related errors

• tensor_split must have a positive total
• Expected a number, got a boolean.
• too many extra llama-server args (limit {MAX_EXTRA_ARG_TOKEN
• extra llama-server args cannot contain unpaired surrogate ch
• extra llama-server args are too large (limit {limit} bytes)

AI-assisted analysis of unslothai/unsloth@203007d190 (2026-08-15). Data as JSON: /api/errors/0a1bd7ed8483b090. Report an issue: GitHub.