{"record":{"id":"0a1bd7ed8483b090","repo":"unslothai/unsloth","slug":"tensor-split-entries-must-be-finite-and-non-negati","errorCode":null,"errorMessage":"tensor_split entries must be finite and non-negative","messagePattern":"tensor_split entries must be finite and non-negative","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"studio/backend/models/inference.py","lineNumber":260,"sourceCode":"        # a 422. Mirrors ModelOverrideRequest._no_booleans so /load and /settings agree.\n        # Kept off the annotation: an Annotated BeforeValidator stops the Field constraints\n        # folding into the int core schema, and they leak into OpenAPI as ge/le.\n        if isinstance(value, bool):\n            raise ValueError(\"Expected a number, got a boolean.\")\n        return value\n\n    @field_validator(\"tensor_split\")\n    @classmethod\n    def _reject_degenerate_tensor_split(cls, value: Optional[List[float]]) -> Optional[List[float]]:\n        # A negative / non-finite / all-zero split is silently dropped at launch\n        # (stored as None) yet still compared raw in the reload dedupe, so an\n        # identical Apply reloads forever. Reject it up front; [] = no split.\n        if not value:\n            return value\n        import math\n\n        if any((not math.isfinite(v)) or v < 0 for v in value):\n            raise ValueError(\"tensor_split entries must be finite and non-negative\")\n        if sum(value) <= 0:\n            raise ValueError(\"tensor_split must have a positive total\")\n        return value\n\n    llama_extra_args: Optional[List[str]] = Field(\n        None,\n        description = (\n            \"Extra arguments forwarded verbatim to llama-server for GGUF models. \"\n            \"One token per list entry, e.g. ['--top-k', '20', '--seed', '42']. \"\n            \"Unsloth-managed flags (model identity, port, context length, GPU placement, \"\n            \"auth, UI/server mode) are rejected. Ignored for non-GGUF models.\"\n        ),\n    )\n    force_cancel_active: bool = Field(\n        False,\n        description = (\n            \"Stop chats still generating instead of refusing with 409. A load \"\n            \"replaces the llama-server every open conversation decodes on.\"","sourceCodeStart":242,"sourceCodeEnd":278,"githubUrl":"https://github.com/unslothai/unsloth/blob/203007d19051dcd2ae33876786d117c99f6b0368/studio/backend/models/inference.py#L242-L278","documentation":"Raised by the tensor_split field_validator when any entry is non-finite (NaN/Inf) or negative. The comment explains why: a degenerate split is silently dropped at launch (stored as None) yet still compared raw in the reload dedupe, so an identical Apply would reload forever; rejecting up front makes the failure visible as a 422.","triggerScenarios":"POSTing tensor_split like [-1, 1.0], [0.5, NaN], or [Infinity, 0.25] with /load or /settings. JSON cannot express NaN natively, so this usually arrives via Python json.dumps(allow_nan=True) or a hand-built dict from float('nan').","commonSituations":"Computing splits from GPU memory ratios that divide by zero; serializing with the non-strict JSON default (Python emits NaN/Infinity literals); normalizing a user's '50/50' input and producing -0.0 or negative remainder; copying tensor-split examples from llama.cpp docs with typo'd signs.","solutions":["Clamp/validate entries client-side: all finite, all >= 0 (see validationCode).","Trace where NaN enters: usually a ratio computed as x/total with total == 0.","Serialize with json.dumps(..., allow_nan=False) so bad floats fail loudly on the client, not the server.","Use [] for 'no split' rather than [0, 0]."],"exampleFix":"# before\nsplit = [vram_a / total_vram for vram_a in gpus]  # total_vram can be 0 -> NaN/inf\npayload = {\"tensor_split\": split}\n\n# after\nimport math\nsplit = [vram_a / total_vram for vram_a in gpus]\nassert total_vram > 0 and all(math.isfinite(v) and v >= 0 for v in split)\npayload = {\"tensor_split\": split}","handlingStrategy":"validation","validationCode":"import math\ndef valid_tensor_split(split: list[float] | None) -> bool:\n    if not split:\n        return True\n    return all(math.isfinite(v) and v >= 0 for v in split) and sum(split) > 0","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Serialize with json.dumps(..., allow_nan=False) so NaN/Inf fail client-side","Guard ratio math against zero denominators","Use [] for 'no split', never [0, 0]"],"tags":["pydantic","validation","nan","gpu","tensor-split","llama-server"],"backgroundTag":null,"analyzedSha":"203007d19051dcd2ae33876786d117c99f6b0368","analyzedAt":"2026-08-15T02:48:39.846Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}