unslothai/unsloth · error · ValueError

Expected a number, got a boolean.

Error message

Expected a number, got a boolean.

What it means

Raised by a mode='before' field_validator on n_batch/n_ubatch when the incoming JSON value is a Python bool. Because bool subclasses int and Pydantic parses non-strictly, `true` would silently coerce to 1 and the load would launch llama-server with --batch-size 1, which aborts and surfaces as a 500. The validator converts that into a clean 422 at request time.

Source

Thrown at studio/backend/models/inference.py:246

        description = (
            "Manual mode only: relative share of the model per GPU (--tensor-split), "
            "in the order of the GPUs in use, e.g. [2, 1] for 2:1. Omit it to let "
            "llama.cpp use its default, which splits by free VRAM. Any list given is "
            "passed through as-is, so send [1, 1] to force an even split. Ignored "
            "unless gpu_memory_mode is 'manual' with gpu_layers >= 0."
        ),
    )

    @field_validator("n_batch", "n_ubatch", mode = "before")
    @classmethod
    def _no_booleans(cls, value: Any) -> Any:
        # bool subclasses int and pydantic parses non-strictly, so `true` arrives as 1 and
        # the load launches --batch-size 1, which llama-server aborts on: a 500 rather than
        # a 422. Mirrors ModelOverrideRequest._no_booleans so /load and /settings agree.
        # Kept off the annotation: an Annotated BeforeValidator stops the Field constraints
        # folding into the int core schema, and they leak into OpenAPI as ge/le.
        if isinstance(value, bool):
            raise ValueError("Expected a number, got a boolean.")
        return value

    @field_validator("tensor_split")
    @classmethod
    def _reject_degenerate_tensor_split(cls, value: Optional[List[float]]) -> Optional[List[float]]:
        # A negative / non-finite / all-zero split is silently dropped at launch
        # (stored as None) yet still compared raw in the reload dedupe, so an
        # identical Apply reloads forever. Reject it up front; [] = no split.
        if not value:
            return value
        import math

        if any((not math.isfinite(v)) or v < 0 for v in value):
            raise ValueError("tensor_split entries must be finite and non-negative")
        if sum(value) <= 0:
            raise ValueError("tensor_split must have a positive total")
        return value

View on GitHub (pinned to 203007d190)

Solutions

1. Change the JSON payload to a real integer, e.g. "n_batch": 512 instead of true.
2. Fix the client-side config typing so numeric knobs are ints, not bools.
3. If the value comes from YAML/env, coerce explicitly with int(value) after a bool check.

Example fix

# before
cfg = {"n_batch": True}  # coerced to 1 -> llama-server abort

# after
cfg = {"n_batch": 512}

def normalize_int_field(value):
    if isinstance(value, bool):
        raise TypeError('boolean where an integer is required')
    return int(value)

def is_plain_int(v) -> bool:
    return isinstance(v, int) and not isinstance(v, bool)

When it happens

Trigger: Sending {"n_batch": true} or {"n_ubatch": false} (or a YAML/env-derived value that a config layer typed as bool) to /load or /settings. The same guard exists on ModelOverrideRequest so both endpoints agree.

Common situations: Hand-written JSON configs where true was used instead of 1; templating engines that render booleans for numeric options; YAML 1.1 parsers coercing 'yes'/'no' to bool; client code doing `n_batch: use_fast and 512` style expressions that evaluate to a bool.

Related errors

• tensor_split entries must be finite and non-negative
• tensor_split must have a positive total
• too many extra llama-server args (limit {MAX_EXTRA_ARG_TOKEN
• extra llama-server args cannot contain unpaired surrogate ch
• extra llama-server args are too large (limit {limit} bytes)

AI-assisted analysis of unslothai/unsloth@203007d190 (2026-08-15). Data as JSON: /api/errors/1f37a3c3e1e46e42. Report an issue: GitHub.