mudler/LocalAI · error · ValueError

use_int8 is supported only by LongCat-Video-Avatar-1.5

Error message

use_int8 is supported only by LongCat-Video-Avatar-1.5

What it means

ValueError raised at longcat-video load time when use_int8=true but the loaded model classifies as MODEL_KIND_BASE (LongCat-Video base) rather than the Avatar-1.5 variant. INT8 weights only exist for LongCat-Video-Avatar-1.5 (the base_model_int8 subfolder its loader looks for), so requesting int8 on the base model is rejected before download/load.

Source

Thrown at backend/python/longcat-video/backend.py:176

                    grpc.StatusCode.UNIMPLEMENTED,
                    "longcat-video currently supports one GPU per backend process",
                )
            self._import_runtime()

            attention_name = str(options.get("attention_backend", "sdpa")).lower()
            attention_overrides(attention_name)
            resolution = str(options.get("resolution", "480p")).lower()
            if resolution not in {"480p", "720p"}:
                raise ValueError("resolution must be 480p or 720p")

            use_distill_default = model_kind == MODEL_KIND_AVATAR
            use_distill = require_bool(
                options.get("use_distill", use_distill_default),
                "use_distill",
            )
            use_int8 = require_bool(options.get("use_int8", False), "use_int8")
            if model_kind == MODEL_KIND_BASE and use_int8:
                raise ValueError(
                    "use_int8 is supported only by LongCat-Video-Avatar-1.5"
                )

            self.options = {
                **options,
                "attention_backend": attention_name,
                "resolution": resolution,
                "use_distill": use_distill,
                "use_int8": use_int8,
                "max_segments": require_int(
                    options.get("max_segments", 8),
                    "max_segments",
                    minimum=1,
                    maximum=64,
                ),
            }

            self._release_model()

View on GitHub (pinned to 44413a9d06)

Solutions

1. Remove use_int8 (or set it to false) when loading the base LongCat-Video model
2. Or switch the model to LongCat-Video-Avatar-1.5 if int8 quantization is required

Example fix

# before
model: LongCat-Video/Live-LongCat-Video
options:
  use_int8: true

# after
model: LongCat-Video/Live-LongCat-Video
options:
  use_int8: false

def validate_longcat_load_options(model_id: str, options: dict) -> dict:
    is_avatar = "avatar" in model_id.lower()  # mirror of classify_model heuristics
    if options.get("use_int8") and not is_avatar:
        options = {k: v for k, v in options.items() if k != "use_int8"}
        # or raise, if silent dropping is undesirable
    return options

try:
    stub.LoadModel(opts)
except grpc.RpcError as e:
    if "use_int8" in (e.details() or ""):
        opts["options"].pop("use_int8", None)
        stub.LoadModel(opts)
    else:
        raise

When it happens

Trigger: LoadModel with model=LongCat-Video (base) and options {"use_int8": true}; pointing at a base checkpoint while keeping use_int8 from a previous avatar config.

Common situations: Copying an avatar model's option block onto the base model to save VRAM; renaming model repos and the kind inference now classifies as base.

Related errors

• resolution must be 480p or 720p
• num_frames must not be negative
• base_model must point to a LongCat-Video checkpoint
• request needs {segments} avatar segments, but max_segments i
• failed to mux avatar audio: {details}

AI-assisted analysis of mudler/LocalAI@44413a9d06 (2026-08-15). Data as JSON: /api/errors/584c53b12391101c. Report an issue: GitHub.