hiyouga/LlamaFactory · error · ValueError

Unknown Liger op(s) {sorted(ops)} for model_type={model_type

Error message

Unknown Liger op(s) {sorted(ops)} for model_type={model_type}. Valid: {sorted(togglable)}

What it means

When use_kernels is an explicit list (not 'auto'), each requested Liger op is normalized (aliases like lce/fused_ce -> fused_linear_cross_entropy) and must exist in the togglable set for the current model_type. Unknown ops raise ValueError listing the invalid names and the valid set for that architecture.

Source

Thrown at src/llamafactory/v1/plugins/model_plugins/kernels/liger_kernel_ops.py:118

        def _normalize_op_name(raw: str) -> str:
            key = raw.strip().lower().replace("-", "_")
            aliases = {
                "rmsnorm": "rms_norm",
                "flce": "fused_linear_cross_entropy",
                "lce": "fused_linear_cross_entropy",
                "fused_ce": "fused_linear_cross_entropy",
            }
            return aliases.get(key, key)

        if use_kernels is not None and len(use_kernels) == 0:
            return model

        if use_kernels != "auto":
            selected = {_normalize_op_name(k) for k in use_kernels}
            ops = selected - set(togglable)
            if ops:
                raise ValueError(
                    f"Unknown Liger op(s) {sorted(ops)} for model_type={model_type}. Valid: {sorted(togglable)}"
                )
            if "cross_entropy" in selected and "fused_linear_cross_entropy" in selected:
                raise ValueError("cross_entropy and fused_linear_cross_entropy cannot both be enabled.")
            call_kwargs = {name: (name in selected) for name in togglable}
            call_kwargs["model"] = model
        else:
            # Mirror ``liger_kernel`` signature defaults so patches match upstream defaults
            # and logging reflects enabled ops (omitted kwargs only live in the callee).
            call_kwargs = {"model": model}
            for name in togglable:
                param = sig[name]
                if param.default is not inspect.Parameter.empty:
                    call_kwargs[name] = param.default

        if require_logits and "fused_linear_cross_entropy" in sig:
            logger.warning_rank0("Current training stage does not support chunked cross entropy.")
            call_kwargs["fused_linear_cross_entropy"] = False

View on GitHub (pinned to f28afaf635)

Solutions

1. Read the error: it prints the valid ops for your exact model_type — use only those.
2. Fix typos and stale names; check the model's apply_liger_kernel_to_<model_type> signature in your installed liger_kernel version.
3. Use use_kernels="auto" to take the signature defaults for the model.
4. Upgrade/downgrade liger-kernel to the version whose op names match your config.

Example fix

# before
use_kernels = ["fused_ce", "swiglu"]  # swiglu not togglable for this model

# after
use_kernels = "auto"  # or list only names from the error's 'Valid:' set

import inspect
from liger_kernel.transformers import monkey_patch
fn = getattr(monkey_patch, f'apply_liger_kernel_to_{model_type}', None)
assert fn is not None, f'no liger patch fn for model_type={model_type}'
togglable = {p for p in inspect.signature(fn).parameters if p != 'model'}
unknown = set(use_kernels) - togglable - {'lce', 'fused_ce'}
assert not unknown, f'unknown ops {unknown}; valid: {sorted(togglable)}'

def ops_valid_for(model_type: str, ops: list[str]) -> bool:
    """True when every requested op is togglable for this model_type."""
    fn = getattr(monkey_patch, f'apply_liger_kernel_to_{model_type}', None)
    if fn is None:
        return False
    togglable = set(inspect.signature(fn).parameters) - {'model'}
    return set(ops) <= togglable | {'lce', 'fused_ce'}

try:
    model = KernelPlugin('liger_kernel').apply(model=model, use_kernels=ops)
except ValueError as e:
    if 'Unknown Liger op' in str(e):
        use_kernels = 'auto'  # fall back to defaults
        model = KernelPlugin('liger_kernel').apply(model=model, use_kernels=use_kernels)
    else:
        raise

When it happens

Trigger: Passing use_kernels=["swiglu"] for a model whose apply_liger_kernel_to_* signature exposes no swiglu toggle; typo'd op names; using ops valid for LLaMA on a different architecture (e.g. Qwen MoE-specific ops).

Common situations: Copying a v0 enable_liger_kernel option list tuned for one model family to another; renaming drift between liger-kernel versions (ops added/renamed upstream); stale docs.

Related errors

• kernel_config.name must be a string.
• kernel_config.name must contain at least one kernel name.
• cross_entropy and fused_linear_cross_entropy cannot both be
• Unknown mixing strategy: {data_args.mix_strategy}.
• Cannot specify `val_size` if `eval_dataset` is not None.

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/6f6084f42df821f2. Report an issue: GitHub.