{"record":{"id":"f2c1e64e3855a7ac","repo":"hiyouga/LlamaFactory","slug":"cross-entropy-and-fused-linear-cross-entropy-canno","errorCode":null,"errorMessage":"cross_entropy and fused_linear_cross_entropy cannot both be enabled.","messagePattern":"cross_entropy and fused_linear_cross_entropy cannot both be enabled\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/llamafactory/v1/plugins/model_plugins/kernels/liger_kernel_ops.py","lineNumber":122,"sourceCode":"                \"rmsnorm\": \"rms_norm\",\n                \"flce\": \"fused_linear_cross_entropy\",\n                \"lce\": \"fused_linear_cross_entropy\",\n                \"fused_ce\": \"fused_linear_cross_entropy\",\n            }\n            return aliases.get(key, key)\n\n        if use_kernels is not None and len(use_kernels) == 0:\n            return model\n\n        if use_kernels != \"auto\":\n            selected = {_normalize_op_name(k) for k in use_kernels}\n            ops = selected - set(togglable)\n            if ops:\n                raise ValueError(\n                    f\"Unknown Liger op(s) {sorted(ops)} for model_type={model_type}. Valid: {sorted(togglable)}\"\n                )\n            if \"cross_entropy\" in selected and \"fused_linear_cross_entropy\" in selected:\n                raise ValueError(\"cross_entropy and fused_linear_cross_entropy cannot both be enabled.\")\n            call_kwargs = {name: (name in selected) for name in togglable}\n            call_kwargs[\"model\"] = model\n        else:\n            # Mirror ``liger_kernel`` signature defaults so patches match upstream defaults\n            # and logging reflects enabled ops (omitted kwargs only live in the callee).\n            call_kwargs = {\"model\": model}\n            for name in togglable:\n                param = sig[name]\n                if param.default is not inspect.Parameter.empty:\n                    call_kwargs[name] = param.default\n\n        if require_logits and \"fused_linear_cross_entropy\" in sig:\n            logger.warning_rank0(\"Current training stage does not support chunked cross entropy.\")\n            call_kwargs[\"fused_linear_cross_entropy\"] = False\n            call_kwargs[\"cross_entropy\"] = True\n\n        apply_liger_kernel(**call_kwargs)\n","sourceCodeStart":104,"sourceCodeEnd":140,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/src/llamafactory/v1/plugins/model_plugins/kernels/liger_kernel_ops.py#L104-L140","documentation":"Within Liger's togglable ops, cross_entropy and fused_linear_cross_entropy are mutually exclusive strategies for the LM head + loss (fused skips materializing logits). Enabling both in one use_kernels list is contradictory and raises ValueError before patching.","triggerScenarios":"use_kernels containing both names (directly or via aliases, e.g. [\"cross_entropy\", \"fused_ce\"]) for a model whose signature exposes both toggles.","commonSituations":"Assembling an 'all ops on' list by enumerating every signature parameter; merging configs from two sources each enabling one variant; alias confusion (lce/fused_ce resolving to the fused variant).","solutions":["Keep exactly one: fused_linear_cross_entropy for fused speed (when logits not needed), cross_entropy otherwise.","If require_logits is true, prefer plain cross_entropy — the code already forces non-fused CE in that case.","Prefer use_kernels=\"auto\" which resolves defaults without conflicts."],"exampleFix":"# before\nuse_kernels = [\"cross_entropy\", \"fused_ce\"]\n\n# after\nuse_kernels = [\"fused_linear_cross_entropy\"]","handlingStrategy":"validation","validationCode":"ALIASES = {'lce': 'fused_linear_cross_entropy', 'fused_ce': 'fused_linear_cross_entropy'}\nnorm = {ALIASES.get(k, k) for k in use_kernels}\nassert not ({'cross_entropy', 'fused_linear_cross_entropy'} <= norm), 'mutually exclusive CE ops both enabled'","typeGuard":"def ce_ops_consistent(ops: list[str]) -> bool:\n    \"\"\"True when at most one of the two cross-entropy strategies is selected.\"\"\"\n    norm = {ALIASES.get(k, k) for k in ops}\n    return not ({'cross_entropy', 'fused_linear_cross_entropy'} <= norm)","tryCatchPattern":"try:\n    model = KernelPlugin('liger_kernel').apply(model=model, use_kernels=ops)\nexcept ValueError as e:\n    if 'cannot both be enabled' in str(e):\n        ops = [o for o in ops if o != 'cross_entropy']\n        model = KernelPlugin('liger_kernel').apply(model=model, use_kernels=ops)\n    else:\n        raise","preventionTips":["Never enumerate 'all ops' blindly; pick one CE strategy deliberately.","Expand aliases (lce, fused_ce) before validating configs.","When logits are needed (require_logits), choose plain cross_entropy."],"tags":["kernels","liger","config","conflict"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}