{"record":{"id":"4ca0b62d9ccfb263","repo":"hiyouga/LlamaFactory","slug":"device-not-supported-device-name","errorCode":null,"errorMessage":"Device not supported: {device_name}.","messagePattern":"Device not supported: (.+?)\\.","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"scripts/stat_utils/cal_mfu.py","lineNumber":98,"sourceCode":"    if include_flashattn:\n        total_flops += sdpa_flops\n\n    return total_flops\n\n\ndef compute_device_flops(world_size: int) -> float:\n    r\"\"\"Calculate the FLOPs of the device capability per second.\"\"\"\n    device_name = torch.cuda.get_device_name()\n    if \"H100\" in device_name or \"H800\" in device_name:\n        return 989 * 1e12 * world_size\n    elif \"A100\" in device_name or \"A800\" in device_name:\n        return 312 * 1e12 * world_size\n    elif \"V100\" in device_name:\n        return 125 * 1e12 * world_size\n    elif \"4090\" in device_name:\n        return 98 * 1e12 * world_size\n    else:\n        raise NotImplementedError(f\"Device not supported: {device_name}.\")\n\n\ndef calculate_mfu(\n    model_name_or_path: str,\n    batch_size: int = 1,\n    seq_length: int = 1024,\n    num_steps: int = 100,\n    finetuning_type: str = \"lora\",\n    flash_attn: str = \"auto\",\n    deepspeed_stage: int = 0,\n    disable_gc: bool = False,\n    liger_kernel: bool = False,\n    unsloth_gc: bool = False,\n) -> float:\n    r\"\"\"Calculate MFU for given model and hyper-params.\n\n    Usage: python cal_mfu.py --model_name_or_path path_to_model --batch_size 1 --seq_length 1024\n    \"\"\"","sourceCodeStart":80,"sourceCodeEnd":116,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/scripts/stat_utils/cal_mfu.py#L80-L116","documentation":"Identical guard to the DPO trainer: after Trainer.__init__ the KTO trainer requires self.accelerator (trainer.py:92), which only exists in modern transformers. On an outdated transformers the attribute is missing and the trainer stops with AttributeError('Please update `transformers`.') instead of failing later during distributed setup.","triggerScenarios":"Running KTO training (stage: kto) with transformers too old for the Trainer API the code expects; hasattr(self, 'accelerator') is False right after super().__init__.","commonSituations":"Environment pinned to an old transformers for a different model; partial upgrades where transformers was downgraded by another dependency.","solutions":["pip install -U transformers (align with LlamaFactory's declared requirements)","Confirm the active interpreter/venv actually uses the upgraded version (`python -m pip list | grep transformers`) — multiple envs are a frequent cause"],"exampleFix":"# before\ntransformers 4.3x -> AttributeError: Please update `transformers`.\n\n# after\npip install -U transformers\npython -m pip list | grep transformers","handlingStrategy":"validation","validationCode":"import transformers\nfrom packaging.version import parse\nassert parse(transformers.__version__) >= parse('4.37'), 'KTO trainer needs modern transformers; pip install -U transformers'","typeGuard":"def transformers_new_enough_for_kto() -> bool:\n    import transformers\n    from packaging.version import parse\n    return parse(transformers.__version__) >= parse('4.37.0')","tryCatchPattern":"try:\n    from llamafactory.train.kto.workflow import run_kto\n    run_kto(train_args)\nexcept AttributeError as e:\n    if 'update `transformers`' in str(e):\n        raise SystemExit('pip install -U transformers') from e\n    raise","preventionTips":["Run KTO in the same pinned env validated for DPO","Add a version preflight to job scripts covering transformers/trl/accelerate"],"tags":["version","transformers","kto","dependency"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}