{"record":{"id":"3048868838e99681","repo":"hiyouga/LlamaFactory","slug":"deepspeed-only-supports-bf16-mixed-precision-for-n","errorCode":null,"errorMessage":"DeepSpeed only supports bf16 mixed precision for now, fp16 is not supported.","messagePattern":"DeepSpeed only supports bf16 mixed precision for now, fp16 is not supported\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/llamafactory/v1/plugins/model_plugins/deepspeed_utils.py","lineNumber":64,"sourceCode":"        if value_lower == \"true\":\n            return True\n        if value_lower == \"false\":\n            return False\n        if value_lower == \"auto\":\n            return \"auto\"\n    return value\n\n\ndef infer_deepspeed_mixed_precision(ds_config: dict[str, Any]) -> str:\n    ds_config.setdefault(\"fp16\", {})\n    ds_config.setdefault(\"bf16\", {})\n\n    fp16_enabled = _normalize_precision_enabled(ds_config[\"fp16\"].get(\"enabled\", \"auto\"))\n    bf16_enabled = _normalize_precision_enabled(ds_config[\"bf16\"].get(\"enabled\", \"auto\"))\n\n    # This project only supports DeepSpeed bf16 or no mixed precision.\n    if fp16_enabled is True:\n        raise ValueError(\"DeepSpeed only supports bf16 mixed precision for now, fp16 is not supported.\")\n\n    if bf16_enabled is True:\n        mixed_precision = \"bf16\"\n    elif bf16_enabled is False:\n        mixed_precision = \"no\"\n    elif fp16_enabled is False:\n        mixed_precision = \"no\"\n    else:\n        # When both bf16/fp16 are left as auto (or absent), default to bf16.\n        mixed_precision = \"bf16\"\n\n    ds_config[\"fp16\"][\"enabled\"] = False\n    ds_config[\"bf16\"][\"enabled\"] = mixed_precision == \"bf16\"\n    return mixed_precision\n\n\ndef _unset_hf_deepspeed_config() -> None:\n    try:","sourceCodeStart":46,"sourceCodeEnd":82,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/src/llamafactory/v1/plugins/model_plugins/deepspeed_utils.py#L46-L82","documentation":"The v1 DeepSpeed integration deliberately supports only bf16 (or no) mixed precision. If the DeepSpeed config file explicitly enables fp16 (\"fp16\": {\"enabled\": true}), infer_deepspeed_mixed_precision raises immediately rather than silently training with numerically worse fp16.","triggerScenarios":"A DeepSpeed JSON config with \"fp16\": {\"enabled\": true} (or truthy non-auto value) passed via the dist config's config_file under USE_V1.","commonSituations":"Reusing a community DeepSpeed config written for older V100-era fp16 training; copying examples from other frameworks (axolotl/trl) that set fp16.enabled; hardware without bf16 support prompting users to flip to fp16.","solutions":["Set \"fp16\": {\"enabled\": false} and \"bf16\": {\"enabled\": true} in the DeepSpeed config.","Leave both as \"auto\" — the code defaults auto/auto to bf16.","If the GPU genuinely lacks bf16 (pre-Ampere), use v0 with fp16 or the non-DeepSpeed path instead; do not fight the v1 restriction.","After editing, verify the JSON is valid and actually the file referenced by config_file."],"exampleFix":"// before (ds_config.json)\n\"fp16\": {\"enabled\": true},\n\"bf16\": {\"enabled\": false}\n\n// after\n\"fp16\": {\"enabled\": false},\n\"bf16\": {\"enabled\": true}","handlingStrategy":"validation","validationCode":"import json\ncfg = json.load(open(ds_config_path))\nassert cfg.get('fp16', {}).get('enabled', 'auto') is not True, 'v1 DeepSpeed forbids fp16; enable bf16'","typeGuard":null,"tryCatchPattern":"try:\n    setup_deepspeed(...)\nexcept ValueError as e:\n    if 'fp16 is not supported' in str(e):\n        raise SystemExit('set fp16.enabled=false and bf16.enabled=true in the DS config') from None\n    raise","preventionTips":["Lint DeepSpeed JSONs for fp16.enabled=true before launching v1 runs.","Keep one bf16-approved DS config per team and template from it.","On pre-Ampere GPUs, plan for the v0/fp16 path instead of editing the v1 config."],"tags":["deepspeed","precision","config","distributed"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}