hiyouga/LlamaFactory · error · ValueError

DeepSpeed only supports bf16 mixed precision for now, fp16 i

Error message

DeepSpeed only supports bf16 mixed precision for now, fp16 is not supported.

What it means

The v1 DeepSpeed integration deliberately supports only bf16 (or no) mixed precision. If the DeepSpeed config file explicitly enables fp16 ("fp16": {"enabled": true}), infer_deepspeed_mixed_precision raises immediately rather than silently training with numerically worse fp16.

Source

Thrown at src/llamafactory/v1/plugins/model_plugins/deepspeed_utils.py:64

        if value_lower == "true":
            return True
        if value_lower == "false":
            return False
        if value_lower == "auto":
            return "auto"
    return value


def infer_deepspeed_mixed_precision(ds_config: dict[str, Any]) -> str:
    ds_config.setdefault("fp16", {})
    ds_config.setdefault("bf16", {})

    fp16_enabled = _normalize_precision_enabled(ds_config["fp16"].get("enabled", "auto"))
    bf16_enabled = _normalize_precision_enabled(ds_config["bf16"].get("enabled", "auto"))

    # This project only supports DeepSpeed bf16 or no mixed precision.
    if fp16_enabled is True:
        raise ValueError("DeepSpeed only supports bf16 mixed precision for now, fp16 is not supported.")

    if bf16_enabled is True:
        mixed_precision = "bf16"
    elif bf16_enabled is False:
        mixed_precision = "no"
    elif fp16_enabled is False:
        mixed_precision = "no"
    else:
        # When both bf16/fp16 are left as auto (or absent), default to bf16.
        mixed_precision = "bf16"

    ds_config["fp16"]["enabled"] = False
    ds_config["bf16"]["enabled"] = mixed_precision == "bf16"
    return mixed_precision


def _unset_hf_deepspeed_config() -> None:
    try:

View on GitHub (pinned to f28afaf635)

Solutions

Set "fp16": {"enabled": false} and "bf16": {"enabled": true} in the DeepSpeed config.
Leave both as "auto" — the code defaults auto/auto to bf16.
If the GPU genuinely lacks bf16 (pre-Ampere), use v0 with fp16 or the non-DeepSpeed path instead; do not fight the v1 restriction.
After editing, verify the JSON is valid and actually the file referenced by config_file.

Example fix

// before (ds_config.json)
"fp16": {"enabled": true},
"bf16": {"enabled": false}

// after
"fp16": {"enabled": false},
"bf16": {"enabled": true}

Defensive patterns

Strategy: validation

Validate before calling

import json
cfg = json.load(open(ds_config_path))
assert cfg.get('fp16', {}).get('enabled', 'auto') is not True, 'v1 DeepSpeed forbids fp16; enable bf16'

Try / catch

try:
    setup_deepspeed(...)
except ValueError as e:
    if 'fp16 is not supported' in str(e):
        raise SystemExit('set fp16.enabled=false and bf16.enabled=true in the DS config') from None
    raise

Prevention

Lint DeepSpeed JSONs for fp16.enabled=true before launching v1 runs.
Keep one bf16-approved DS config per team and template from it.
On pre-Ampere GPUs, plan for the v0/fp16 path instead of editing the v1 config.

When it happens

Trigger: A DeepSpeed JSON config with "fp16": {"enabled": true} (or truthy non-auto value) passed via the dist config's config_file under USE_V1.

Common situations: Reusing a community DeepSpeed config written for older V100-era fp16 training; copying examples from other frameworks (axolotl/trl) that set fp16.enabled; hardware without bf16 support prompting users to flip to fp16.

Related errors

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/3048868838e99681. Report an issue: GitHub.