hiyouga/LlamaFactory · error · ValueError

Quantization dataset is necessary for exporting.

Error message

Quantization dataset is necessary for exporting.

What it means

Raised in ExportArguments.__post_init__ (model_args.py:397) when export_quantization_bit is set but export_quantization_dataset is None. Post-training quantization (e.g. GPTQ/AWQ-style export) requires calibration data to collect activation statistics, so exporting a quantized model without a dataset is impossible. Checked at export-argument construction time.

Source

Thrown at src/llamafactory/hparams/model_args.py:397

        default=128,
        metadata={"help": "The number of samples used for quantization."},
    )
    export_quantization_maxlen: int = field(
        default=1024,
        metadata={"help": "The maximum length of the model inputs used for quantization."},
    )
    export_legacy_format: bool = field(
        default=False,
        metadata={"help": "Whether or not to save the `.bin` files instead of `.safetensors`."},
    )
    export_hub_model_id: str | None = field(
        default=None,
        metadata={"help": "The name of the repository if push the model to the Hugging Face hub."},
    )

    def __post_init__(self):
        if self.export_quantization_bit is not None and self.export_quantization_dataset is None:
            raise ValueError("Quantization dataset is necessary for exporting.")


@dataclass
class VllmArguments:
    r"""Arguments pertaining to the vLLM worker."""

    vllm_maxlen: int = field(
        default=4096,
        metadata={"help": "Maximum sequence (prompt + response) length of the vLLM engine."},
    )
    vllm_gpu_util: float = field(
        default=0.7,
        metadata={"help": "The fraction of GPU memory in (0,1) to be used for the vLLM engine."},
    )
    vllm_enforce_eager: bool = field(
        default=False,
        metadata={"help": "Whether or not to disable CUDA graph in the vLLM engine."},
    )

View on GitHub (pinned to f28afaf635)

Solutions

1. Add export_quantization_dataset with a dataset name registered in data/dataset_info.json (e.g. 'wikitext')
2. Verify the dataset name exists in dataset_info.json
3. If you actually want an unquantized export, remove export_quantization_bit

Example fix

# before
export_dir: saves/exported
export_quantization_bit: 8

# after
export_dir: saves/exported
export_quantization_bit: 8
export_quantization_dataset: wikitext

if cfg.get('export_quantization_bit') is not None:
    assert cfg.get('export_quantization_dataset'), 'calibration dataset required for quantized export'

def quantized_export_ready(cfg: dict) -> bool:
    return cfg.get('export_quantization_bit') is None or bool(cfg.get('export_quantization_dataset'))

When it happens

Trigger: Running export with export_quantization_bit: 8 but no export_quantization_dataset; a template export YAML where the dataset line is commented out; using a JSON config that includes the bit but not the dataset key.

Common situations: First-time quantized exports copied from a full example with the dataset section trimmed; users assuming quantization needs no data (weight-only rounding intuition); renaming the dataset key (it is export_quantization_dataset, a dataset name from data/dataset_info.json, not a file path).

Related errors

• Please provide `model_name_or_path`.
• Cannot find satisfying example, considering decrease `export
• AutoGPTQ only accepts 2/3/4/8-bit quantization.
• Please merge adapters before quantizing the model.
• Cannot merge adapters to a quantized model.

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/90ab6575e84d066d. Report an issue: GitHub.