hiyouga/LlamaFactory · error · ValueError

Cannot find satisfying example, considering decrease `export

Error message

Cannot find satisfying example, considering decrease `export_quantization_maxlen`.

What it means

When exporting a GPTQ-quantized model, LlamaFactory calibrates with real samples: it repeatedly draws random dataset rows until it finds one longer than export_quantization_maxlen. After 100 failed draws for a single sample it raises ValueError suggesting a smaller export_quantization_maxlen. It is a calibration-data length problem, not a model problem.

Source

Thrown at src/llamafactory/model/model_utils/quantization.py:70

    else:
        data_path = model_args.export_quantization_dataset
        data_files = None

    dataset = load_dataset(
        path=data_path,
        data_files=data_files,
        split="train",
        cache_dir=model_args.cache_dir,
        token=model_args.hf_hub_token,
    )

    samples = []
    maxlen = model_args.export_quantization_maxlen
    for _ in range(model_args.export_quantization_nsamples):
        n_try = 0
        while True:
            if n_try > 100:
                raise ValueError("Cannot find satisfying example, considering decrease `export_quantization_maxlen`.")

            sample_idx = random.randint(0, len(dataset) - 1)
            sample: dict[str, torch.Tensor] = tokenizer(dataset[sample_idx]["text"], return_tensors="pt")
            n_try += 1
            if sample["input_ids"].size(1) > maxlen:
                break  # TODO: fix large maxlen

        word_idx = random.randint(0, sample["input_ids"].size(1) - maxlen - 1)
        input_ids = sample["input_ids"][:, word_idx : word_idx + maxlen]
        attention_mask = sample["attention_mask"][:, word_idx : word_idx + maxlen]
        samples.append({"input_ids": input_ids.tolist(), "attention_mask": attention_mask.tolist()})

    return samples


def configure_quantization(
    config: "PretrainedConfig",
    tokenizer: "PreTrainedTokenizer",

View on GitHub (pinned to f28afaf635)

Solutions

1. Decrease export_quantization_maxlen in the export YAML to below the dataset's typical token length (inspect p99 length first).
2. Switch export_quantization_dataset to a long-form corpus (e.g. a code or book dataset) that reliably exceeds maxlen.
3. Increase export_quantization_nsamples only after maxlen is realistic — it does not help if no sample is long enough.
4. Pre-check token lengths: tokenize the dataset offline and choose maxlen ~ p90 of lengths.

Example fix

# before (export yaml)
export_quantization_maxlen: 8192   # dataset max ~1k tokens

# after
export_quantization_maxlen: 1024

lens = [len(tokenizer(x["text"]).input_ids) for x in dataset][:2000]
p95 = sorted(lens)[int(0.95 * len(lens))]
assert export_quantization_maxlen < p95, (
    f"export_quantization_maxlen {export_quantization_maxlen} >= dataset p95 {p95}; lower it"
)

When it happens

Trigger: llamafactory-cli export with export_quantization_bit set and export_quantization_dataset configured, where random samples keep having input_ids.size(1) <= export_quantization_maxlen for >100 tries — e.g. maxlen=4096 against a dataset of short documents.

Common situations: Default export_quantization_maxlen (2048/4096) used with a short-text dataset (e.g. alpaca-style samples of a few hundred tokens); small calibration datasets where the few long docs are unlucky to draw; tokenizers that compress text far below the expected length.

Related errors

• AutoGPTQ only accepts 2/3/4/8-bit quantization.
• Quantization dataset is necessary for exporting.
• vLLM engine does not support bnb quantization (GPTQ and AWQ
• DoRA is not compatible with PTQ-quantized models.
• DeepSpeed ZeRO-3 or FSDP is incompatible with PTQ-quantized

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/a7caedd04e1edeca. Report an issue: GitHub.