{"record":{"id":"a7caedd04e1edeca","repo":"hiyouga/LlamaFactory","slug":"cannot-find-satisfying-example-considering-decrea","errorCode":null,"errorMessage":"Cannot find satisfying example, considering decrease `export_quantization_maxlen`.","messagePattern":"Cannot find satisfying example, considering decrease `export_quantization_maxlen`\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/llamafactory/model/model_utils/quantization.py","lineNumber":70,"sourceCode":"    else:\n        data_path = model_args.export_quantization_dataset\n        data_files = None\n\n    dataset = load_dataset(\n        path=data_path,\n        data_files=data_files,\n        split=\"train\",\n        cache_dir=model_args.cache_dir,\n        token=model_args.hf_hub_token,\n    )\n\n    samples = []\n    maxlen = model_args.export_quantization_maxlen\n    for _ in range(model_args.export_quantization_nsamples):\n        n_try = 0\n        while True:\n            if n_try > 100:\n                raise ValueError(\"Cannot find satisfying example, considering decrease `export_quantization_maxlen`.\")\n\n            sample_idx = random.randint(0, len(dataset) - 1)\n            sample: dict[str, torch.Tensor] = tokenizer(dataset[sample_idx][\"text\"], return_tensors=\"pt\")\n            n_try += 1\n            if sample[\"input_ids\"].size(1) > maxlen:\n                break  # TODO: fix large maxlen\n\n        word_idx = random.randint(0, sample[\"input_ids\"].size(1) - maxlen - 1)\n        input_ids = sample[\"input_ids\"][:, word_idx : word_idx + maxlen]\n        attention_mask = sample[\"attention_mask\"][:, word_idx : word_idx + maxlen]\n        samples.append({\"input_ids\": input_ids.tolist(), \"attention_mask\": attention_mask.tolist()})\n\n    return samples\n\n\ndef configure_quantization(\n    config: \"PretrainedConfig\",\n    tokenizer: \"PreTrainedTokenizer\",","sourceCodeStart":52,"sourceCodeEnd":88,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/src/llamafactory/model/model_utils/quantization.py#L52-L88","documentation":"When exporting a GPTQ-quantized model, LlamaFactory calibrates with real samples: it repeatedly draws random dataset rows until it finds one longer than export_quantization_maxlen. After 100 failed draws for a single sample it raises ValueError suggesting a smaller export_quantization_maxlen. It is a calibration-data length problem, not a model problem.","triggerScenarios":"llamafactory-cli export with export_quantization_bit set and export_quantization_dataset configured, where random samples keep having input_ids.size(1) <= export_quantization_maxlen for >100 tries — e.g. maxlen=4096 against a dataset of short documents.","commonSituations":"Default export_quantization_maxlen (2048/4096) used with a short-text dataset (e.g. alpaca-style samples of a few hundred tokens); small calibration datasets where the few long docs are unlucky to draw; tokenizers that compress text far below the expected length.","solutions":["Decrease export_quantization_maxlen in the export YAML to below the dataset's typical token length (inspect p99 length first).","Switch export_quantization_dataset to a long-form corpus (e.g. a code or book dataset) that reliably exceeds maxlen.","Increase export_quantization_nsamples only after maxlen is realistic — it does not help if no sample is long enough.","Pre-check token lengths: tokenize the dataset offline and choose maxlen ~ p90 of lengths."],"exampleFix":"# before (export yaml)\nexport_quantization_maxlen: 8192   # dataset max ~1k tokens\n\n# after\nexport_quantization_maxlen: 1024","handlingStrategy":"validation","validationCode":"lens = [len(tokenizer(x[\"text\"]).input_ids) for x in dataset][:2000]\np95 = sorted(lens)[int(0.95 * len(lens))]\nassert export_quantization_maxlen < p95, (\n    f\"export_quantization_maxlen {export_quantization_maxlen} >= dataset p95 {p95}; lower it\"\n)","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Measure dataset token-length distribution before choosing export_quantization_maxlen.","Default to a long-form calibration corpus for high maxlen exports."],"tags":["gptq","export","quantization","calibration","dataset"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}