{"record":{"id":"a12bae1d1e5c4554","repo":"hiyouga/LlamaFactory","slug":"cannot-find-sufficient-samples-consider-increasin","errorCode":null,"errorMessage":"Cannot find sufficient samples, consider increasing dataset size.","messagePattern":"Cannot find sufficient samples, consider increasing dataset size\\.","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"src/llamafactory/data/loader.py","lineNumber":269,"sourceCode":"            load_from_cache_file=(not data_args.overwrite_cache) or (training_args.local_process_index != 0),\n            desc=\"Running tokenizer on dataset\",\n        )\n\n    dataset = dataset.map(\n        dataset_processor.preprocess_dataset,\n        batched=True,\n        batch_size=data_args.preprocessing_batch_size,\n        remove_columns=column_names,\n        **kwargs,\n    )\n\n    if training_args.should_log:\n        try:\n            print(\"eval example:\" if is_eval else \"training example:\")\n            dataset_processor.print_data_example(next(iter(dataset)))\n        except StopIteration:\n            if stage == \"pt\":\n                raise RuntimeError(\"Cannot find sufficient samples, consider increasing dataset size.\")\n            else:\n                raise RuntimeError(\"Cannot find valid samples, check `data/README.md` for the data format.\")\n\n    return dataset\n\n\ndef get_dataset(\n    template: \"Template\",\n    model_args: \"ModelArguments\",\n    data_args: \"DataArguments\",\n    training_args: \"Seq2SeqTrainingArguments\",\n    stage: Literal[\"pt\", \"sft\", \"rm\", \"ppo\", \"kto\"],\n    tokenizer: \"PreTrainedTokenizer\",\n    processor: Optional[\"ProcessorMixin\"] = None,\n) -> \"DatasetModule\":\n    r\"\"\"Get the train dataset and optionally gets the evaluation dataset.\"\"\"\n    # Load tokenized dataset if path exists\n    if data_args.tokenized_path is not None:","sourceCodeStart":251,"sourceCodeEnd":287,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/src/llamafactory/data/loader.py#L251-L287","documentation":"RuntimeError raised after tokenization when iterating the preprocessed pt (pretrain) dataset yields StopIteration on the very first sample: the dataset became empty after filtering (e.g. max_samples, cutoff_len dropping everything, or token counts exceeding limits). For the pretraining stage the message points at insufficient raw sample count.","triggerScenarios":"stage: pt with max_samples set smaller than the number of samples filtered out, cutoff_len so short that every concatenated sequence is discarded, or a corpus whose documents all fail preprocessing (e.g. empty texts).","commonSituations":"Smoke-testing with max_samples: 4 while a tokenized buffer/packing step consumes them; pretraining a corpus with very short lines and a large cutoff; empty or whitespace-only documents in the text field.","solutions":["Increase max_samples or remove it so the pt dataset retains samples after preprocessing.","Check that the dataset's text column is non-empty and matches the columns mapping in dataset_info.json.","Review cutoff_len and packing settings — make sure sequences survive the length filtering.","Inspect the tokenized dataset length (or enable logging) to confirm samples survive before the print_data_example call."],"exampleFix":"# before\nmax_samples: 2\ncutoff_len: 8\n\n# after\nmax_samples: 1000\ncutoff_len: 2048","handlingStrategy":"validation","validationCode":"def pt_sample_count_ok(rows: list[dict], max_samples: int | None) -> bool:\n    usable = [r for r in rows if r.get(\"text\", \"\").strip()]\n    n = len(usable) if max_samples is None else min(max_samples, len(usable))\n    return n > 0","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Smoke-test with max_samples large enough to survive filtering (>=10).","Check the text column is non-empty and correctly mapped.","Keep cutoff_len at a realistic value (>=512) for pt runs."],"tags":["dataset","pretrain","empty-dataset","preprocessing"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}