{"record":{"id":"a609b109245c0a7e","repo":"hiyouga/LlamaFactory","slug":"cannot-specify-val-size-if-eval-dataset-is-not","errorCode":null,"errorMessage":"Cannot specify `val_size` if `eval_dataset` is not None.","messagePattern":"Cannot specify `val_size` if `eval_dataset` is not None\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/llamafactory/data/data_utils.py","lineNumber":100,"sourceCode":"        raise ValueError(f\"Unknown mixing strategy: {data_args.mix_strategy}.\")\n\n\ndef split_dataset(\n    dataset: Optional[Union[\"Dataset\", \"IterableDataset\"]],\n    eval_dataset: Optional[Union[\"Dataset\", \"IterableDataset\", dict[str, \"Dataset\"]]],\n    data_args: \"DataArguments\",\n    seed: int,\n) -> tuple[dict, dict]:\n    r\"\"\"Split the dataset and returns two dicts containing train set and validation set.\n\n    Support both map dataset and iterable dataset.\n\n    Returns:\n        train_dict: Dictionary containing training data with key \"train\"\n        eval_dict: Dictionary containing evaluation data with keys \"validation\" or \"validation_{name}\"\n    \"\"\"\n    if eval_dataset is not None and data_args.val_size > 1e-6:\n        raise ValueError(\"Cannot specify `val_size` if `eval_dataset` is not None.\")\n\n    # the train and eval better to in dict dtype and separately return for cpode clearly and good handle outside\n    train_dict, eval_dict = {}, {}\n\n    if dataset is not None:\n        if data_args.streaming:\n            dataset = dataset.shuffle(buffer_size=data_args.buffer_size, seed=seed)\n\n        if data_args.val_size > 1e-6:\n            if data_args.streaming:\n                eval_dict[\"validation\"] = dataset.take(int(data_args.val_size))\n                train_dict[\"train\"] = dataset.skip(int(data_args.val_size))\n            else:\n                val_size = int(data_args.val_size) if data_args.val_size > 1 else data_args.val_size\n                split_result = dataset.train_test_split(test_size=val_size, seed=seed)\n                train_dict[\"train\"] = split_result[\"train\"]\n                eval_dict[\"validation\"] = split_result[\"test\"]\n        else:","sourceCodeStart":82,"sourceCodeEnd":118,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/src/llamafactory/data/data_utils.py#L82-L118","documentation":"split_dataset raises ValueError when an explicit eval_dataset is provided AND data_args.val_size > 1e-6. The two are mutually exclusive: val_size means 'carve this fraction/count out of the train set', which is meaningless when eval data is already supplied.","triggerScenarios":"A training config/run that passes eval_dataset (e.g. a separate eval dataset from dataset_info) together with val_size > 0 in the YAML — common when users leave val_size set after switching from splitting to an explicit eval set.","commonSituations":"Reusing a base YAML that had val_size: 0.1 and then adding a dedicated eval dataset; tutorial configs carried forward while the data setup changed.","solutions":["Remove val_size (or set val_size: 0) from the training YAML if you supply an eval dataset.","Conversely, drop the eval dataset if you want val_size-based splitting.","Never set both — the parser will not fix this for you."],"exampleFix":"# before (yaml)\ndataset: train_data\neval_dataset: eval_data\nval_size: 0.1\n\n# after (yaml)\ndataset: train_data\neval_dataset: eval_data","handlingStrategy":"validation","validationCode":"if eval_dataset is not None:\n    assert data_args.val_size <= 1e-6, \"set val_size: 0 when eval_dataset is given\"","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Treat val_size and eval_dataset as either/or in config templates.","Remove stale keys when evolving shared YAML bases."],"tags":["dataset","config","validation","training"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}