{"record":{"id":"c522d8c9215f581c","repo":"hiyouga/LlamaFactory","slug":"interleave-probs-is-only-valid-for-interleaved-m","errorCode":null,"errorMessage":"`interleave_probs` is only valid for interleaved mixing.","messagePattern":"`interleave_probs` is only valid for interleaved mixing\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/llamafactory/hparams/data_args.py","lineNumber":167,"sourceCode":"            if isinstance(arg, str):\n                return [item.strip() for item in arg.split(\",\")]\n            return arg\n\n        self.dataset = split_arg(self.dataset)\n        self.eval_dataset = split_arg(self.eval_dataset)\n\n        if self.media_dir is None:\n            self.media_dir = self.dataset_dir\n\n        if self.dataset is None and self.val_size > 1e-6:\n            raise ValueError(\"Cannot specify `val_size` if `dataset` is None.\")\n\n        if self.eval_dataset is not None and self.val_size > 1e-6:\n            raise ValueError(\"Cannot specify `val_size` if `eval_dataset` is not None.\")\n\n        if self.interleave_probs is not None:\n            if self.mix_strategy == \"concat\":\n                raise ValueError(\"`interleave_probs` is only valid for interleaved mixing.\")\n\n            self.interleave_probs = list(map(float, split_arg(self.interleave_probs)))\n            if self.dataset is not None and len(self.dataset) != len(self.interleave_probs):\n                raise ValueError(\"The length of dataset and interleave probs should be identical.\")\n\n            if self.eval_dataset is not None and len(self.eval_dataset) != len(self.interleave_probs):\n                raise ValueError(\"The length of eval dataset and interleave probs should be identical.\")\n\n        if self.streaming and self.val_size > 1e-6 and self.val_size < 1:\n            raise ValueError(\"Streaming mode should have an integer val size.\")\n\n        if self.streaming and self.max_samples is not None:\n            raise ValueError(\"`max_samples` is incompatible with `streaming`.\")\n\n        if self.mask_history and self.train_on_prompt:\n            raise ValueError(\"`mask_history` is incompatible with `train_on_prompt`.\")\n\n        if self.neat_packing:","sourceCodeStart":149,"sourceCodeEnd":185,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/src/llamafactory/hparams/data_args.py#L149-L185","documentation":"interleave_probs assigns sampling probabilities per dataset and only makes sense when datasets are mixed with an interleaved strategy (HF datasets interleave_datasets). With mix_strategy: concat the datasets are simply concatenated, so probabilities are meaningless and DataArguments.post_init raises ValueError.","triggerScenarios":"YAML containing both interleave_probs: 0.5,0.5 and mix_strategy: concat (or omitting mix_strategy, since concat is not the interleaving path — the check fires whenever mix_strategy == 'concat').","commonSituations":"Copy-pasting interleave settings from an interleaved example into a concat-style config; assuming interleave_probs works as weights for concat mixing.","solutions":["Set mix_strategy: interleave (or interleave_under, interleave_over) in the same config as interleave_probs.","Or remove interleave_probs if you want plain concatenation with equal treatment.","Double-check dataset_info.json and the YAML both agree on the mixing strategy."],"exampleFix":"# before\nmix_strategy: concat\ninterleave_probs: 0.7,0.3\n\n# after\nmix_strategy: interleave\ninterleave_probs: 0.7,0.3","handlingStrategy":"validation","validationCode":"if data_args.interleave_probs is not None:\n    assert data_args.mix_strategy != \"concat\", \"set mix_strategy to an interleave variant or drop interleave_probs\"","typeGuard":null,"tryCatchPattern":null,"preventionTips":["interleave_probs implies interleaved mixing; set mix_strategy accordingly in the same edit.","Keep mixing settings grouped in one YAML block to avoid partial edits."],"tags":["config","data-args","mixing"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}