{"record":{"id":"2f52282e73f441b3","repo":"huggingface/transformers","slug":"mlm-probability-should-be-between-0-and-1","errorCode":null,"errorMessage":"mlm_probability should be between 0 and 1.","messagePattern":"mlm_probability should be between 0 and 1\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/transformers/data/data_collator.py","lineNumber":700,"sourceCode":"    tokenizer: PreTrainedTokenizerBase\n    mlm: bool = True\n    whole_word_mask: bool = False\n    mlm_probability: float | None = 0.15\n    mask_replace_prob: float = 0.8\n    random_replace_prob: float = 0.1\n    pad_to_multiple_of: int | None = None\n    return_tensors: str = \"pt\"\n    seed: int | None = None\n\n    def __post_init__(self):\n        if self.mlm:\n            if self.tokenizer.mask_token is None:\n                raise ValueError(\n                    \"This tokenizer does not have a mask token which is necessary for masked language modeling. \"\n                    \"You should pass `mlm=False` to train on causal language modeling instead.\"\n                )\n            if self.mlm_probability is None or self.mlm_probability < 0 or self.mlm_probability > 1:\n                raise ValueError(\"mlm_probability should be between 0 and 1.\")\n            self.mlm_probability = float(self.mlm_probability)\n        elif self.whole_word_mask:\n            raise ValueError(\n                \"Whole word masking can only be used with mlm=True.\"\n                \"If you want to use whole word masking, please set mlm=True.\"\n            )\n        if self.mask_replace_prob + self.random_replace_prob > 1:\n            raise ValueError(\"The sum of mask_replace_prob and random_replace_prob should not exceed 1\")\n        if self.mask_replace_prob < 0 or self.mask_replace_prob > 1:\n            raise ValueError(\"mask_replace_prob should be between 0 and 1.\")\n        if self.random_replace_prob < 0 or self.random_replace_prob > 1:\n            raise ValueError(\"random_replace_prob should be between 0 and 1.\")\n\n        if self.whole_word_mask:\n            if not self.tokenizer.is_fast:\n                warnings.warn(\n                    \"Whole word masking depends on offset mapping which is only natively available with fast tokenizers.\",\n                    UserWarning,","sourceCodeStart":682,"sourceCodeEnd":718,"githubUrl":"https://github.com/huggingface/transformers/blob/a597f974857b3d92939971296bc0deb93d33d780/src/transformers/data/data_collator.py#L682-L718","documentation":"Raised in DataCollatorForLanguageModeling.__post_init__ (data_collator.py:700). When mlm=True, mlm_probability must be a number in [0, 1] (it is cast to float after validation); None or out-of-range values (e.g. 15 meaning 15 percent, or a negative value) are rejected at construction. It is only checked in the mlm branch — with mlm=False an invalid value is ignored.","triggerScenarios":"DataCollatorForLanguageModeling(tokenizer, mlm=True, mlm_probability=15) or mlm_probability=None or a negative value; raised immediately in __post_init__. Note the default is 0.15, so this only fires when the user overrides it.","commonSituations":"Users writing probabilities as percentages (15 instead of 0.15), or passing None intending 'default' — the field default is 0.15 and None is explicitly rejected; often introduced when the value comes from a config/CLI that failed to parse.","solutions":["Use a fraction in [0, 1]: mlm_probability=0.15 for 15%.","If you wanted the default, simply omit the argument instead of passing None.","Validate values sourced from CLI/config with a range check before constructing the collator."],"exampleFix":"# before\ncollator = DataCollatorForLanguageModeling(tokenizer, mlm=True, mlm_probability=15)  # raises\n\n# after\ncollator = DataCollatorForLanguageModeling(tokenizer, mlm=True, mlm_probability=0.15)","handlingStrategy":"validation","validationCode":"if mlm:\n    assert mlm_probability is not None and 0 <= mlm_probability <= 1, (\n        f'mlm_probability must be in [0,1], got {mlm_probability!r} (use 0.15 for 15%)'\n    )","typeGuard":"def is_valid_probability(p) -> bool:\n    return p is not None and 0 <= p <= 1","tryCatchPattern":null,"preventionTips":["Express probabilities as fractions (0.15), never percentages (15).","Omit mlm_probability to use the 0.15 default instead of passing None.","Range-check CLI/config-sourced hyperparameters before constructing trainers."],"tags":["data-collator","mlm","validation","probability","training"],"backgroundTag":null,"analyzedSha":"a597f974857b3d92939971296bc0deb93d33d780","analyzedAt":"2026-08-14T18:24:08.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}