{"record":{"id":"a65e8c4bf560c23e","repo":"huggingface/transformers","slug":"mask-replace-prob-should-be-between-0-and-1","errorCode":null,"errorMessage":"mask_replace_prob should be between 0 and 1.","messagePattern":"mask_replace_prob should be between 0 and 1\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/transformers/data/data_collator.py","lineNumber":710,"sourceCode":"    def __post_init__(self):\n        if self.mlm:\n            if self.tokenizer.mask_token is None:\n                raise ValueError(\n                    \"This tokenizer does not have a mask token which is necessary for masked language modeling. \"\n                    \"You should pass `mlm=False` to train on causal language modeling instead.\"\n                )\n            if self.mlm_probability is None or self.mlm_probability < 0 or self.mlm_probability > 1:\n                raise ValueError(\"mlm_probability should be between 0 and 1.\")\n            self.mlm_probability = float(self.mlm_probability)\n        elif self.whole_word_mask:\n            raise ValueError(\n                \"Whole word masking can only be used with mlm=True.\"\n                \"If you want to use whole word masking, please set mlm=True.\"\n            )\n        if self.mask_replace_prob + self.random_replace_prob > 1:\n            raise ValueError(\"The sum of mask_replace_prob and random_replace_prob should not exceed 1\")\n        if self.mask_replace_prob < 0 or self.mask_replace_prob > 1:\n            raise ValueError(\"mask_replace_prob should be between 0 and 1.\")\n        if self.random_replace_prob < 0 or self.random_replace_prob > 1:\n            raise ValueError(\"random_replace_prob should be between 0 and 1.\")\n\n        if self.whole_word_mask:\n            if not self.tokenizer.is_fast:\n                warnings.warn(\n                    \"Whole word masking depends on offset mapping which is only natively available with fast tokenizers.\",\n                    UserWarning,\n                )\n\n            if self.mask_replace_prob < 1:\n                warnings.warn(\n                    \"Random token replacement is not supported with whole word masking. \"\n                    \"Setting mask_replace_prob to 1.\",\n                )\n                self.mask_replace_prob = 1\n                self.random_replace_prob = 0\n","sourceCodeStart":692,"sourceCodeEnd":728,"githubUrl":"https://github.com/huggingface/transformers/blob/a597f974857b3d92939971296bc0deb93d33d780/src/transformers/data/data_collator.py#L692-L728","documentation":"Raised by DataCollatorForLanguageModeling.__post_init__ when mask_replace_prob is outside [0,1]. This parameter is the probability that a masked token is replaced by the tokenizer's mask token, so like any probability it must lie between 0 and 1; anything else indicates a misconfigured ratio (typically percentages passed instead of fractions).","triggerScenarios":"Constructing DataCollatorForLanguageModeling(tokenizer, mask_replace_prob=80) or any negative value; also triggered by NaN values introduced through config parsing.","commonSituations":"Porting a paper's '80% mask replacement' setting as 80 instead of 0.8; loading the value from a YAML/JSON config where the wrong field or scale was used.","solutions":["Pass a fraction in [0,1], e.g. mask_replace_prob=0.8 for the standard BERT behavior.","Fix the upstream config file / CLI value if the wrong number flows into the constructor.","Sanity-check all probability arguments (mlm_probability, mask_replace_prob, random_replace_prob) before building the collator."],"exampleFix":"# before\ncollator = DataCollatorForLanguageModeling(tokenizer=tok, mask_replace_prob=80)\n\n# after\ncollator = DataCollatorForLanguageModeling(tokenizer=tok, mask_replace_prob=0.8)","handlingStrategy":"validation","validationCode":"mask_replace_prob = float(cfg['mask_replace_prob'])\nif not 0.0 <= mask_replace_prob <= 1.0:\n    raise ValueError(f'mask_replace_prob={mask_replace_prob} outside [0,1] (percent vs fraction bug?)')","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Add JSON-schema / argparse validation with minimum=0, maximum=1 for all probability fields.","Auto-detect the classic percent mistake: if value > 1, divide by 100 or fail loudly at config load."],"tags":["data-collator","configuration","validation"],"backgroundTag":null,"analyzedSha":"a597f974857b3d92939971296bc0deb93d33d780","analyzedAt":"2026-08-14T18:24:08.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}