{"record":{"id":"dff1bcc8bc69db56","repo":"huggingface/transformers","slug":"random-replace-prob-should-be-between-0-and-1","errorCode":null,"errorMessage":"random_replace_prob should be between 0 and 1.","messagePattern":"random_replace_prob should be between 0 and 1\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/transformers/data/data_collator.py","lineNumber":712,"sourceCode":"            if self.tokenizer.mask_token is None:\n                raise ValueError(\n                    \"This tokenizer does not have a mask token which is necessary for masked language modeling. \"\n                    \"You should pass `mlm=False` to train on causal language modeling instead.\"\n                )\n            if self.mlm_probability is None or self.mlm_probability < 0 or self.mlm_probability > 1:\n                raise ValueError(\"mlm_probability should be between 0 and 1.\")\n            self.mlm_probability = float(self.mlm_probability)\n        elif self.whole_word_mask:\n            raise ValueError(\n                \"Whole word masking can only be used with mlm=True.\"\n                \"If you want to use whole word masking, please set mlm=True.\"\n            )\n        if self.mask_replace_prob + self.random_replace_prob > 1:\n            raise ValueError(\"The sum of mask_replace_prob and random_replace_prob should not exceed 1\")\n        if self.mask_replace_prob < 0 or self.mask_replace_prob > 1:\n            raise ValueError(\"mask_replace_prob should be between 0 and 1.\")\n        if self.random_replace_prob < 0 or self.random_replace_prob > 1:\n            raise ValueError(\"random_replace_prob should be between 0 and 1.\")\n\n        if self.whole_word_mask:\n            if not self.tokenizer.is_fast:\n                warnings.warn(\n                    \"Whole word masking depends on offset mapping which is only natively available with fast tokenizers.\",\n                    UserWarning,\n                )\n\n            if self.mask_replace_prob < 1:\n                warnings.warn(\n                    \"Random token replacement is not supported with whole word masking. \"\n                    \"Setting mask_replace_prob to 1.\",\n                )\n                self.mask_replace_prob = 1\n                self.random_replace_prob = 0\n\n        self.mask_replace_prob = float(self.mask_replace_prob)\n        self.random_replace_prob = float(self.random_replace_prob)","sourceCodeStart":694,"sourceCodeEnd":730,"githubUrl":"https://github.com/huggingface/transformers/blob/a597f974857b3d92939971296bc0deb93d33d780/src/transformers/data/data_collator.py#L694-L730","documentation":"Raised by DataCollatorForLanguageModeling.__post_init__ when random_replace_prob is outside [0,1]. This parameter is the probability that a masked token is replaced by a random vocabulary token (the classic 10% branch of BERT masking), so it must be a valid probability; values like 10 (percent instead of 0.1) or negatives are rejected.","triggerScenarios":"Constructing DataCollatorForLanguageModeling(tokenizer, random_replace_prob=10) or any value < 0 or > 1.","commonSituations":"Transcribing '10% random token' from the BERT paper as 10 rather than 0.1; config-file scale mismatches; copy-paste between scripts that use different units.","solutions":["Pass a fraction in [0,1], e.g. random_replace_prob=0.1.","Fix the config/CLI source of the value if it was scaled incorrectly.","Verify the sum rule at the same time: mask_replace_prob + random_replace_prob must not exceed 1."],"exampleFix":"# before\ncollator = DataCollatorForLanguageModeling(tokenizer=tok, random_replace_prob=10)\n\n# after\ncollator = DataCollatorForLanguageModeling(tokenizer=tok, random_replace_prob=0.1)","handlingStrategy":"validation","validationCode":"random_replace_prob = float(cfg['random_replace_prob'])\nif not 0.0 <= random_replace_prob <= 1.0:\n    raise ValueError(f'random_replace_prob={random_replace_prob} outside [0,1]')","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Validate all three probabilities (mlm_probability, mask_replace_prob, random_replace_prob) together at config load.","Keep hyperparameter names/scales identical between config files and constructor arguments."],"tags":["data-collator","configuration","validation"],"backgroundTag":null,"analyzedSha":"a597f974857b3d92939971296bc0deb93d33d780","analyzedAt":"2026-08-14T18:24:08.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}