huggingface/transformers · error · ValueError

mlm_probability should be between 0 and 1.

Error message

mlm_probability should be between 0 and 1.

What it means

Raised in DataCollatorForLanguageModeling.__post_init__ (data_collator.py:700). When mlm=True, mlm_probability must be a number in [0, 1] (it is cast to float after validation); None or out-of-range values (e.g. 15 meaning 15 percent, or a negative value) are rejected at construction. It is only checked in the mlm branch — with mlm=False an invalid value is ignored.

Source

Thrown at src/transformers/data/data_collator.py:700

    tokenizer: PreTrainedTokenizerBase
    mlm: bool = True
    whole_word_mask: bool = False
    mlm_probability: float | None = 0.15
    mask_replace_prob: float = 0.8
    random_replace_prob: float = 0.1
    pad_to_multiple_of: int | None = None
    return_tensors: str = "pt"
    seed: int | None = None

    def __post_init__(self):
        if self.mlm:
            if self.tokenizer.mask_token is None:
                raise ValueError(
                    "This tokenizer does not have a mask token which is necessary for masked language modeling. "
                    "You should pass `mlm=False` to train on causal language modeling instead."
                )
            if self.mlm_probability is None or self.mlm_probability < 0 or self.mlm_probability > 1:
                raise ValueError("mlm_probability should be between 0 and 1.")
            self.mlm_probability = float(self.mlm_probability)
        elif self.whole_word_mask:
            raise ValueError(
                "Whole word masking can only be used with mlm=True."
                "If you want to use whole word masking, please set mlm=True."
            )
        if self.mask_replace_prob + self.random_replace_prob > 1:
            raise ValueError("The sum of mask_replace_prob and random_replace_prob should not exceed 1")
        if self.mask_replace_prob < 0 or self.mask_replace_prob > 1:
            raise ValueError("mask_replace_prob should be between 0 and 1.")
        if self.random_replace_prob < 0 or self.random_replace_prob > 1:
            raise ValueError("random_replace_prob should be between 0 and 1.")

        if self.whole_word_mask:
            if not self.tokenizer.is_fast:
                warnings.warn(
                    "Whole word masking depends on offset mapping which is only natively available with fast tokenizers.",
                    UserWarning,

View on GitHub (pinned to a597f97485)

Solutions

1. Use a fraction in [0, 1]: mlm_probability=0.15 for 15%.
2. If you wanted the default, simply omit the argument instead of passing None.
3. Validate values sourced from CLI/config with a range check before constructing the collator.

Example fix

# before
collator = DataCollatorForLanguageModeling(tokenizer, mlm=True, mlm_probability=15)  # raises

# after
collator = DataCollatorForLanguageModeling(tokenizer, mlm=True, mlm_probability=0.15)

if mlm:
    assert mlm_probability is not None and 0 <= mlm_probability <= 1, (
        f'mlm_probability must be in [0,1], got {mlm_probability!r} (use 0.15 for 15%)'
    )

def is_valid_probability(p) -> bool:
    return p is not None and 0 <= p <= 1

When it happens

Trigger: DataCollatorForLanguageModeling(tokenizer, mlm=True, mlm_probability=15) or mlm_probability=None or a negative value; raised immediately in __post_init__. Note the default is 0.15, so this only fires when the user overrides it.

Common situations: Users writing probabilities as percentages (15 instead of 0.15), or passing None intending 'default' — the field default is 0.15 and None is explicitly rejected; often introduced when the value comes from a config/CLI that failed to parse.

Related errors

• This tokenizer does not have a mask token which is necessary
• The sum of mask_replace_prob and random_replace_prob should
• mask_replace_prob should be between 0 and 1.
• random_replace_prob should be between 0 and 1.
• PUSH_TO_HUB_TOKEN is not set, cannot push results to the Hub

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/2f52282e73f441b3. Report an issue: GitHub.