{"record":{"id":"a8bc9b6d1e6769ca","repo":"huggingface/transformers","slug":"this-collator-requires-that-sequence-lengths-be-ev","errorCode":null,"errorMessage":"This collator requires that sequence lengths be even to create a leakage-free perm_mask. Please see relevant comments in source code for details.","messagePattern":"This collator requires that sequence lengths be even to create a leakage-free perm_mask\\. Please see relevant comments in source code for details\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/transformers/data/data_collator.py","lineNumber":1188,"sourceCode":"            0. Start from the beginning of the sequence by setting `cur_len = 0` (number of tokens processed so far).\n            1. Sample a `span_length` from the interval `[1, max_span_length]` (length of span of tokens to be masked)\n            2. Reserve a context of length `context_length = span_length / plm_probability` to surround span to be\n               masked\n            3. Sample a starting point `start_index` from the interval `[cur_len, cur_len + context_length -\n               span_length]` and mask tokens `start_index:start_index + span_length`\n            4. Set `cur_len = cur_len + context_length`. If `cur_len < max_len` (i.e. there are tokens remaining in the\n               sequence to be processed), repeat from Step 1.\n        \"\"\"\n        import torch\n\n        if self.tokenizer.mask_token is None:\n            raise ValueError(\n                \"This tokenizer does not have a mask token which is necessary for permutation language modeling.\"\n                \" Please add a mask token if you want to use this tokenizer.\"\n            )\n\n        if inputs.size(1) % 2 != 0:\n            raise ValueError(\n                \"This collator requires that sequence lengths be even to create a leakage-free perm_mask. Please see\"\n                \" relevant comments in source code for details.\"\n            )\n\n        labels = inputs.clone()\n        # Creating the mask and target_mapping tensors\n        masked_indices = torch.full(labels.shape, 0, dtype=torch.bool)\n        target_mapping = torch.zeros((labels.size(0), labels.size(1), labels.size(1)), dtype=torch.float32)\n\n        for i in range(labels.size(0)):\n            # Start from the beginning of the sequence by setting `cur_len = 0` (number of tokens processed so far).\n            cur_len = 0\n            max_len = labels.size(1)\n\n            while cur_len < max_len:\n                # Sample a `span_length` from the interval `[1, max_span_length]` (length of span of tokens to be masked)\n                span_length = torch.randint(1, self.max_span_length + 1, (1,)).item()\n                # Reserve a context of length `context_length = span_length / plm_probability` to surround the span to be masked","sourceCodeStart":1170,"sourceCodeEnd":1206,"githubUrl":"https://github.com/huggingface/transformers/blob/a597f974857b3d92939971296bc0deb93d33d780/src/transformers/data/data_collator.py#L1170-L1206","documentation":"Raised by DataCollatorForPermutationLanguageModeling.torch_call when inputs.size(1) (sequence length) is odd. The collator builds a leakage-free perm_mask/target_mapping pair for the two-stream XLNet attention; the construction as implemented requires an even number of positions per sequence, so odd-length batches are rejected rather than producing a subtly wrong mask.","triggerScenarios":"Calling the collator on a batch whose padded sequence length is odd, e.g. max length 15 after padding, or a tokenizer with no padding that yields odd-length samples.","commonSituations":"Tokenizing without pad_to_max_length and letting dynamic padding produce an odd max length; truncating to max_length=255; batching a single odd-length sample.","solutions":["Set tokenizer.padding_side='right' and pad to an even length, e.g. DataCollator... with padding='max_length', max_length=128, or pad_to_multiple_of=2 on the tokenizer.","Use a tokenizer with a pad token (tokenizer.pad_token) so the collator pads the batch to a common even length.","Truncate/pad inputs to an even max_length before collating."],"exampleFix":"# before\ntok = AutoTokenizer.from_pretrained('xlnet-base-cased')  # no padding configured\ncollator = DataCollatorForPermutationLanguageModeling(tokenizer=tok)\nbatch = collator(samples)  # odd seq len -> raises\n\n# after\nfrom transformers import DataCollatorForPermutationLanguageModeling\ncollator = DataCollatorForPermutationLanguageModeling(tok)\nencoded = tok(texts, padding='max_length', max_length=128, truncation=True)\nbatch = collator([dict(e) for e in encoded])","handlingStrategy":"validation","validationCode":"def pad_to_even(tokenizer, texts, max_length):\n    max_length += max_length % 2\n    return tokenizer(texts, padding='max_length', max_length=max_length, truncation=True)\n\nenc = pad_to_even(tokenizer, texts, max_length=127)  # becomes 128","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Always tokenize with padding='max_length' (or pad_to_multiple_of=2) for even sequence lengths.","Choose even max_length values in preprocessing configs (128, 256, 384...)."],"tags":["data-collator","padding","sequence-length","permutation-language-modeling"],"backgroundTag":null,"analyzedSha":"a597f974857b3d92939971296bc0deb93d33d780","analyzedAt":"2026-08-14T18:24:08.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}