{"record":{"id":"0e42e58036ff6332","repo":"huggingface/transformers","slug":"the-model-vocabulary-size-is-vocabulary-size-bu","errorCode":null,"errorMessage":"The model vocabulary size is {vocabulary_size}, but the following tokens were being biased: {invalid_biases}","messagePattern":"The model vocabulary size is (.+?), but the following tokens were being biased: (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/transformers/generation/logits_process.py","lineNumber":1331,"sourceCode":"                torch.tensor(sequence_bias, device=input_ids.device),\n                torch.tensor(0.0, device=input_ids.device),\n            )\n\n        # 5 - apply the bias to the scores\n        scores_processed = scores + bias\n        return scores_processed\n\n    def _prepare_bias_variables(self, scores: torch.FloatTensor):\n        vocabulary_size = scores.shape[-1]\n\n        # Check biased tokens out of bounds\n        invalid_biases = []\n        for sequence_ids in self.sequence_bias:\n            for token_id in sequence_ids:\n                if token_id >= vocabulary_size:\n                    invalid_biases.append(token_id)\n        if len(invalid_biases) > 0:\n            raise ValueError(\n                f\"The model vocabulary size is {vocabulary_size}, but the following tokens were being biased: \"\n                f\"{invalid_biases}\"\n            )\n\n        # Precompute the bias tensors to be applied. Sequences of length 1 are kept separately, as they can be applied\n        # with simpler logic.\n        self.length_1_bias = torch.zeros((vocabulary_size,), dtype=torch.float, device=scores.device)\n        # Extract single-token sequences and their biases\n        single_token_ids = []\n        single_token_biases = []\n        for sequence_ids, bias in self.sequence_bias.items():\n            if len(sequence_ids) == 1:\n                single_token_ids.append(sequence_ids[0])\n                single_token_biases.append(bias)\n\n        if single_token_ids:  # Only if we have any single-token sequences\n            self.length_1_bias[single_token_ids] = torch.tensor(single_token_biases, device=scores.device)\n        self.prepared_bias_variables = True","sourceCodeStart":1313,"sourceCodeEnd":1349,"githubUrl":"https://github.com/huggingface/transformers/blob/a597f974857b3d92939971296bc0deb93d33d780/src/transformers/generation/logits_process.py#L1313-L1349","documentation":"Thrown by SequenceBiasLogitsProcessor._prepare_bias_variables on the first __call__ when sequence_bias contains token ids >= the model's vocabulary size (scores.shape[-1]). Biased token ids index directly into the logits tensor, so out-of-range ids would corrupt memory or fail silently; the processor checks them lazily against the actual runtime vocabulary size.","triggerScenarios":"Passing sequence_bias={(123456,): 5.0} to a model whose logits width is smaller; model.generate(sequence_bias=...) built with token ids from a different tokenizer; using raw ids computed against a larger vocab model then switching models.","commonSituations":"Swapping tokenizer/model versions where the vocab shrank or ids shifted; hard-coded token ids copied from another project; tokenizing a phrase with one tokenizer and biasing generation of a model with another; multi-token sequences whose ids are valid but a stale id in the tuple is not.","solutions":["Verify every biased id with the same tokenizer used for generation: tokenizer.convert_tokens_to_ids(token) and confirm it is not the unk id / out of range","Check ids against model.config.vocab_size (or len(tokenizer)) before building sequence_bias","Recompute the bias dict whenever you change the model or tokenizer","Drop or remap the offending ids listed in the error message"],"exampleFix":"# before\nsequence_bias = {(123456,): 5.0}  # id beyond vocab -> ValueError at first step\nout = model.generate(**inputs, sequence_bias=sequence_bias)\n\n# after\nvocab_size = model.config.vocab_size\ntarget_id = tokenizer.convert_tokens_to_ids('Paris')\nsequence_bias = {(target_id,): 5.0} if 0 <= target_id < vocab_size else {}\nout = model.generate(**inputs, sequence_bias=sequence_bias)","handlingStrategy":"validation","validationCode":"vocab_size = model.config.vocab_size\nvalid_bias = {\n    ids: b for ids, b in sequence_bias.items()\n    if all(0 <= tid < vocab_size for tid in ids)\n}\n# inspect dropped ids: set(sequence_bias) - set(valid_bias)","typeGuard":"def is_valid_sequence_bias(sequence_bias, vocab_size) -> bool:\n    return all(\n        isinstance(ids, tuple) and all(0 <= tid < vocab_size for tid in ids)\n        for ids in sequence_bias\n    )","tryCatchPattern":"try:\n    out = model.generate(**inputs, sequence_bias=sequence_bias)\nexcept ValueError as e:\n    if 'vocabulary size' in str(e):\n        # recompute ids with the current tokenizer and retry once\n        raise\n    raise","preventionTips":["Always derive biased ids via tokenizer.convert_tokens_to_ids with the model's own tokenizer","Check ids against model.config.vocab_size before generate()","Rebuild the bias dict whenever model or tokenizer version changes","The check runs lazily at the first generation step, so validate eagerly to fail fast"],"tags":["generation","sequence-bias","vocabulary","token-ids"],"backgroundTag":null,"analyzedSha":"a597f974857b3d92939971296bc0deb93d33d780","analyzedAt":"2026-08-14T18:24:08.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}