{"record":{"id":"6cb42b15d2496ffd","repo":"huggingface/transformers","slug":"the-main-and-assistant-models-have-different-token","errorCode":null,"errorMessage":"The main and assistant models have different tokenizers. Please provide `tokenizer` and `assistant_tokenizer` to `generate()` {doc_reference}.","messagePattern":"The main and assistant models have different tokenizers\\. Please provide `tokenizer` and `assistant_tokenizer` to `generate\\(\\)` (.+?)\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/transformers/generation/utils.py","lineNumber":1610,"sourceCode":"                    getattr(self.config, attr) == getattr(assistant_model.config, attr) for attr in attributes_to_check\n                )\n                if not are_equal:\n                    raise ValueError(\n                        \"The main model and the assistant don't have compatible encoder-dependent input shapes. \"\n                        \"Ensure you load the assistant with the correct encoder-decoder class, e.g. `AutoModelForSpeechSeq2Seq` for Whisper.\"\n                    )\n\n            doc_reference = (\n                \"(see https://huggingface.co/docs/transformers/en/generation_strategies#universal-assisted-decoding)\"\n            )\n            if self.config.get_text_config().vocab_size == assistant_model.config.get_text_config().vocab_size:\n                if \"assistant_tokenizer\" in generation_mode_kwargs:\n                    raise ValueError(\n                        f\"`assistant_tokenizer` is not required when the main and assistant models use the same tokenizer. Please omit `assistant_tokenizer` from `generate()` {doc_reference}.\"\n                    )\n            else:\n                if \"tokenizer\" not in generation_mode_kwargs or \"assistant_tokenizer\" not in generation_mode_kwargs:\n                    raise ValueError(\n                        f\"The main and assistant models have different tokenizers. Please provide `tokenizer` and `assistant_tokenizer` to `generate()` {doc_reference}.\"\n                    )\n\n    def _validate_model_kwargs(self: \"GenerativePreTrainedModel\", model_kwargs: dict[str, Any]):\n        \"\"\"Validates model kwargs for generation. Generate argument typos will also be caught here.\"\"\"\n        # Excludes arguments that are handled before calling any model function\n        if self.config.is_encoder_decoder:\n            for key in [\"decoder_input_ids\"]:\n                model_kwargs.pop(key, None)\n\n        unused_model_args = []\n        model_args = set(inspect.signature(self.prepare_inputs_for_generation).parameters)\n        # `kwargs`/`model_kwargs` is often used to handle optional forward pass inputs like `attention_mask`. If\n        # `prepare_inputs_for_generation` doesn't accept them, then a stricter check can be made ;)\n        if \"kwargs\" in model_args or \"model_kwargs\" in model_args:\n            model_args |= set(inspect.signature(self.forward).parameters)\n\n        # Encoder-Decoder models may also need Encoder arguments from `model_kwargs`","sourceCodeStart":1592,"sourceCodeEnd":1628,"githubUrl":"https://github.com/huggingface/transformers/blob/a597f974857b3d92939971296bc0deb93d33d780/src/transformers/generation/utils.py#L1592-L1628","documentation":"The mirror case of the same-tokenizer check: when the main and assistant text configs have DIFFERENT `vocab_size`, they necessarily use different tokenizers, and universal assisted decoding needs both to map tokens between the two spaces. `generate` raises when `tokenizer` and/or `assistant_tokenizer` is missing from the call.","triggerScenarios":"`model.generate(**inputs, assistant_model=assistant)` where the two `vocab_size`s differ and one or both of `tokenizer`/`assistant_tokenizer` kwargs are absent — e.g. passing only `input_ids` prepared with the main tokenizer.","commonSituations":"Pairing models from different families (e.g. a main model and an assistant with a different tokenizer/vocab); following the assisted-decoding quickstart (same-tokenizer) and swapping in a cross-family assistant without adding both tokenizers; tokenizing inputs manually and forgetting kwargs get forwarded.","solutions":["Pass both: `model.generate(**tokenizer(prompt, return_tensors=\"pt\"), assistant_model=assistant, tokenizer=tokenizer, assistant_tokenizer=assistant_tokenizer)`.","Ensure `assistant_tokenizer` is the tokenizer the assistant checkpoint was trained with, not another copy of the main one.","If you expected same-tokenizer behavior, verify which assistant checkpoint you loaded (its vocab_size differs from the main model)."],"exampleFix":"# before\nout = model.generate(**inputs, assistant_model=assistant)  # different vocab_size, no tokenizers -> ValueError\n\n# after\nout = model.generate(\n    **inputs,\n    assistant_model=assistant,\n    tokenizer=tokenizer,\n    assistant_tokenizer=assistant_tokenizer,\n)","handlingStrategy":"validation","validationCode":"if model.config.get_text_config().vocab_size != assistant.config.get_text_config().vocab_size:\n    for key in (\"tokenizer\", \"assistant_tokenizer\"):\n        if key not in kwargs:\n            raise ValueError(f\"{key} required: main and assistant use different tokenizers\")","typeGuard":null,"tryCatchPattern":null,"preventionTips":["When pairing cross-family models, always pass both tokenizer and assistant_tokenizer.","Wrap assisted generation in a helper that checks vocab sizes and injects the right tokenizers.","Use each checkpoint's own tokenizer (from_pretrained per model), never one tokenizer for both."],"tags":["generation","assisted-decoding","tokenizer","universal-assisted-decoding"],"backgroundTag":null,"analyzedSha":"a597f974857b3d92939971296bc0deb93d33d780","analyzedAt":"2026-08-14T18:24:08.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}