{"record":{"id":"4d957ecee462881e","repo":"huggingface/transformers","slug":"model-outputs-cannot-be-none-and-they-need-to-c-4d957e","errorCode":null,"errorMessage":"`model_outputs` cannot be None, and they need to contain `hiden_states`","messagePattern":"`model_outputs` cannot be None, and they need to contain `hiden_states`","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/transformers/generation/candidate_generator.py","lineNumber":1472,"sourceCode":"\n    def get_candidates(\n        self,\n        input_ids: torch.LongTensor,\n        model_kwargs: dict[str, Any],\n        model_outputs: ModelOutput,\n        is_first_iteration: bool,\n        n_last_matches: int,\n        **kwargs,\n    ) -> tuple[torch.LongTensor, torch.FloatTensor | None]:\n        # This is a trick to skip the first loop of the main model's `_assisted_decoding` method. Since we need the\n        # main model's outputs here to get the candidates, we skip the first loop to allow the main model to get the outputs\n        # (this is because usually `get_candidates` is called first in the main `_assisted_decoding` loop)\n        if is_first_iteration:\n            return input_ids, None\n\n        # Make sure we correctly collected all the main model outputs we needed\n        if model_outputs is None or not hasattr(model_outputs, \"hidden_states\"):\n            raise ValueError(\"`model_outputs` cannot be None, and they need to contain `hiden_states`\")\n\n        last_hidden_states: torch.Tensor = model_outputs.hidden_states[-1]\n\n        # Here `input_ids`/`attention_mask`/`position_ids` are the full sequence inputs, including the last token that was\n        # just drafted from the main model. We need to slice to get only what the main model just processed, shifted by 1 to the\n        # right to take the new token as well. Say the main model just had token positions [2, 3] as input, the tensors\n        # contains the data for position [0, 1, 2, 3, 4], i.e. full inputs + new drafted token from last position 3 that was processed.\n        # We want to slice to get data for positions [3, 4] for the 1st mtp layer, i.e. same as main model, shifted by 1 to the right.\n        # On the other hand, the `full_seq_last_hidden_states` contains data for only already processed positions by the main_model, i.e.\n        # one less than `input_ids`/`positions_ids`/`attention_mask`\n        num_last_main_model_tokens = n_last_matches + 1 if not self.is_main_model_prefill else input_ids.shape[1] - 1\n\n        # The hidden states have seq_len equal to the last main model's forward pass on all the candidates. We need the\n        # last hidden states of only the last validated tokens\n        last_hidden_states = last_hidden_states[:, :num_last_main_model_tokens].to(self.device)\n\n        # We need to cache the full last_hidden_states from the main model to be able to correct the mtp cache based on validated tokens\n        if self.num_mtp_layers > 1:","sourceCodeStart":1454,"sourceCodeEnd":1490,"githubUrl":"https://github.com/huggingface/transformers/blob/a597f974857b3d92939971296bc0deb93d33d780/src/transformers/generation/candidate_generator.py#L1454-L1490","documentation":"The MTP candidate generator needs the main model's last hidden state to feed the first MTP layer. On non-first iterations it requires model_outputs to be non-None and expose hidden_states; otherwise drafting cannot proceed. (Note the message's 'hiden_states' is a typo for hidden_states in the source.)","triggerScenarios":"Manually calling this generator's get_candidates with model_outputs=None or outputs produced without output_hidden_states=True; or a custom generate loop that skips collecting hidden states before invoking the MTP generator.","commonSituations":"Custom speculative-decoding loops, integrations that reuse outputs objects from a different code path (e.g. ModelOutput without hidden_states), or disabling hidden state output for performance.","solutions":["Request hidden states from the main model: output_hidden_states=True on every forward feeding the generator","Use the built-in model.generate flow with MTP enabled rather than calling get_candidates by hand","Ensure get_candidates is not called with None outputs after the first iteration (check your loop order)"],"exampleFix":"# before\noutputs = main_model(input_ids, past_key_values=cache)\n# after\noutputs = main_model(input_ids, past_key_values=cache, output_hidden_states=True)","handlingStrategy":"validation","validationCode":"def outputs_have_hidden_states(model_outputs) -> bool:\n    return model_outputs is not None and getattr(model_outputs, \"hidden_states\", None) is not None","typeGuard":"def has_hidden_states(outputs) -> bool:\n    return outputs is not None and getattr(outputs, \"hidden_states\", None) is not None","tryCatchPattern":null,"preventionTips":["Set output_hidden_states=True on every main-model forward in MTP decoding loops","Cache ModelOutput objects per iteration rather than reusing stale ones"],"tags":["python","transformers","generation","mtp","hidden-states"],"backgroundTag":null,"analyzedSha":"a597f974857b3d92939971296bc0deb93d33d780","analyzedAt":"2026-08-14T18:24:08.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}