{"record":{"id":"5d985cd1f40a0af5","repo":"huggingface/transformers","slug":"model-outputs-cannot-be-none-and-they-need-to-co","errorCode":null,"errorMessage":"`model_outputs` cannot be None and they need to contain `hidden_states`!","messagePattern":"`model_outputs` cannot be None and they need to contain `hidden_states`!","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/transformers/generation/candidate_generator.py","lineNumber":1623,"sourceCode":"        is_first_iteration: bool,\n        n_last_matches: int,\n        **kwargs,\n    ) -> tuple[torch.LongTensor, torch.FloatTensor | None]:\n        \"\"\"Generate draft token candidates using the drafter.\"\"\"\n        # This is a trick to skip the first loop of the main model's `_assisted_decoding` method. Since we need the\n        # main model's outputs here to get the candidates, we skip the first loop to allow the main model to get the outputs\n        # (this is because usually `get_candidates` is called first in the main `_assisted_decoding` loop)\n        if is_first_iteration:\n            return input_ids, None\n\n        # Early exit if we cannot generate new tokens.\n        max_new_tokens = min(int(self.block_size), self.main_model_max_length - input_ids.shape[1] - 1)\n        if max_new_tokens <= 0:\n            return input_ids, None\n\n        # Make sure we correctly collected all the main model outputs we needed\n        if model_outputs is None or not hasattr(model_outputs, \"hidden_states\"):\n            raise ValueError(\"`model_outputs` cannot be None and they need to contain `hidden_states`!\")\n\n        num_last_main_model_tokens = n_last_matches + 1 if not self.is_main_model_prefill else input_ids.shape[1] - 1\n        # The hidden states hold all tokens from the last main model's forward on all the candidates. We need the\n        # hidden states of only accepted tokens thus crop out the rest\n        context_hidden_states: torch.Tensor = torch.cat(\n            [model_outputs.hidden_states[i + 1][:, :num_last_main_model_tokens] for i in self.target_layer_ids], dim=-1\n        )\n\n        # We need to tell the cache how many new states to expect into its k/v states, additional to the \"noise\" or \"diffusion window\"\n        self.cache.set_previous_accepted_tokens(num_last_main_model_tokens)\n        # We need to remvoe the previous \"noise\" from the cache\n        if not self.is_main_model_prefill:\n            self.cache.crop(-self.block_size)\n\n        # Here `position_ids`/`attention_mask` are the full sequence inputs, including the last \"bonus\" token that was just drafted\n        # from the main model. We need to slice to get only what the main model just processed. Say the main model just had token\n        # positions [2, 3] as input, the tensors contains the data for position [0, 1, 2, 3, 4], i.e. full inputs + new drafted token\n        # from last position 3 that was processed","sourceCodeStart":1605,"sourceCodeEnd":1641,"githubUrl":"https://github.com/huggingface/transformers/blob/a597f974857b3d92939971296bc0deb93d33d780/src/transformers/generation/candidate_generator.py#L1605-L1641","documentation":"A block/diffusion-style candidate generator (used in blockwise parallel/diffusion LLM decoding) concatenates hidden states from specific target layers (target_layer_ids) to build candidates. It requires non-None model_outputs exposing hidden_states; otherwise it cannot assemble the cross-layer context.","triggerScenarios":"Calling this generator after a main-model forward that did not set output_hidden_states=True, or passing model_outputs=None on a non-first iteration of a custom assisted-decoding loop.","commonSituations":"Diffusion-LLM or blockwise generation integrations where hidden state collection was skipped; custom loops reusing stale/empty ModelOutput objects.","solutions":["Always call the main model with output_hidden_states=True before this generator runs","Verify the model actually returns per-layer hidden states (output_hidden_states supported and not stripped)","Rely on model.generate's built-in path for this generator instead of manual invocation"],"exampleFix":"# before\noutputs = main_model(input_ids, use_cache=True)\n# after\noutputs = main_model(input_ids, use_cache=True, output_hidden_states=True)","handlingStrategy":"validation","validationCode":"def outputs_have_hidden_states(model_outputs) -> bool:\n    return model_outputs is not None and getattr(model_outputs, \"hidden_states\", None) is not None","typeGuard":"def has_hidden_states(outputs) -> bool:\n    hs = getattr(outputs, \"hidden_states\", None)\n    return hs is not None and len(hs) > 0","tryCatchPattern":null,"preventionTips":["Request output_hidden_states=True whenever the block/diffusion candidate generator is active","Assert the number of hidden-state layers covers target_layer_ids before generating"],"tags":["python","transformers","generation","diffusion","hidden-states"],"backgroundTag":null,"analyzedSha":"a597f974857b3d92939971296bc0deb93d33d780","analyzedAt":"2026-08-14T18:24:08.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}