{"record":{"id":"fb1df5d5bb6eb433","repo":"huggingface/transformers","slug":"streamer-cannot-be-used-with-beam-search-yet","errorCode":null,"errorMessage":"`streamer` cannot be used with beam search (yet!). Make sure that `num_beams` is set to 1.","messagePattern":"`streamer` cannot be used with beam search \\(yet!\\)\\. Make sure that `num_beams` is set to 1\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/transformers/generation/utils.py","lineNumber":1568,"sourceCode":"        transition_scores = stacked_scores.gather(0, indices)\n\n        # 9. Mask out transition_scores of beams that stopped early\n        transition_scores[beam_indices_mask] = 0\n\n        return transition_scores\n\n    def _validate_generation_mode(\n        self: \"GenerativePreTrainedModel\", generation_mode, generation_config, generation_mode_kwargs\n    ):\n        supported_modes = getattr(self, \"_supported_generation_modes\", None)\n        if supported_modes is not None and generation_mode not in supported_modes:\n            raise ValueError(\n                f\"{self.__class__.__name__} only supports {supported_modes}, but got \"\n                f\"generation mode '{generation_mode}'.\"\n            )\n\n        if generation_mode == GenerationMode.BEAM_SEARCH and \"streamer\" in generation_mode_kwargs:\n            raise ValueError(\n                \"`streamer` cannot be used with beam search (yet!). Make sure that `num_beams` is set to 1.\"\n            )\n\n        if generation_mode == GenerationMode.ASSISTED_GENERATION:\n            if generation_config.num_return_sequences > 1:\n                raise ValueError(\n                    \"num_return_sequences has to be 1 when doing assisted generate, \"\n                    f\"but is {generation_config.num_return_sequences}.\"\n                )\n            if self._is_stateful:\n                # In assisted generation we need the ability to confirm whether the model would pick certain tokens,\n                # which is not possible with stateful models (they can't reset to a previous subset of generated text)\n                raise ValueError(\n                    f\"assisted generation is not supported with stateful models, such as {self.__class__.__name__}\"\n                )\n\n        if (\n            assistant_model := generation_mode_kwargs.get(\"assistant_model\")","sourceCodeStart":1550,"sourceCodeEnd":1586,"githubUrl":"https://github.com/huggingface/transformers/blob/a597f974857b3d92939971296bc0deb93d33d780/src/transformers/generation/utils.py#L1550-L1586","documentation":"Text streaming (`streamer=`) is only implemented for single-path decoding. Beam search maintains multiple candidate sequences per step, which the streamer API cannot represent, so `generate` rejects the combination of `GenerationMode.BEAM_SEARCH` with a `streamer` kwarg.","triggerScenarios":"`model.generate(**inputs, streamer=TextStreamer(tokenizer), num_beams=4)` — any parameterization that resolves to beam search mode (e.g. `num_beams>1` without sampling overrides that keep mode greedy/sample) together with a streamer.","commonSituations":"Adding a `TextIteratorStreamer` for a chat UI onto an existing beam-search config; notebooks that streamed with greedy decoding and later enabled beams for quality; `num_beams` inherited from the model's `generation_config.json` while the code adds a streamer.","solutions":["Set `num_beams=1` in the generate call (or remove beam parameters) so decoding stays greedy/sampling while streaming.","Keep beam search but drop `streamer` and print the final output instead.","If beams came from the model's saved `generation_config.json`, override at call time: `model.generate(..., num_beams=1, streamer=streamer)` or set `model.generation_config.num_beams = 1`.","For streaming chat UX, use sampling (`do_sample=True`) which streams token-by-token."],"exampleFix":"# before\nstreamer = TextIteratorStreamer(tokenizer)\nout = model.generate(**inputs, num_beams=5, streamer=streamer)  # ValueError\n\n# after\nout = model.generate(**inputs, num_beams=1, do_sample=True, streamer=streamer)","handlingStrategy":"validation","validationCode":"if streamer is not None and kwargs.get(\"num_beams\", getattr(model.generation_config, \"num_beams\", 1)) > 1:\n    kwargs[\"num_beams\"] = 1","typeGuard":null,"tryCatchPattern":null,"preventionTips":["In UI/serving code that streams, force num_beams=1 at the call site.","Don't inherit num_beams from saved generation_config.json when adding a streamer.","Gate on streamer presence in shared wrappers and sanitize beam parameters."],"tags":["generation","streamer","beam-search","incompatible-arguments"],"backgroundTag":null,"analyzedSha":"a597f974857b3d92939971296bc0deb93d33d780","analyzedAt":"2026-08-14T18:24:08.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}