{"record":{"id":"853d239f06956b79","repo":"huggingface/transformers","slug":"a-generationconfig-must-be-provided-or-set-in-the","errorCode":null,"errorMessage":"A GenerationConfig must be provided or set in the model.","messagePattern":"A GenerationConfig must be provided or set in the model\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/transformers/generation/continuous_batching/continuous_api.py","lineNumber":1129,"sourceCode":"        \"\"\"\n        # Mandatory attributes\n        if not hasattr(self, \"config\") or not hasattr(self, \"device\") or not hasattr(self, \"dtype\"):\n            raise AttributeError(\"Model must have 'config', 'device', and 'dtype' attributes.\")\n\n        # If a persistent manager is found we return it\n        cached_manager = getattr(self, \"_cached_continuous_batching_manager\", None)\n        if isinstance(cached_manager, ContinuousBatchingManager):\n            logger.info(\n                \"Cached continuous batching manager found: it will be re-used instead of creating a new one. If you\"\n                \" want to create a new manager, you should call `destroy_cached_continuous_batching_manager` first.\"\n            )\n            cached_manager.switch_to_cb_friendly_attn(self)  # might have switched in .stop\n            return cached_manager\n\n        # Retrieve generation config\n        gen_config = generation_config if generation_config is not None else self.generation_config\n        if gen_config is None:\n            raise ValueError(\"A GenerationConfig must be provided or set in the model.\")\n        # Warn about EOS\n        if gen_config.eos_token_id is None:\n            logger.warning(\"`eos_token_id` not set in GenerationConfig. Setting to -1 (disabled).\")\n            gen_config.eos_token_id = -1\n\n        # Retrieve continuous batching config, or create it if none is provided\n        if continuous_batching_config is None:\n            if isinstance(getattr(gen_config, \"continuous_batching_config\", None), ContinuousBatchingConfig):\n                logger.warning(\n                    \"Passing ContinuousBatchingConfig through GenerationConfig is deprecated. Please pass it separately\"\n                    \" using the continuous_batching_config kwarg.\"\n                )\n                continuous_batching_config = gen_config.continuous_batching_config\n            else:\n                continuous_batching_config = ContinuousBatchingConfig()\n\n        # Create and return the manager\n        return ContinuousBatchingManager(","sourceCodeStart":1111,"sourceCodeEnd":1147,"githubUrl":"https://github.com/huggingface/transformers/blob/a597f974857b3d92939971296bc0deb93d33d780/src/transformers/generation/continuous_batching/continuous_api.py#L1111-L1147","documentation":"Raised during continuous-batching manager init when no generation_config argument was passed and the model has generation_config = None. The manager needs sampling/EOS settings from a GenerationConfig and refuses to guess defaults beyond that.","triggerScenarios":"model.generation_config is None (some freshly assembled from_config/torchscript models, or users who explicitly set it to None) and the caller omits generation_config in the continuous_batching call.","commonSituations":"Loading sharded/converted checkpoints that skip generation_config creation; manually constructing a model then deleting generation_config; library code that passes generation_config conditionally.","solutions":["Pass an explicit GenerationConfig: model.continuous_batching(generation_config=GenerationConfig(...))","Set model.generation_config = GenerationConfig.from_pretrained(checkpoint) before the call","For minimal use, GenerationConfig(eos_token_id=tokenizer.eos_token_id) satisfies the requirement"],"exampleFix":"# before\nmanager = model.continuous_batching()  # model.generation_config is None\n\n# after\nfrom transformers import GenerationConfig\nmanager = model.continuous_batching(generation_config=GenerationConfig(eos_token_id=tokenizer.eos_token_id))","handlingStrategy":"validation","validationCode":"gen_cfg = generation_config or getattr(model, 'generation_config', None)\nif gen_cfg is None:\n    from transformers import GenerationConfig\n    gen_cfg = GenerationConfig(eos_token_id=tokenizer.eos_token_id)\n    model.generation_config = gen_cfg","typeGuard":"from transformers import GenerationConfig\ndef has_generation_config(model) -> bool:\n    return getattr(model, 'generation_config', None) is not None or isinstance(getattr(model, 'generation_config', None), GenerationConfig)","tryCatchPattern":null,"preventionTips":["Always pass generation_config explicitly in library code","Restore generation_config after from_config on unusual checkpoints"],"tags":["generation-config","validation","continuous-batching"],"backgroundTag":null,"analyzedSha":"a597f974857b3d92939971296bc0deb93d33d780","analyzedAt":"2026-08-14T18:24:08.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}