{"record":{"id":"2c875ef377f273ea","repo":"huggingface/transformers","slug":"model-must-have-config-device-and-dtype-at","errorCode":null,"errorMessage":"Model must have 'config', 'device', and 'dtype' attributes.","messagePattern":"Model must have 'config', 'device', and 'dtype' attributes\\.","errorType":"validation","errorClass":"AttributeError","httpStatus":null,"severity":"error","filePath":"src/transformers/generation/continuous_batching/continuous_api.py","lineNumber":1114,"sourceCode":"    def init_continuous_batching(\n        self,\n        generation_config: GenerationConfig | None = None,\n        continuous_batching_config: ContinuousBatchingConfig | None = None,\n        workload_hints: WorkloadHints | None = None,\n    ) -> ContinuousBatchingManager:\n        \"\"\"Initialize a manager for continuous batching inference.\n\n        Args:\n            generation_config: An optional generation configuration, which may contain a CompileConfig object\n            continuous_batching_config: An optional continuous batching configuration\n            workload_hints: Optional WorkloadHints to help the continuous batching manager make better decisions for\n                default values\n        Returns:\n            `ContinuousBatchingManager`: The manager instance to add requests and retrieve results.\n        \"\"\"\n        # Mandatory attributes\n        if not hasattr(self, \"config\") or not hasattr(self, \"device\") or not hasattr(self, \"dtype\"):\n            raise AttributeError(\"Model must have 'config', 'device', and 'dtype' attributes.\")\n\n        # If a persistent manager is found we return it\n        cached_manager = getattr(self, \"_cached_continuous_batching_manager\", None)\n        if isinstance(cached_manager, ContinuousBatchingManager):\n            logger.info(\n                \"Cached continuous batching manager found: it will be re-used instead of creating a new one. If you\"\n                \" want to create a new manager, you should call `destroy_cached_continuous_batching_manager` first.\"\n            )\n            cached_manager.switch_to_cb_friendly_attn(self)  # might have switched in .stop\n            return cached_manager\n\n        # Retrieve generation config\n        gen_config = generation_config if generation_config is not None else self.generation_config\n        if gen_config is None:\n            raise ValueError(\"A GenerationConfig must be provided or set in the model.\")\n        # Warn about EOS\n        if gen_config.eos_token_id is None:\n            logger.warning(\"`eos_token_id` not set in GenerationConfig. Setting to -1 (disabled).\")","sourceCodeStart":1096,"sourceCodeEnd":1132,"githubUrl":"https://github.com/huggingface/transformers/blob/a597f974857b3d92939971296bc0deb93d33d780/src/transformers/generation/continuous_batching/continuous_api.py#L1096-L1132","documentation":"Raised at the top of continuous-batching manager initialization: the object it is attached to must expose config, device, and dtype attributes. It is an AttributeError guard ensuring the API is called on a model-like object, not a bare nn.Module, a function output, or a misplaced call.","triggerScenarios":"Calling model.continuous_batching(...) on a wrapper/module lacking PreTrainedModel attributes; calling the free function continuous_batching(obj) with obj = some submodule (e.g. model.decoder) or a Pipeline/optimizer by accident.","commonSituations":"Passing a torch.compile'd or wrapped model; using a custom inference wrapper class that forwards generate() but not config/device/dtype; typo passing the tokenizer as the model.","solutions":["Call the API on the full PreTrainedModel instance, not a submodule or wrapper","If using a wrapper, expose config/device/dtype properties delegating to the inner model","Check the call site order of arguments — a swapped model/tokenizer argument triggers this"],"exampleFix":"# before\nmanager = my_custom_wrapper.continuous_batching()  # wrapper lacks attrs\n\n# after\nclass MyWrapper:\n    @property\n    def config(self): return self.model.config\n    @property\n    def device(self): return self.model.device\n    @property\n    def dtype(self): return self.model.dtype\nmanager = my_custom_wrapper.continuous_batching()","handlingStrategy":"validation","validationCode":"for attr in ('config', 'device', 'dtype'):\n    assert hasattr(model, attr), f'model lacks .{attr} — pass the full PreTrainedModel'","typeGuard":"from transformers import PreTrainedModel\ndef is_cb_capable(obj) -> bool:\n    return isinstance(obj, PreTrainedModel) or all(hasattr(obj, a) for a in ('config', 'device', 'dtype'))","tryCatchPattern":null,"preventionTips":["Call continuous_batching on the top-level model object","Wrappers must forward config/device/dtype","Double-check argument order at call sites"],"tags":["api-misuse","attributes","continuous-batching","model"],"backgroundTag":null,"analyzedSha":"a597f974857b3d92939971296bc0deb93d33d780","analyzedAt":"2026-08-14T18:24:08.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}