{"record":{"id":"cb7926451615a70f","repo":"huggingface/transformers","slug":"assistant-ensemble-weight-must-be-in-the-open-in","errorCode":null,"errorMessage":"`assistant_ensemble_weight` must be in the open interval `(0.0, 1.0)`, but is {}. Use `None` for standard (lossless) speculative decoding.","messagePattern":"`assistant_ensemble_weight` must be in the open interval `\\(0\\.0, 1\\.0\\)`, but is (.+?)\\. Use `None` for standard \\(lossless\\) speculative decoding\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/transformers/generation/configuration_utils.py","lineNumber":672,"sourceCode":"\n        Args:\n            strict (bool): If True, raise an exception for any issues found. If False, only log issues.\n            user_set_attributes (set[str], *optional*): Names of attributes the caller explicitly provided. When\n                supplied, \"minor issue\" warnings about conflicting flag combinations (e.g. sampling-only flags set\n                while `do_sample=False`) only fire if the conflicting flag is in this set -- avoiding noisy warnings\n                when the value was inherited from a model's default `generation_config.json`. When `None`, all set\n                attributes are considered user-set (backward-compatible behavior for direct `validate()` calls).\n        \"\"\"\n        minor_issues = {}  # format: {attribute_name: issue_description}\n\n        # 1. Validation of individual attributes\n        # 1.1. Decoding attributes\n        if self.early_stopping not in {None, True, False, \"never\"}:\n            raise ValueError(f\"`early_stopping` must be a boolean or 'never', but is {self.early_stopping}.\")\n        if self.max_new_tokens is not None and self.max_new_tokens <= 0:\n            raise ValueError(f\"`max_new_tokens` must be greater than 0, but is {self.max_new_tokens}.\")\n        if self.assistant_ensemble_weight is not None and not (0.0 < self.assistant_ensemble_weight < 1.0):\n            raise ValueError(\n                f\"`assistant_ensemble_weight` must be in the open interval `(0.0, 1.0)`, \"\n                f\"but is {self.assistant_ensemble_weight}. Use `None` for standard (lossless) speculative decoding.\"\n            )\n        if self.pad_token_id is not None and self.pad_token_id < 0:\n            minor_issues[\"pad_token_id\"] = (\n                f\"`pad_token_id` should be positive but got {self.pad_token_id}. This will cause errors when batch \"\n                \"generating, if there is padding. Please set `pad_token_id` explicitly as \"\n                \"`model.generation_config.pad_token_id=PAD_TOKEN_ID` to avoid errors in generation\"\n            )\n        # 1.2. Cache attributes\n        # \"paged\" re-routes to continuous batching and so it is a valid cache implementation. But we do not want to test\n        # it with the `generate` as the other would be, so we we cannot add it to ALL_CACHE_IMPLEMENTATIONS\n        valid_cache_implementations = ALL_CACHE_IMPLEMENTATIONS + (\"paged\",)\n        if self.cache_implementation is not None and self.cache_implementation not in valid_cache_implementations:\n            raise ValueError(\n                f\"Invalid `cache_implementation` ({self.cache_implementation}). Choose one of: \"\n                f\"{valid_cache_implementations}\"\n            )","sourceCodeStart":654,"sourceCodeEnd":690,"githubUrl":"https://github.com/huggingface/transformers/blob/a597f974857b3d92939971296bc0deb93d33d780/src/transformers/generation/configuration_utils.py#L654-L690","documentation":"assistant_ensemble_weight blends target and assistant distributions in lossy speculative decoding (self-speculative ensemble). validate() requires it strictly inside the open interval (0.0, 1.0) or None; 0, 1, or values outside would degenerate or invert the ensemble and are rejected.","triggerScenarios":"GenerationConfig(assistant_ensemble_weight=0.0 or 1.0 or 1.5), or reading a generation_config.json where the weight was written as 0/1; validation fires when generate() runs with the config.","commonSituations":"Users probing boundary values (0 = 'off', 1 = 'full assistant') without realizing both are invalid; configs edited to disable the feature instead of removing the key.","solutions":["Set assistant_ensemble_weight=None to use standard lossless speculative decoding","Otherwise pick a value strictly between 0 and 1, e.g. 0.5","Remove the key from generation_config.json rather than writing 0/1 to disable it"],"exampleFix":"# before\ncfg = GenerationConfig(assistant_ensemble_weight=1.0)\n# after\ncfg = GenerationConfig(assistant_ensemble_weight=None)","handlingStrategy":"validation","validationCode":"def valid_assistant_ensemble_weight(v) -> bool:\n    return v is None or (isinstance(v, (int, float)) and 0.0 < v < 1.0)","typeGuard":"def is_valid_ensemble_weight(v) -> bool:\n    return v is None or (isinstance(v, float) and 0.0 < v < 1.0)","tryCatchPattern":null,"preventionTips":["Use None to disable lossy ensemble decoding rather than boundary values","Validate externally sourced generation configs before passing them to generate()"],"tags":["python","transformers","generation","speculative-decoding","generation-config"],"backgroundTag":null,"analyzedSha":"a597f974857b3d92939971296bc0deb93d33d780","analyzedAt":"2026-08-14T18:24:08.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}