{"record":{"id":"19ff4c9568c38879","repo":"huggingface/transformers","slug":"assisted-generate-requires-use-cache-true","errorCode":null,"errorMessage":"assisted generate requires `use_cache=True`","messagePattern":"assisted generate requires `use_cache=True`","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/transformers/generation/utils.py","lineNumber":3621,"sourceCode":"                The model used to assist the generation process. If not provided, the main model will be used.\n            assistant_tokenizer (`PreTrainedTokenizerBase`, *optional*):\n                The tokenizer used for the assistant model. If not provided, the token space is assumed to be the same.\n            tokenizer (`PreTrainedTokenizerBase`, *optional*):\n                The tokenizer used for the main model. If not provided, the token space is assumed to be the same.\n            model_kwargs:\n                Additional model specific keyword arguments will be forwarded to the `forward` function of the model.\n                If model is an encoder-decoder model the kwargs should include `encoder_outputs`.\n\n        Return:\n            [`~generation.GenerateDecoderOnlyOutput`], [`~generation.GenerateEncoderDecoderOutput`] or\n            `torch.LongTensor`: A `torch.LongTensor` containing the generated tokens (default behaviour) or a\n            [`~generation.GenerateDecoderOnlyOutput`] if `model.config.is_encoder_decoder=False` and\n            `return_dict_in_generate=True` or a [`~generation.GenerateEncoderDecoderOutput`] if\n            `model.config.is_encoder_decoder=True`.\n        \"\"\"\n        # The cache must be dynamic for assisted generation, and the check must happen AFTER preparing cache\n        if not model_kwargs[\"use_cache\"]:\n            raise ValueError(\"assisted generate requires `use_cache=True`\")\n        if (\n            generation_config.cache_implementation in [\"static\", \"hybrid\", \"sliding_window\"]\n            or type(model_kwargs.get(\"past_key_values\")) is StaticCache\n        ):\n            raise ValueError(\"assisted generate is not supported with Static cache classes`\")\n\n        # Make sure we can record past on the cache\n        cache = model_kwargs.get(\"past_key_values\")\n        if cache is None:\n            raise RuntimeError(\"assisted decoding requires a cache\")\n        cache.activate_past_recording()\n\n        # Get the candidate generator, given the parameterization\n        candidate_generator = self._get_candidate_generator(\n            generation_config=generation_config,\n            input_ids=input_ids,\n            inputs_tensor=inputs_tensor,\n            assistant_model=assistant_model,","sourceCodeStart":3603,"sourceCodeEnd":3639,"githubUrl":"https://github.com/huggingface/transformers/blob/a597f974857b3d92939971296bc0deb93d33d780/src/transformers/generation/utils.py#L3603-L3639","documentation":"Error \"assisted generate requires `use_cache=True`\" thrown in huggingface/transformers.","triggerScenarios":"Raised in assisted generation when use_cache=False in the generation config.","commonSituations":"Speculative decoding with an assistant model while the model or generation config disables the KV cache.","solutions":["Set `use_cache=True` in the generation config for assisted generation."],"exampleFix":null,"handlingStrategy":null,"validationCode":null,"typeGuard":null,"tryCatchPattern":null,"preventionTips":[],"tags":[],"backgroundTag":null,"analyzedSha":"a597f974857b3d92939971296bc0deb93d33d780","analyzedAt":"2026-08-14T18:24:08.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}