huggingface/transformers · error · NotImplementedError

{} is an abstract class. Only classes inheriting this class

Error message

{} is an abstract class. Only classes inheriting this class can call `update_candidate_strategy`.

What it means

The second abstract method of CandidateGenerator: update_candidate_strategy lets generators adapt after each verification round (e.g. dynamic num_assistant_tokens). A subclass that does not override it will crash during assisted generation after the first candidate validation step.

Source

Thrown at src/transformers/generation/candidate_generator.py:74

        """
        raise NotImplementedError(
            f"{self.__class__} is an abstract class. Only classes inheriting this class can call `get_candidates`."
        )

    def update_candidate_strategy(self, input_ids: torch.LongTensor, scores: torch.FloatTensor, num_matches: int):
        """
        Updates the candidate generation strategy based on the outcomes.

        Args:
            input_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`):
                Indices of input sequence tokens in the vocabulary. [What are input IDs?](../glossary#input-ids)
            scores (`torch.FloatTensor` of shape `(batch_size, candidate_length, config.vocab_size)`):
                Prediction scores of a language modeling head. These can be logits for each vocabulary when not using
                beam search or log softmax for each vocabulary token when using beam search
            num_matches (`int`):
                The number of matches between the candidate sequences and the model predictions.
        """
        raise NotImplementedError(
            f"{self.__class__} is an abstract class. Only classes inheriting this class can call "
            "`update_candidate_strategy`."
        )


class AssistedCandidateGenerator(CandidateGenerator):
    """
    `CandidateGenerator` class to be used for assisted generation and speculative decoding. This class generates
    candidates through the use of a smaller model. Read the following blog post for more information:
    https://huggingface.co/blog/assisted-generation

    Args:
        input_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`):
            Indices of input sequence tokens in the vocabulary. [What are input IDs?](../glossary#input-ids)
        assistant_model (`PreTrainedModel`):
            The model to be used for generating candidates. This model should be smaller than the main model.
        generation_config (`~generation.GenerationConfig`, *optional*):
            The generation configuration to be used as base parametrization for the generation call.

View on GitHub (pinned to a597f97485)

Solutions

1. Implement update_candidate_strategy(input_ids, scores, num_matches) in your subclass (a no-op pass is acceptable if no adaptation is needed)
2. Mirror the signatures from AssistedCandidateGenerator to stay compatible with the current loop

Example fix

class MyGenerator(CandidateGenerator):
    def get_candidates(self, input_ids, **kwargs): ...
    # after: add the missing method
    def update_candidate_strategy(self, input_ids, scores, num_matches):
        pass

from transformers.generation.candidate_generator import CandidateGenerator

def implements_update_strategy(gen) -> bool:
    return type(gen).update_candidate_strategy is not CandidateGenerator.update_candidate_strategy

from transformers.generation.candidate_generator import CandidateGenerator

def is_complete_candidate_generator(cls) -> bool:
    return (
        isinstance(cls, type) and issubclass(cls, CandidateGenerator)
        and cls.get_candidates is not CandidateGenerator.get_candidates
        and cls.update_candidate_strategy is not CandidateGenerator.update_candidate_strategy
    )

When it happens

Trigger: Custom CandidateGenerator subclass implementing only get_candidates; the error surfaces inside model.generate's _assisted_decoding loop after the first batch of candidates is scored, when update_candidate_strategy is invoked.

Common situations: Partial implementations of custom candidate generators, or copy-pasting an old generator whose signature changed after a transformers upgrade.

Related errors

• {} is an abstract class. Only classes inheriting this class
• Invalid max_matching_ngram_size or num_output_tokens
• Expected assistant_model to be a Gemma4AssistantForCausalLM
• `model_outputs` cannot be None, and they need to contain `hi
• Could not find `num_mtp_layers` in the model config. This mo

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/497fd90bb6f5d5e4. Report an issue: GitHub.