huggingface/transformers · error · NotImplementedError

{} is an abstract class. Only classes inheriting this class

Error message

{} is an abstract class. Only classes inheriting this class can call `get_candidates`.

What it means

CandidateGenerator is an abstract base class for assisted-generation candidate sources; get_candidates is intentionally unimplemented. Instantiating the base class or a subclass that forgot to override get_candidates raises NotImplementedError when generation asks for candidates.

Source

Thrown at src/transformers/generation/candidate_generator.py:57

class CandidateGenerator:
    """Abstract base class for all candidate generators that can be applied during assisted generation."""

    requires_model_outputs: bool = False

    def get_candidates(self, input_ids: torch.LongTensor, **kwargs) -> tuple[torch.LongTensor, torch.FloatTensor]:
        """
        Fetches the candidates to be tried for the current input.

        Args:
            input_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`):
                Indices of input sequence tokens in the vocabulary. [What are input IDs?](../glossary#input-ids)

        Return:
            `torch.LongTensor` of shape `(batch_size, candidate_length)` containing the candidate sequences to be
            assessed by the model and, optionally, a `torch.FloatTensor` of shape `(batch_size, candidate_length,
            vocabulary_size)` containing the logits associated to each candidate.
        """
        raise NotImplementedError(
            f"{self.__class__} is an abstract class. Only classes inheriting this class can call `get_candidates`."
        )

    def update_candidate_strategy(self, input_ids: torch.LongTensor, scores: torch.FloatTensor, num_matches: int):
        """
        Updates the candidate generation strategy based on the outcomes.

        Args:
            input_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`):
                Indices of input sequence tokens in the vocabulary. [What are input IDs?](../glossary#input-ids)
            scores (`torch.FloatTensor` of shape `(batch_size, candidate_length, config.vocab_size)`):
                Prediction scores of a language modeling head. These can be logits for each vocabulary when not using
                beam search or log softmax for each vocabulary token when using beam search
            num_matches (`int`):
                The number of matches between the candidate sequences and the model predictions.
        """
        raise NotImplementedError(
            f"{self.__class__} is an abstract class. Only classes inheriting this class can call "

View on GitHub (pinned to a597f97485)

Solutions

1. Implement get_candidates(input_ids, **kwargs) in your CandidateGenerator subclass
2. Use an existing concrete generator (AssistedCandidateGenerator, PromptLookupCandidateGenerator) instead of the base class
3. If ABC-style protection is desired earlier, subclass with abc.abstractmethod so instantiation fails fast

Example fix

class MyGenerator(CandidateGenerator):
    def get_candidates(self, input_ids, **kwargs):
        # before: method missing -> NotImplementedError
        # after:
        return input_ids[:, -1:].repeat(1, self.num_output_tokens), None
    def update_candidate_strategy(self, input_ids, scores, num_matches):
        pass

from transformers.generation.candidate_generator import CandidateGenerator

def can_generate_candidates(gen) -> bool:
    return not getattr(type(gen).get_candidates, "is_abstract", False) and type(gen).get_candidates is not CandidateGenerator.get_candidates

def is_concrete_candidate_generator(obj) -> bool:
    from transformers.generation.candidate_generator import CandidateGenerator
    return isinstance(obj, CandidateGenerator) and type(obj).get_candidates is not CandidateGenerator.get_candidates

try:
    candidates = gen.get_candidates(input_ids)
except NotImplementedError:
    raise TypeError(f"{type(gen).__name__} is not usable for assisted generation: implement get_candidates")

When it happens

Trigger: Directly instantiating CandidateGenerator(), or subclassing it (e.g. a custom candidate generator) without implementing get_candidates, then running model.generate(assistant_model=...) so the assisted-decoding loop calls get_candidates.

Common situations: Writing a custom speculative-decoding candidate generator and missing a required method, or calling the abstract API from tests.

Related errors

• {} is an abstract class. Only classes inheriting this class
• Invalid max_matching_ngram_size or num_output_tokens
• Expected assistant_model to be a Gemma4AssistantForCausalLM
• `model_outputs` cannot be None, and they need to contain `hi
• Could not find `num_mtp_layers` in the model config. This mo

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/ae7785788944db39. Report an issue: GitHub.