{"record":{"id":"ae7785788944db39","repo":"huggingface/transformers","slug":"is-an-abstract-class-only-classes-inheriting-t","errorCode":null,"errorMessage":"{} is an abstract class. Only classes inheriting this class can call `get_candidates`.","messagePattern":"(.+?) is an abstract class\\. Only classes inheriting this class can call `get_candidates`\\.","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"src/transformers/generation/candidate_generator.py","lineNumber":57,"sourceCode":"class CandidateGenerator:\n    \"\"\"Abstract base class for all candidate generators that can be applied during assisted generation.\"\"\"\n\n    requires_model_outputs: bool = False\n\n    def get_candidates(self, input_ids: torch.LongTensor, **kwargs) -> tuple[torch.LongTensor, torch.FloatTensor]:\n        \"\"\"\n        Fetches the candidates to be tried for the current input.\n\n        Args:\n            input_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`):\n                Indices of input sequence tokens in the vocabulary. [What are input IDs?](../glossary#input-ids)\n\n        Return:\n            `torch.LongTensor` of shape `(batch_size, candidate_length)` containing the candidate sequences to be\n            assessed by the model and, optionally, a `torch.FloatTensor` of shape `(batch_size, candidate_length,\n            vocabulary_size)` containing the logits associated to each candidate.\n        \"\"\"\n        raise NotImplementedError(\n            f\"{self.__class__} is an abstract class. Only classes inheriting this class can call `get_candidates`.\"\n        )\n\n    def update_candidate_strategy(self, input_ids: torch.LongTensor, scores: torch.FloatTensor, num_matches: int):\n        \"\"\"\n        Updates the candidate generation strategy based on the outcomes.\n\n        Args:\n            input_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`):\n                Indices of input sequence tokens in the vocabulary. [What are input IDs?](../glossary#input-ids)\n            scores (`torch.FloatTensor` of shape `(batch_size, candidate_length, config.vocab_size)`):\n                Prediction scores of a language modeling head. These can be logits for each vocabulary when not using\n                beam search or log softmax for each vocabulary token when using beam search\n            num_matches (`int`):\n                The number of matches between the candidate sequences and the model predictions.\n        \"\"\"\n        raise NotImplementedError(\n            f\"{self.__class__} is an abstract class. Only classes inheriting this class can call \"","sourceCodeStart":39,"sourceCodeEnd":75,"githubUrl":"https://github.com/huggingface/transformers/blob/a597f974857b3d92939971296bc0deb93d33d780/src/transformers/generation/candidate_generator.py#L39-L75","documentation":"CandidateGenerator is an abstract base class for assisted-generation candidate sources; get_candidates is intentionally unimplemented. Instantiating the base class or a subclass that forgot to override get_candidates raises NotImplementedError when generation asks for candidates.","triggerScenarios":"Directly instantiating CandidateGenerator(), or subclassing it (e.g. a custom candidate generator) without implementing get_candidates, then running model.generate(assistant_model=...) so the assisted-decoding loop calls get_candidates.","commonSituations":"Writing a custom speculative-decoding candidate generator and missing a required method, or calling the abstract API from tests.","solutions":["Implement get_candidates(input_ids, **kwargs) in your CandidateGenerator subclass","Use an existing concrete generator (AssistedCandidateGenerator, PromptLookupCandidateGenerator) instead of the base class","If ABC-style protection is desired earlier, subclass with abc.abstractmethod so instantiation fails fast"],"exampleFix":"class MyGenerator(CandidateGenerator):\n    def get_candidates(self, input_ids, **kwargs):\n        # before: method missing -> NotImplementedError\n        # after:\n        return input_ids[:, -1:].repeat(1, self.num_output_tokens), None\n    def update_candidate_strategy(self, input_ids, scores, num_matches):\n        pass","handlingStrategy":"type-guard","validationCode":"from transformers.generation.candidate_generator import CandidateGenerator\n\ndef can_generate_candidates(gen) -> bool:\n    return not getattr(type(gen).get_candidates, \"is_abstract\", False) and type(gen).get_candidates is not CandidateGenerator.get_candidates","typeGuard":"def is_concrete_candidate_generator(obj) -> bool:\n    from transformers.generation.candidate_generator import CandidateGenerator\n    return isinstance(obj, CandidateGenerator) and type(obj).get_candidates is not CandidateGenerator.get_candidates","tryCatchPattern":"try:\n    candidates = gen.get_candidates(input_ids)\nexcept NotImplementedError:\n    raise TypeError(f\"{type(gen).__name__} is not usable for assisted generation: implement get_candidates\")","preventionTips":["Never instantiate the CandidateGenerator base class","Cover get_candidates in a smoke test for every custom generator subclass"],"tags":["python","transformers","generation","assisted-decoding","abstract-class"],"backgroundTag":null,"analyzedSha":"a597f974857b3d92939971296bc0deb93d33d780","analyzedAt":"2026-08-14T18:24:08.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}