huggingface/transformers · error · ValueError

Expected class name to start with Gemma4 or Gemma3n. Got {se

Error message

Expected class name to start with Gemma4 or Gemma3n. Got {self.__class__.__name__}. Gemma4Assistant models require a target model that provides a shared_kv_states dictionary. Currently, only Gemma4 and Gemma3n provide a shared_kv_states dictionary.

What it means

ValueError in candidate-generator selection: you attached a Gemma4Assistant/Gemma4UnifiedAssistant as the assistant model, which consumes a shared_kv_states dictionary produced by the target model. Only target models whose class names start with Gemma4 or Gemma3n expose shared_kv_states, so any other target is rejected up front rather than failing mid-decode with a KeyError.

Source

Thrown at src/transformers/generation/utils.py:1049

                max_length=generation_config.max_length,
                logits_processor=logits_processor,
                vocab_size=self.config.get_text_config().vocab_size,
            )
        elif generation_config.use_mtp:
            candidate_generator = MTPCandidateGenerator(
                main_model=self,
                generation_config=generation_config,
                logits_processor=logits_processor,
                model_kwargs=model_kwargs,
            )
        # SinglePositionMultiTokenCandidateGenerator requires a target model that can provide, and an assistant model that
        # can work from a shared_kv_states dictionary. Currently, the only models that can provide this are Gemma 3n and
        # Gemma 4, and the only model that can work from it is a Gemma 4 Assistant
        elif assistant_model is not None and assistant_model.__class__.__name__.startswith(
            ("Gemma4Assistant", "Gemma4UnifiedAssistant")
        ):
            if not self.__class__.__name__.startswith(("Gemma4", "Gemma3n")):
                raise ValueError(
                    f"Expected class name to start with Gemma4 or Gemma3n. Got {self.__class__.__name__}."
                    " Gemma4Assistant models require a target model that provides a shared_kv_states dictionary."
                    " Currently, only Gemma4 and Gemma3n provide a shared_kv_states dictionary."
                )

            candidate_generator = SinglePositionMultiTokenCandidateGenerator(
                input_ids=input_ids,
                assistant_model=assistant_model,
                target_model_input_embeddings=self.get_input_embeddings(),
                generation_config=generation_config,
                model_kwargs=model_kwargs,
                inputs_tensor=inputs_tensor,
                logits_processor=logits_processor,
            )
        elif generation_config.speculation_type == "dflash":
            candidate_generator = DFlashTokenCandidateGenerator(
                assistant_model=assistant_model,
                main_model_input_embeddings=self.get_input_embeddings(),

View on GitHub (pinned to a597f97485)

Solutions

1. Use a Gemma4 or Gemma3n target model with the Gemma4Assistant.
2. Or use a target-matched assistant (same architecture family) via the normal AssistantCandidateGenerator path.
3. If you wrapped Gemma4 in a custom class, ensure inspect-compatible class naming or bypass the SinglePosition path explicitly.

Example fix

# before
out = llama_model.generate(**inputs, assistant_model=gemma4_assistant)  # raises

# after
out = gemma4_model.generate(**inputs, assistant_model=gemma4_assistant)

target_ok = model.__class__.__name__.startswith(("Gemma4", "Gemma3n"))
assistant_is_g4 = assistant_model.__class__.__name__.startswith(("Gemma4Assistant", "Gemma4UnifiedAssistant"))
if assistant_is_g4 and not target_ok:
    raise ValueError("Gemma4Assistant requires a Gemma4/Gemma3n target model")

def is_compatible_gemma_pair(target, assistant) -> bool:
    a = assistant.__class__.__name__.startswith(("Gemma4Assistant", "Gemma4UnifiedAssistant"))
    t = target.__class__.__name__.startswith(("Gemma4", "Gemma3n"))
    return (not a) or t

When it happens

Trigger: model.generate(..., assistant_model=gemma4_assistant) where model is e.g. LlamaForCausalLM or any non-Gemma4/Gemma3n decoder; mixing checkpoints when experimenting with MTP-style speculative decoding.

Common situations: Trying to graft a Gemma4 assistant onto a different-architecture target for speed; wrappers that rename classes (custom subclass of Gemma4 whose name no longer starts with 'Gemma4').

Related errors

• Setting `assistant_ensemble_weight` requires candidate logit
• Calling `get_mtp_config` on a config without `num_mtp_layers
• Calling `get_mtp_config` on a config containing `layer_types
• Could not find `num_mtp_layers` in the model config. This mo
• `model_outputs` cannot be None, and they need to contain `hi

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/de5080854c594b73. Report an issue: GitHub.