microsoft/VibeVoice · critical · AttributeError

Cannot find embed_tokens layer

Error message

Cannot find embed_tokens layer

What it means

The model tries three attribute paths to find the token-embedding layer on the wrapped language model (language_model.model.embed_tokens, language_model.embed_tokens, then an inner .language_model.model.embed_tokens chain). If none exist it raises AttributeError, because without embed_tokens it cannot convert input_ids to embeddings or merge audio embeddings.

Source

Thrown at vllm_plugin/model.py:1125

        to embeddings during decode phase.
        
        Returns:
            The embed_tokens module from the language model
        """
        # Get embed_tokens from the language model
        if hasattr(self.language_model, 'model') and hasattr(self.language_model.model, 'embed_tokens'):
            return self.language_model.model.embed_tokens
        elif hasattr(self.language_model, 'embed_tokens'):
            return self.language_model.embed_tokens
        else:
            # Try to get from inner model
            inner = self.language_model
            if hasattr(inner, 'language_model'):
                inner = inner.language_model
            if hasattr(inner, 'model') and hasattr(inner.model, 'embed_tokens'):
                return inner.model.embed_tokens
        
        raise AttributeError("Cannot find embed_tokens layer")
    
    def embed_input_ids(
        self, 
        input_ids: torch.Tensor,
        multimodal_embeddings: Optional[Union[torch.Tensor, List[torch.Tensor]]] = None,
        is_multimodal: Optional[torch.Tensor] = None,
        **kwargs,  # Accept any additional kwargs for compatibility
    ) -> torch.Tensor:
        """Apply token embeddings to input_ids and merge with multimodal embeddings.
        
        This is the preferred method in vLLM V1 for converting token IDs
        to embeddings and merging multimodal (audio) embeddings.
        
        Args:
            input_ids: Tensor of token IDs to embed
            multimodal_embeddings: Pre-computed multimodal embeddings (audio).
                                   Can be a Tensor or a List of Tensors (vLLM standard).
            is_multimodal: Boolean mask indicating which positions are multimodal

View on GitHub (pinned to 94da20d98b)

Solutions

1. Align versions: use the vLLM release the plugin was built and tested against (check the plugin's README/requirements pin).
2. In a debugger, inspect dir(model.language_model) / type(model.language_model) at load time to find where embed_tokens actually lives, then report/patch the lookup chain accordingly.
3. Prefer the standard accessor if available: model.language_model.get_input_embeddings() — patch the helper to call this first with the attribute chain as fallback.
4. Disable experimental quantization wrappers for the language model to rule out proxy-object attribute hiding.

Example fix

# before (model.py lookup)
if hasattr(self.language_model, 'model') and hasattr(self.language_model.model, 'embed_tokens'):
    return self.language_model.model.embed_tokens
...

# after
if hasattr(self.language_model, 'get_input_embeddings'):
    return self.language_model.get_input_embeddings()
if hasattr(self.language_model, 'model') and hasattr(self.language_model.model, 'embed_tokens'):
    return self.language_model.model.embed_tokens
...

def find_embed_tokens(model):
    """Probe the wrapper the same way the plugin does, before serving traffic."""
    lm = model.language_model
    for probe in (
        lambda: lm.model.embed_tokens,
        lambda: lm.embed_tokens,
        lambda: lm.language_model.model.embed_tokens,
        lambda: lm.get_input_embeddings(),
    ):
        try:
            tok = probe()
            if tok is not None:
                return tok
        except AttributeError:
            continue
    raise AttributeError("embed_tokens unreachable — vLLM/plugin version mismatch")

try:
    emb = model.get_embed_tokens()
except AttributeError as e:
    if "Cannot find embed_tokens" in str(e):
        raise SystemExit(
            "vLLM wrapper layout changed; align vllm and plugin versions "
            "or patch get_embed_tokens to use get_input_embeddings()")
    raise

When it happens

Trigger: A vLLM version upgrade that renames/restructures the registered model wrapper (e.g. embed_tokens moved under a different submodule or accessed via a method like get_input_embeddings()); a language model class that lazily creates submodules; a monkey-patched or quantized (bitsandbytes/AWQ) wrapper whose intermediate modules are replaced by proxy objects hiding attributes.

Common situations: Pinning the plugin to an older release while upgrading vLLM (init_vllm_registered_model returns a different wrapper layout per vLLM minor version); quantization backends swapping decoder layers with attribute-passthrough objects; custom language model classes not following the HF module naming convention.

Related errors

• Unsupported audio data type: {type(data)}
• Audio duration ({duration_sec:.1f}s) exceeds the configured
• Missing acoustic/semantic tokenizer config in model config
• acoustic_tokenizer_config has unexpected type: {type(ac_cfg)
• semantic_tokenizer_config has unexpected type: {type(sc_cfg)

AI-assisted analysis of microsoft/VibeVoice@94da20d98b (2026-08-15). Data as JSON: /api/errors/f6fbf231e24d9f41. Report an issue: GitHub.