{"record":{"id":"f6fbf231e24d9f41","repo":"microsoft/VibeVoice","slug":"cannot-find-embed-tokens-layer","errorCode":null,"errorMessage":"Cannot find embed_tokens layer","messagePattern":"Cannot find embed_tokens layer","errorType":"exception","errorClass":"AttributeError","httpStatus":null,"severity":"critical","filePath":"vllm_plugin/model.py","lineNumber":1125,"sourceCode":"        to embeddings during decode phase.\n        \n        Returns:\n            The embed_tokens module from the language model\n        \"\"\"\n        # Get embed_tokens from the language model\n        if hasattr(self.language_model, 'model') and hasattr(self.language_model.model, 'embed_tokens'):\n            return self.language_model.model.embed_tokens\n        elif hasattr(self.language_model, 'embed_tokens'):\n            return self.language_model.embed_tokens\n        else:\n            # Try to get from inner model\n            inner = self.language_model\n            if hasattr(inner, 'language_model'):\n                inner = inner.language_model\n            if hasattr(inner, 'model') and hasattr(inner.model, 'embed_tokens'):\n                return inner.model.embed_tokens\n        \n        raise AttributeError(\"Cannot find embed_tokens layer\")\n    \n    def embed_input_ids(\n        self, \n        input_ids: torch.Tensor,\n        multimodal_embeddings: Optional[Union[torch.Tensor, List[torch.Tensor]]] = None,\n        is_multimodal: Optional[torch.Tensor] = None,\n        **kwargs,  # Accept any additional kwargs for compatibility\n    ) -> torch.Tensor:\n        \"\"\"Apply token embeddings to input_ids and merge with multimodal embeddings.\n        \n        This is the preferred method in vLLM V1 for converting token IDs\n        to embeddings and merging multimodal (audio) embeddings.\n        \n        Args:\n            input_ids: Tensor of token IDs to embed\n            multimodal_embeddings: Pre-computed multimodal embeddings (audio).\n                                   Can be a Tensor or a List of Tensors (vLLM standard).\n            is_multimodal: Boolean mask indicating which positions are multimodal","sourceCodeStart":1107,"sourceCodeEnd":1143,"githubUrl":"https://github.com/microsoft/VibeVoice/blob/94da20d98b2fa7688e9cbfaf7692ddb4954f7600/vllm_plugin/model.py#L1107-L1143","documentation":"The model tries three attribute paths to find the token-embedding layer on the wrapped language model (language_model.model.embed_tokens, language_model.embed_tokens, then an inner .language_model.model.embed_tokens chain). If none exist it raises AttributeError, because without embed_tokens it cannot convert input_ids to embeddings or merge audio embeddings.","triggerScenarios":"A vLLM version upgrade that renames/restructures the registered model wrapper (e.g. embed_tokens moved under a different submodule or accessed via a method like get_input_embeddings()); a language model class that lazily creates submodules; a monkey-patched or quantized (bitsandbytes/AWQ) wrapper whose intermediate modules are replaced by proxy objects hiding attributes.","commonSituations":"Pinning the plugin to an older release while upgrading vLLM (init_vllm_registered_model returns a different wrapper layout per vLLM minor version); quantization backends swapping decoder layers with attribute-passthrough objects; custom language model classes not following the HF module naming convention.","solutions":["Align versions: use the vLLM release the plugin was built and tested against (check the plugin's README/requirements pin).","In a debugger, inspect dir(model.language_model) / type(model.language_model) at load time to find where embed_tokens actually lives, then report/patch the lookup chain accordingly.","Prefer the standard accessor if available: model.language_model.get_input_embeddings() — patch the helper to call this first with the attribute chain as fallback.","Disable experimental quantization wrappers for the language model to rule out proxy-object attribute hiding."],"exampleFix":"# before (model.py lookup)\nif hasattr(self.language_model, 'model') and hasattr(self.language_model.model, 'embed_tokens'):\n    return self.language_model.model.embed_tokens\n...\n\n# after\nif hasattr(self.language_model, 'get_input_embeddings'):\n    return self.language_model.get_input_embeddings()\nif hasattr(self.language_model, 'model') and hasattr(self.language_model.model, 'embed_tokens'):\n    return self.language_model.model.embed_tokens\n...","handlingStrategy":"fallback","validationCode":"def find_embed_tokens(model):\n    \"\"\"Probe the wrapper the same way the plugin does, before serving traffic.\"\"\"\n    lm = model.language_model\n    for probe in (\n        lambda: lm.model.embed_tokens,\n        lambda: lm.embed_tokens,\n        lambda: lm.language_model.model.embed_tokens,\n        lambda: lm.get_input_embeddings(),\n    ):\n        try:\n            tok = probe()\n            if tok is not None:\n                return tok\n        except AttributeError:\n            continue\n    raise AttributeError(\"embed_tokens unreachable — vLLM/plugin version mismatch\")","typeGuard":null,"tryCatchPattern":"try:\n    emb = model.get_embed_tokens()\nexcept AttributeError as e:\n    if \"Cannot find embed_tokens\" in str(e):\n        raise SystemExit(\n            \"vLLM wrapper layout changed; align vllm and plugin versions \"\n            \"or patch get_embed_tokens to use get_input_embeddings()\")\n    raise","preventionTips":["Pin vLLM to the version the plugin release was tested with; upgrade both together.","Run a one-request smoke test after any vLLM upgrade before admitting traffic.","Prefer get_input_embeddings() style accessors over attribute chains when patching."],"tags":["vllm","version-compatibility","embedding","model-wrappers","attribute-error"],"backgroundTag":null,"analyzedSha":"94da20d98b2fa7688e9cbfaf7692ddb4954f7600","analyzedAt":"2026-08-15T04:12:07.418Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}