microsoft/VibeVoice · critical · TypeError

semantic_tokenizer_config has unexpected type: {type(sc_cfg)

Error message

semantic_tokenizer_config has unexpected type: {type(sc_cfg)}

What it means

Identical guard to the acoustic one, but for `semantic_tokenizer_config`: the constructor accepts only a VibeVoiceSemanticTokenizerConfig instance or a plain dict (unpacked via **sc_cfg). Any other type raises this TypeError and model construction aborts.

Source

Thrown at vllm_plugin/model.py:233

        sc_cfg = get_cfg(config, "semantic_tokenizer_config")
        
        if ac_cfg is None or sc_cfg is None:
            raise ValueError("Missing acoustic/semantic tokenizer config in model config")

        # Handle both dict and already-constructed config objects
        if isinstance(ac_cfg, VibeVoiceAcousticTokenizerConfig):
            acoustic_config = ac_cfg
        elif isinstance(ac_cfg, dict):
            acoustic_config = VibeVoiceAcousticTokenizerConfig(**ac_cfg)
        else:
            raise TypeError(f"acoustic_tokenizer_config has unexpected type: {type(ac_cfg)}")
        
        if isinstance(sc_cfg, VibeVoiceSemanticTokenizerConfig):
            semantic_config = sc_cfg
        elif isinstance(sc_cfg, dict):
            semantic_config = VibeVoiceSemanticTokenizerConfig(**sc_cfg)
        else:
            raise TypeError(f"semantic_tokenizer_config has unexpected type: {type(sc_cfg)}")
        
        # Tokenizers use float32 for numerical precision
        self.acoustic_tokenizer = VibeVoiceAcousticTokenizerModel(acoustic_config)
        self.semantic_tokenizer = VibeVoiceSemanticTokenizerModel(semantic_config)
        
        # Get audio encoder dtype from config (defaults to float32 for precision)
        root_torch_dtype = get_cfg(config, "torch_dtype", None)
        if root_torch_dtype is not None:
            if isinstance(root_torch_dtype, str):
                self._audio_encoder_dtype = getattr(torch, root_torch_dtype)
            else:
                self._audio_encoder_dtype = root_torch_dtype
        else:
            self._audio_encoder_dtype = torch.float32
        
        self.acoustic_connector = SpeechConnector(self.acoustic_vae_dim, self.hidden_size)
        self.semantic_connector = SpeechConnector(self.semantic_vae_dim, self.hidden_size)
        

View on GitHub (pinned to 94da20d98b)

Solutions

1. Make `semantic_tokenizer_config` a plain JSON object/dict in config.json (parse it out of any string form with json.loads).
2. Convert non-dict mapping objects to dicts: OmegaConf.to_container(...) or dict(vars(...)).
3. Verify a single vibevoice package install so the isinstance(sc_cfg, VibeVoiceSemanticTokenizerConfig) check uses the same class the config was built from.
4. Set it explicitly before engine init: config.semantic_tokenizer_config = VibeVoiceSemanticTokenizerConfig(**d).

Example fix

# before
"semantic_tokenizer_config": "[1, 2, 3]"  # list/string, rejected

# after
"semantic_tokenizer_config": {"hidden_size": 1024, "n_codebooks": 8, ...}

import json

def validate_semantic_cfg(cfg: dict) -> bool:
    v = cfg.get("semantic_tokenizer_config")
    if isinstance(v, str):
        try:
            v = json.loads(v)
        except json.JSONDecodeError:
            return False
    return isinstance(v, dict) and len(v) > 0

def is_semantic_cfg_ok(cfg) -> bool:
    from omegaconf import OmegaConf
    v = cfg.get("semantic_tokenizer_config")
    if OmegaConf.is_config(v):
        v = OmegaConf.to_container(v, resolve=True)
    return isinstance(v, dict)

try:
    llm = LLM(model=model_id)
except TypeError as e:
    if "semantic_tokenizer_config has unexpected type" in str(e):
        raise SystemExit("semantic_tokenizer_config must be a dict/JSON object in config.json")
    raise

When it happens

Trigger: Same shape as the acoustic case: `semantic_tokenizer_config` present in the config but stored as a string, list, namespace, or OmegaConf DictConfig; or a constructed config object from a mismatched vibevoice version so both isinstance checks fail.

Common situations: Configs edited by hand where one section was re-serialized correctly but the other was left as an escaped string; training frameworks wrapping only part of the config; version skew between the checkpoint's config classes and the plugin's imported classes.

Related errors

• acoustic_tokenizer_config has unexpected type: {type(ac_cfg)
• Unsupported audio data type: {type(data)}
• Missing acoustic/semantic tokenizer config in model config
• Audio duration ({duration_sec:.1f}s) exceeds the configured
• Audio at index {item_idx} is too short to be represented

AI-assisted analysis of microsoft/VibeVoice@94da20d98b (2026-08-15). Data as JSON: /api/errors/01fb66548d528f96. Report an issue: GitHub.