microsoft/VibeVoice · critical · ValueError

Missing acoustic/semantic tokenizer config in model config

Error message

Missing acoustic/semantic tokenizer config in model config

What it means

Thrown while constructing the VibeVoice vLLM model: the HF model config must contain both `acoustic_tokenizer_config` and `semantic_tokenizer_config` sections (fetched via get_cfg on the model config). These describe the two tokenizer heads the model instantiates (VibeVoiceAcousticTokenizerModel / VibeVoiceSemanticTokenizerModel). If either is absent (get_cfg returns None), the model refuses to build because there is no sane default for tokenizer hyperparameters.

Source

Thrown at vllm_plugin/model.py:218

            target_hidden_size = get_cfg(decoder_config, "hidden_size")

        if target_hidden_size is None and text_config is not None:
            target_hidden_size = get_cfg(text_config, "hidden_size")

        if target_hidden_size is None:
            target_hidden_size = get_cfg(config, "hidden_size")

        if target_hidden_size is None:
            print("[VibeVoice] WARN: Could not find hidden_size in config! Defaulting to 3584 (7B).", file=sys.stderr)
            self.hidden_size = 3584
        else:
            self.hidden_size = target_hidden_size

        ac_cfg = get_cfg(config, "acoustic_tokenizer_config")
        sc_cfg = get_cfg(config, "semantic_tokenizer_config")
        
        if ac_cfg is None or sc_cfg is None:
            raise ValueError("Missing acoustic/semantic tokenizer config in model config")

        # Handle both dict and already-constructed config objects
        if isinstance(ac_cfg, VibeVoiceAcousticTokenizerConfig):
            acoustic_config = ac_cfg
        elif isinstance(ac_cfg, dict):
            acoustic_config = VibeVoiceAcousticTokenizerConfig(**ac_cfg)
        else:
            raise TypeError(f"acoustic_tokenizer_config has unexpected type: {type(ac_cfg)}")
        
        if isinstance(sc_cfg, VibeVoiceSemanticTokenizerConfig):
            semantic_config = sc_cfg
        elif isinstance(sc_cfg, dict):
            semantic_config = VibeVoiceSemanticTokenizerConfig(**sc_cfg)
        else:
            raise TypeError(f"semantic_tokenizer_config has unexpected type: {type(sc_cfg)}")
        
        # Tokenizers use float32 for numerical precision
        self.acoustic_tokenizer = VibeVoiceAcousticTokenizerModel(acoustic_config)

View on GitHub (pinned to 94da20d98b)

Solutions

Verify the target repo's config.json contains top-level `acoustic_tokenizer_config` and `semantic_tokenizer_config` objects; if missing, copy those sections from the official VibeVoice model repo of the matching version.
Confirm you are loading the actual VibeVoice model ID, not the underlying Qwen2.5 base model.
Re-save the config from a working checkout: load the official model, model.config.save_pretrained(...), then merge your weight changes.
If the keys exist under different names (e.g. nested in `decoder_config`), normalize them to the top level expected by get_cfg.

Example fix

# before (config.json)
{"architectures": ["Qwen2ForCausalLM"], "hidden_size": 3584, ...}

# after (config.json)
{
  "architectures": ["VibeVoiceForConditionalGeneration"],
  "hidden_size": 3584,
  "acoustic_tokenizer_config": {"hidden_size": 1024, "vocab_size": 8194, ...},
  "semantic_tokenizer_config": {"hidden_size": 1024, "vocab_size": 8194, ...},
  ...
}

Defensive patterns

Strategy: validation

Validate before calling

import json, urllib.request

def validate_model_config(repo_id_or_path: str) -> bool:
    if "/" in repo_id_or_path and not repo_id_or_path.startswith(("/", ".")):
        cfg = json.load(urllib.request.urlopen(
            f"https://huggingface.co/{repo_id_or_path}/raw/main/config.json"))
    else:
        cfg = json.load(open(f"{repo_id_or_path}/config.json"))
    missing = [k for k in ("acoustic_tokenizer_config", "semantic_tokenizer_config")
               if not isinstance(cfg.get(k), dict)]
    if missing:
        print(f"config.json missing/invalid sections: {missing}")
        return False
    return True

Try / catch

try:
    llm = LLM(model=model_id, ...)
except ValueError as e:
    if "Missing acoustic/semantic tokenizer config" in str(e):
        raise SystemExit(
            f"{model_id} is not a full VibeVoice checkpoint; "
            "use the official repo or merge tokenizer config sections into config.json")
    raise

Prevention

Pin the official VibeVoice model repo ID in deployment configs; never point at the base Qwen repo.
After any checkpoint merge/quantization, diff config.json against the official release's config.json before serving.
Add a startup smoke test that constructs the engine once in CI.

When it happens

Trigger: Loading a fine-tune or converted checkpoint whose config.json was stripped down to LLM-only keys; pointing --model at a base Qwen2.5 repo instead of the VibeVoice repo; a config.json saved from a dataclass that dropped nested config sections; using a VibeVoice checkpoint older than the plugin's expected config schema.

Common situations: Merging/quantizing checkpoints (mergekit, AWQ, GGUF round-trips) where nested config dicts were flattened or lost; users copying only safetensors weights into a new repo without the full config.json; plugin version newer than the checkpoint's config format.

Related errors

AI-assisted analysis of microsoft/VibeVoice@94da20d98b (2026-08-15). Data as JSON: /api/errors/10d160e2eecd27f3. Report an issue: GitHub.