microsoft/VibeVoice · error · RuntimeError

Unified forward is disabled. Use `forward_lm`, `forward_tts_

Error message

Unified forward is disabled. Use `forward_lm`, `forward_tts_lm`, or `generate` instead.

What it means

The streaming inference wrapper disables unified forward on purpose: generation is staged (base text LM prefill, windowed TTS LM stepping, diffusion sampling, audio assembly) and a single forward call would hide the required sequencing. Any model(...) call raises RuntimeError pointing to forward_lm, forward_tts_lm, or generate.

Source

Thrown at vibevoice/modular/modeling_vibevoice_streaming_inference.py:494

        )

    def forward(self, *args, **kwargs):
        """
        Unified forward is intentionally disabled.

        Reasons:
          1. The inference pipeline is staged: base text LM, then TTS LM, plus streaming & diffusion handled in `generate`.
          2. A monolithic call would hide required sequencing (prefill, window stepping, speech diffusion sampling).

        Use instead:
          - self.forward_lm(...)       for a base text LM step (prefill or incremental).
          - self.forward_tts_lm(...)   for a single TTS LM step (needs LM hidden states).
          - self.generate(...)         for full streaming (text + speech + diffusion + audio assembly).

        Raises:
            RuntimeError: Always (by design).
        """
        raise RuntimeError(
            "Unified forward is disabled. Use `forward_lm`, `forward_tts_lm`, or `generate` instead."
        )

    def _build_generate_config_model_kwargs(self, generation_config, inputs, tokenizer, return_processors=False, **kwargs):
        if generation_config is None:
            generation_config = GenerationConfig(
                bos_token_id=tokenizer.bos_token_id,
                eos_token_id=tokenizer.eos_token_id,
                pad_token_id = tokenizer.pad_token_id
            )
        else:
            generation_config = GenerationConfig(
                **generation_config,
                bos_token_id=tokenizer.bos_token_id,
                eos_token_id=tokenizer.eos_token_id,
                pad_token_id = tokenizer.pad_token_id
            )

View on GitHub (pinned to 94da20d98b)

Solutions

Use model.generate(...) for full streaming text+speech generation.
Use model.forward_lm(...) / model.forward_tts_lm(...) for the individual staged steps.
Wrap the staged calls in your own function if a callable interface is needed by external tooling.
Read the docstring at modeling_vibevoice_streaming_inference.py:494 for the intended sequencing before writing custom loops.

Example fix

# before
outputs = inference_model(input_ids=ids)  # RuntimeError

# after
outputs = inference_model.generate(input_ids=ids, ...)  # full streaming pipeline

Defensive patterns

Strategy: type-guard

Validate before calling

def run_generation(model, **kwargs):
    # unified forward is disabled on the streaming inference wrapper
    return model.generate(**kwargs)

Type guard

def is_streaming_inference_wrapper(model) -> bool:
    return hasattr(model, "forward_lm") and hasattr(model, "forward_tts_lm") and hasattr(model, "generate")

Try / catch

try:
    out = model(input_ids=ids)
except RuntimeError as e:
    if "forward is disabled" in str(e):
        out = model.generate(input_ids=ids)
    else:
        raise

Prevention

Use generate() for full streaming inference
Call forward_lm/forward_tts_lm only for staged custom loops
Keep generic serving harnesses away from direct module calls on this wrapper

When it happens

Trigger: Calling inference_model(inputs) or inference_model.forward(...); passing the wrapper to libraries that call module(*args) (accelerate, torch.compile entry, generic serving harnesses).

Common situations: Migrating code from the non-streaming model that had a working forward; generic inference servers that just call model(batch); notebook usage copy-pasted from standard transformers examples.

Related errors

AI-assisted analysis of microsoft/VibeVoice@94da20d98b (2026-08-15). Data as JSON: /api/errors/7a91c6a95c88c2a3. Report an issue: GitHub.