microsoft/VibeVoice · error · NotImplementedError

Prediction type {prediction_type} not implemented

Error message

Prediction type {prediction_type} not implemented

What it means

The diffusion training loss in modeling_vibevoice.py supports only prediction_type 'epsilon' and 'v_prediction' (read from config.diffusion_head_config.prediction_type); any other value raises NotImplementedError when computing the denoising target. This mirrors the standard diffusers DDPM contract.

Source

Thrown at vibevoice/modular/modeling_vibevoice.py:457

            noisy_speech_features = self.model.noise_scheduler.add_noise(
                speech_features_repeated, noise, timesteps
            )
            
            model_output = self.model.prediction_head(
                noisy_speech_features, 
                timesteps.type_as(x), 
                condition_features_repeated
            )

            prediction_type = self.config.diffusion_head_config.prediction_type
            if prediction_type == "epsilon":
                target_for_loss = noise
            elif prediction_type == "v_prediction":
                target_for_loss = self.model.noise_scheduler.get_velocity(
                    speech_features_repeated, noise, timesteps
                )
            else:
                raise NotImplementedError(f"Prediction type {prediction_type} not implemented")

            diffusion_loss = F.mse_loss(model_output.float(), target_for_loss.float(), reduction='sum')
            if latent_size > 0 and ddpm_batch_mul > 0:
                diffusion_loss = diffusion_loss / latent_size / ddpm_batch_mul
            else:
                diffusion_loss = torch.tensor(0.0, device=diffusion_loss.device)
        
        else:
            # Dummy loss for DDP to work when there are no speech samples in a batch,
            # but we are in a speech context.
            diffusion_loss = sum(p.sum() for p in self.model.prediction_head.parameters()) * 0.0
            diffusion_loss += sum(p.sum() for p in self.model.acoustic_connector.parameters()) * 0.0
            diffusion_loss += sum(p.sum() for p in self.model.semantic_connector.parameters()) * 0.0
        # --- End Diffusion Loss Calculation ---

        if not return_dict:
            output = (logits, speech_len) + outputs.to_tuple()[1:]
            return (loss, diffusion_loss) + output

View on GitHub (pinned to 94da20d98b)

Solutions

  1. Set diffusion_head_config.prediction_type to 'epsilon' or 'v_prediction' in the config used for training.
  2. Inspect checkpoint config.json: diffusion_head_config.prediction_type must be one of the two values.
  3. If 'sample' prediction is required, implement the target = speech_features_repeated branch before the raise.
  4. Add a startup assert validating prediction_type against the supported set.

Example fix

# before
cfg.diffusion_head_config.prediction_type = "sample"  # -> NotImplementedError

# after
cfg.diffusion_head_config.prediction_type = "v_prediction"
Defensive patterns

Strategy: validation

Validate before calling

pt = config.diffusion_head_config.prediction_type
assert pt in {"epsilon", "v_prediction"}, f"Unsupported prediction_type {pt!r}"

Type guard

def is_supported_prediction_type(pt: object) -> bool:
    return pt in ("epsilon", "v_prediction")

Try / catch

try:
    loss = trainer_step(...)
except NotImplementedError as e:
    raise SystemExit(f"Fix diffusion config: {e}") from e

Prevention

When it happens

Trigger: Training with diffusion_head_config.prediction_type set to 'sample', 'sample_prediction', or an empty/typo value; loading a checkpoint whose diffusion head config carries an unsupported prediction type.

Common situations: Copy-pasting scheduler settings from a diffusers pipeline that uses 'sample'; fine-tunes editing diffusion_head_config; older config files with a different key name defaulting to None.

Related errors

AI-assisted analysis of microsoft/VibeVoice@94da20d98b (2026-08-15). Data as JSON: /api/errors/d6de259fb16ac9f3. Report an issue: GitHub.