What causes "Loss computation is not implemented in this version." in microsoft/VibeVoice?

Call forward_lm without labels and compute your own loss from last_hidden_state if fine-tuning.

microsoft/VibeVoice · error · NotImplementedError

Loss computation is not implemented in this version.

Error message

Loss computation is not implemented in this version.

What it means

forward_lm in the streaming inference wrapper is inference-only: it runs the base text LM and returns BaseModelOutputWithPast; if labels is not None it raises NotImplementedError. The wrapper ships without training loss computation for the text stage.

Source

Thrown at vibevoice/modular/modeling_vibevoice_streaming_inference.py:392

            inputs_embeds = self.model.get_input_embeddings()(input_ids)

        outputs = self.model.language_model(
            inputs_embeds=inputs_embeds,
            attention_mask=attention_mask,
            position_ids=position_ids,
            past_key_values=past_key_values,
            use_cache=use_cache,
            output_attentions=output_attentions,
            output_hidden_states=output_hidden_states,
            return_dict=return_dict,
            cache_position=cache_position,
            **kwargs,
        )

        hidden_states = outputs[0] if not return_dict else outputs.last_hidden_state
                
        if labels is not None:
            raise NotImplementedError("Loss computation is not implemented in this version.")

        return BaseModelOutputWithPast(
            past_key_values=outputs.past_key_values,
            last_hidden_state=hidden_states,
            attentions=outputs.attentions,
        )

    # @can_return_tuple
    def forward_tts_lm(
        self,
        input_ids: torch.LongTensor = None,
        attention_mask: Optional[torch.Tensor] = None,
        position_ids: Optional[torch.LongTensor] = None,
        past_key_values: Optional[Tuple[Tuple[torch.FloatTensor]]] = None,
        inputs_embeds: Optional[torch.FloatTensor] = None,
        labels: Optional[torch.LongTensor] = None,
        use_cache: Optional[bool] = None,
        output_attentions: Optional[bool] = None,

View on GitHub (pinned to 94da20d98b)

Solutions

Call forward_lm without labels and compute your own loss from last_hidden_state if fine-tuning.
Use the non-streaming VibeVoiceModel classes (modeling_vibevoice.py) for training, which implement loss paths.
If Trainer passes labels automatically, subclass/override compute_loss to drop them.
Do not rely on .loss in the returned BaseModelOutputWithPast — it is never set here.

Example fix

# before
out = model.forward_lm(input_ids=ids, labels=labels)  # NotImplementedError

# after
out = model.forward_lm(input_ids=ids)  # inference only
loss = my_own_loss(out.last_hidden_state, targets)  # if training

Defensive patterns

Strategy: validation

Validate before calling

kwargs.pop("labels", None)  # inference wrapper has no loss path
out = model.forward_lm(**kwargs)

Type guard

def is_inference_only_forward(fn) -> bool:
    return getattr(fn, "__name__", "") == "forward_lm"

Try / catch

try:
    out = model.forward_lm(input_ids=ids, labels=labels)
except NotImplementedError:
    out = model.forward_lm(input_ids=ids)

Prevention

Never pass labels to the streaming inference wrapper
Compute custom losses from last_hidden_state if fine-tuning
Use the training model classes for loss-based training

When it happens

Trigger: Calling model.forward_lm(input_ids=..., labels=labels) — any non-None labels value triggers the raise immediately after the submodule call.

Common situations: Adapting a training script from the non-streaming model that passes labels; HF Trainer-style pipelines that always forward labels; SFT code reused against the streaming checkpoint.

Related errors

AI-assisted analysis of microsoft/VibeVoice@94da20d98b (2026-08-15). Data as JSON: /api/errors/04ad8b669a8ba822. Report an issue: GitHub.