hiyouga/LlamaFactory · error · NotImplementedError

Unknown loss type: {self.loss_type}.

Error message

Unknown loss type: {self.loss_type}.

What it means

Error "Unknown loss type: {self.loss_type}." thrown in hiyouga/LlamaFactory.

Source

Thrown at src/llamafactory/train/dpo/trainer.py:201

            -(self.beta * rejected_logratios - delta)
        )
        return bco_loss

    def compute_preference_loss(
        self,
        policy_chosen_logps: "torch.Tensor",
        policy_rejected_logps: "torch.Tensor",
        reference_chosen_logps: Optional["torch.Tensor"],
        reference_rejected_logps: Optional["torch.Tensor"],
    ) -> tuple["torch.Tensor", "torch.Tensor", "torch.Tensor"]:
        r"""Compute loss for preference learning."""
        if not self.finetuning_args.use_ref_model:
            if self.loss_type == "orpo":
                losses = self.odds_ratio_loss(policy_chosen_logps, policy_rejected_logps)
            elif self.loss_type == "simpo":
                losses = self.simpo_loss(policy_chosen_logps, policy_rejected_logps)
            else:
                raise NotImplementedError(f"Unknown loss type: {self.loss_type}.")

            chosen_rewards = self.beta * policy_chosen_logps.to(self.accelerator.device).detach()
            rejected_rewards = self.beta * policy_rejected_logps.to(self.accelerator.device).detach()
        else:
            losses, chosen_rewards, rejected_rewards = self.dpo_loss(
                policy_chosen_logps, policy_rejected_logps, reference_chosen_logps, reference_rejected_logps
            )

            if self.bco_gemma > 1e-6:
                bco_losses = self.bco_loss(
                    policy_chosen_logps, policy_rejected_logps, reference_chosen_logps, reference_rejected_logps
                )
                losses = (losses + bco_losses * self.bco_gemma) / (1.0 + self.bco_gemma)  # re-weight W_p and W_q

        return losses, chosen_rewards, rejected_rewards

    @override
    def concatenated_forward(

View on GitHub (pinned to f28afaf635)

Solutions

1. Set pref_loss to a supported loss type for the chosen reference-model mode (e.g. sigmoid, hinge, ipo, kto_pair with a ref model, or orpo/simpo without a ref model).

Example fix

pref_loss: sigmoid

When it happens

Trigger: Thrown at src/llamafactory/train/dpo/trainer.py:201 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/7540c900d31febfc. Report an issue: GitHub.