hiyouga/LlamaFactory · error · ValueError

Cannot process the logits.

Error message

Cannot process the logits.

What it means

`eval_logit_processor` (src/llamafactory/train/sft/metric.py:56) is the `preprocess_logits_for_metrics` hook used with `compute_accuracy`. It first unwraps tuple/list logits (MoE aux-loss tensors) and then requires the result to be a rank-3 tensor of shape (batch_size, seq_len, vocab_size) so it can argmax over the vocab. If the tensor is not 3-D after unwrapping, it raises ValueError because the argmax reduction would be meaningless.

Source

Thrown at src/llamafactory/train/sft/metric.py:56

if is_nltk_available():
    from nltk.translate.bleu_score import SmoothingFunction, sentence_bleu  # type: ignore


if is_rouge_available():
    from rouge_chinese import Rouge  # type: ignore


def eval_logit_processor(logits: "torch.Tensor", labels: "torch.Tensor") -> "torch.Tensor":
    r"""Compute the token with the largest likelihood to reduce memory footprint."""
    if isinstance(logits, (list, tuple)):
        if logits[0].dim() == 3:  # (batch_size, seq_len, vocab_size)
            logits = logits[0]
        else:  # moe models have aux loss
            logits = logits[1]

    if logits.dim() != 3:
        raise ValueError("Cannot process the logits.")

    return torch.argmax(logits, dim=-1)


@dataclass
class ComputeAccuracy:
    r"""Compute accuracy and support `batch_eval_metrics`."""

    def _dump(self) -> Optional[dict[str, float]]:
        result = None
        if hasattr(self, "score_dict"):
            result = {k: float(np.mean(v)) for k, v in self.score_dict.items()}

        self.score_dict = {"accuracy": []}
        return result

    def __post_init__(self):
        self._dump()

View on GitHub (pinned to f28afaf635)

Solutions

1. Confirm the model is a causal LM (CausalLM head) and not a classification/reward-head model when using compute_accuracy.
2. Switch to `predict_with_generate: true` (ComputeSimilarity) which does not rely on this logit shape, or disable both metrics.
3. Upgrade/downgrade transformers to a version known to work with your model family so forward returns standard (batch, seq, vocab) logits.
4. If you control the model wrapper, ensure eval forward returns full 3-D logits or a (loss, logits) pair where logits is 3-D.

Example fix

# before (data/config yaml)
compute_accuracy: true   # fails on non-causal-LM logit shapes

# after
predict_with_generate: true

def logits_ok_for_accuracy(logits) -> bool:
    if isinstance(logits, (list, tuple)):
        logits = logits[0] if logits[0].dim() == 3 else logits[1]
    return logits.dim() == 3

When it happens

Trigger: Running SFT evaluation with `finetuning_args.compute_accuracy: true` when the model's forward returns logits that are not (batch, seq, vocab): e.g. a sequence-classification head (batch, num_labels), logits truncated by num_logits_to_keep producing unexpected ranks after list-unwrap, or a custom model wrapper returning a differently shaped tensor.

Common situations: Evaluating a model whose architecture/config is not a plain causal LM (classification head, custom head), mixing an incompatible transformers version that changes logit shapes, or MoE models whose aux-loss list layout differs so logits[0]/logits[1] selection picks the wrong element.

Related errors

• `predict_with_generate` cannot be set as True except SFT.
• `predict_with_generate` is not supported in KTransformers SF
• `compute_accuracy` is not supported in KTransformers SFT yet
• The length of packed example should be identical to the cuto
• `save_dir` already exists, use another one.

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/d8b17990a2f229c4. Report an issue: GitHub.