{"record":{"id":"d8b17990a2f229c4","repo":"hiyouga/LlamaFactory","slug":"cannot-process-the-logits","errorCode":null,"errorMessage":"Cannot process the logits.","messagePattern":"Cannot process the logits\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/llamafactory/train/sft/metric.py","lineNumber":56,"sourceCode":"\nif is_nltk_available():\n    from nltk.translate.bleu_score import SmoothingFunction, sentence_bleu  # type: ignore\n\n\nif is_rouge_available():\n    from rouge_chinese import Rouge  # type: ignore\n\n\ndef eval_logit_processor(logits: \"torch.Tensor\", labels: \"torch.Tensor\") -> \"torch.Tensor\":\n    r\"\"\"Compute the token with the largest likelihood to reduce memory footprint.\"\"\"\n    if isinstance(logits, (list, tuple)):\n        if logits[0].dim() == 3:  # (batch_size, seq_len, vocab_size)\n            logits = logits[0]\n        else:  # moe models have aux loss\n            logits = logits[1]\n\n    if logits.dim() != 3:\n        raise ValueError(\"Cannot process the logits.\")\n\n    return torch.argmax(logits, dim=-1)\n\n\n@dataclass\nclass ComputeAccuracy:\n    r\"\"\"Compute accuracy and support `batch_eval_metrics`.\"\"\"\n\n    def _dump(self) -> Optional[dict[str, float]]:\n        result = None\n        if hasattr(self, \"score_dict\"):\n            result = {k: float(np.mean(v)) for k, v in self.score_dict.items()}\n\n        self.score_dict = {\"accuracy\": []}\n        return result\n\n    def __post_init__(self):\n        self._dump()","sourceCodeStart":38,"sourceCodeEnd":74,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/src/llamafactory/train/sft/metric.py#L38-L74","documentation":"`eval_logit_processor` (src/llamafactory/train/sft/metric.py:56) is the `preprocess_logits_for_metrics` hook used with `compute_accuracy`. It first unwraps tuple/list logits (MoE aux-loss tensors) and then requires the result to be a rank-3 tensor of shape (batch_size, seq_len, vocab_size) so it can argmax over the vocab. If the tensor is not 3-D after unwrapping, it raises ValueError because the argmax reduction would be meaningless.","triggerScenarios":"Running SFT evaluation with `finetuning_args.compute_accuracy: true` when the model's forward returns logits that are not (batch, seq, vocab): e.g. a sequence-classification head (batch, num_labels), logits truncated by num_logits_to_keep producing unexpected ranks after list-unwrap, or a custom model wrapper returning a differently shaped tensor.","commonSituations":"Evaluating a model whose architecture/config is not a plain causal LM (classification head, custom head), mixing an incompatible transformers version that changes logit shapes, or MoE models whose aux-loss list layout differs so logits[0]/logits[1] selection picks the wrong element.","solutions":["Confirm the model is a causal LM (CausalLM head) and not a classification/reward-head model when using compute_accuracy.","Switch to `predict_with_generate: true` (ComputeSimilarity) which does not rely on this logit shape, or disable both metrics.","Upgrade/downgrade transformers to a version known to work with your model family so forward returns standard (batch, seq, vocab) logits.","If you control the model wrapper, ensure eval forward returns full 3-D logits or a (loss, logits) pair where logits is 3-D."],"exampleFix":"# before (data/config yaml)\ncompute_accuracy: true   # fails on non-causal-LM logit shapes\n\n# after\npredict_with_generate: true","handlingStrategy":"validation","validationCode":"def logits_ok_for_accuracy(logits) -> bool:\n    if isinstance(logits, (list, tuple)):\n        logits = logits[0] if logits[0].dim() == 3 else logits[1]\n    return logits.dim() == 3","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Only enable compute_accuracy for causal-LM model types.","Smoke-test one eval batch with the hook before launching a long eval run."],"tags":["sft","evaluation","logits","metrics"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}