{"record":{"id":"dac99c1a9ec1d915","repo":"hiyouga/LlamaFactory","slug":"logits-batchsize-x-seqlen-and-labels-must-have-t","errorCode":null,"errorMessage":"Logits (batchsize x seqlen) and labels must have the same shape.","messagePattern":"Logits \\(batchsize x seqlen\\) and labels must have the same shape\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"critical","filePath":"src/llamafactory/train/trainer_utils.py","lineNumber":606,"sourceCode":"        for param in optimizer_dict.keys():\n            param.register_post_accumulate_grad_hook(scheduler_hook)\n\n\ndef get_batch_logps(\n    logits: \"torch.Tensor\",\n    labels: \"torch.Tensor\",\n    label_pad_token_id: int = IGNORE_INDEX,\n    ld_alpha: Optional[float] = None,\n) -> tuple[\"torch.Tensor\", \"torch.Tensor\"]:\n    r\"\"\"Compute the log probabilities of the given labels under the given logits.\n\n    Returns:\n        logps: A tensor of shape (batch_size,) containing the sum of log probabilities.\n        valid_length: A tensor of shape (batch_size,) containing the number of non-masked tokens.\n\n    \"\"\"\n    if logits.shape[:-1] != labels.shape:\n        raise ValueError(\"Logits (batchsize x seqlen) and labels must have the same shape.\")\n\n    labels = labels[:, 1:].clone()\n    logits = logits[:, :-1, :]\n    loss_mask = labels != label_pad_token_id\n    labels[labels == label_pad_token_id] = 0  # dummy token\n    per_token_logps = torch.gather(logits.log_softmax(-1), dim=2, index=labels.unsqueeze(2)).squeeze(2)\n\n    valid_length = loss_mask.sum(-1)\n    if ld_alpha is not None:\n        num_examples = labels.shape[0] // 2\n        chosen_lengths = valid_length[:num_examples]\n        rejected_lengths = valid_length[num_examples:]\n        min_lengths = torch.min(chosen_lengths, rejected_lengths)\n        start_positions = torch.argmax(loss_mask.int(), dim=1)\n        public_lengths = start_positions + torch.cat([min_lengths, min_lengths], dim=0)\n\n        seq_len = labels.shape[-1]\n        position_ids = torch.arange(seq_len, device=per_token_logps.device).expand_as(per_token_logps)","sourceCodeStart":588,"sourceCodeEnd":624,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/src/llamafactory/train/trainer_utils.py#L588-L624","documentation":"`get_batch_logps` (src/llamafactory/train/trainer_utils.py:606) computes per-sequence log-probabilities used by DPO/SimPO-style losses. Before shifting labels, it asserts logits.shape[:-1] == labels.shape, i.e. logits must be (batch, seq_len, vocab) matching labels (batch, seq_len). A mismatch means the model emitted logits for a different sequence length than the labels (e.g. only last k positions via num_logits_to_keep, or truncated/extended sequences).","triggerScenarios":"Calling get_batch_logps with logits from a forward that used num_logits_to_keep/logits_to_keep (shorter first dim), labels sliced to a different length, packing/neat-packing mismatches, or a custom model that returns padded/truncated logits.","commonSituations":"Preference-training (DPO/KTO) code paths after a transformers version change that enables logits_to_keep by default; hand-written loops that pass labels[:, :-1] or logits before the function's own shift; multimodal setups where image tokens change expected lengths.","solutions":["Pass full-sequence logits: run forward without num_logits_to_keep (or set it to 0/None) before calling get_batch_logps.","Pass raw (batch, seq_len) labels — the function performs the [1:]/[:-1] shift itself; do not pre-shift.","Check that cutoff_len/packing settings are applied identically to inputs and labels in your custom loop.","Pin/align the transformers version with the one your LlamaFactory release expects (logits_to_keep behavior changed across versions)."],"exampleFix":"# before\noutputs = model(**inputs, num_logits_to_keep=16)\nlogps, seqlens = get_batch_logps(logits=outputs.logits, labels=labels)  # logits shorter than labels\n\n# after\noutputs = model(**inputs)\nlogps, seqlens = get_batch_logps(logits=outputs.logits, labels=labels)","handlingStrategy":"type-guard","validationCode":"def check_logps_inputs(logits, labels):\n    assert logits.dim() == 3 and logits.shape[:-1] == labels.shape, (\n        f\"logits {tuple(logits.shape)} vs labels {tuple(labels.shape)}\"\n    )","typeGuard":"def is_full_seq_logits_pair(logits: \"torch.Tensor\", labels: \"torch.Tensor\") -> bool:\n    return logits.dim() == 3 and logits.shape[:-1] == labels.shape","tryCatchPattern":"try:\n    logps, len = get_batch_logps(logits, labels)\nexcept ValueError as e:\n    if \"same shape\" in str(e):\n        raise RuntimeError(\"model returned truncated logits; disable num_logits_to_keep\") from e\n    raise","preventionTips":["Always forward full sequences (no num_logits_to_keep) before get_batch_logps.","Never pre-shift labels; the function shifts internally.","Add a shape assert in custom training loops to fail before the loss, not inside it."],"tags":["dpo","logits","shape-mismatch","preference-training"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}