{"record":{"id":"83004253627a8499","repo":"hankcs/HanLP","slug":"invalid-reduction-reduction","errorCode":null,"errorMessage":"invalid reduction: {reduction}","messagePattern":"invalid reduction: (.+?)","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"hanlp/layers/crf/crf.py","lineNumber":111,"sourceCode":"                ``(seq_length, batch_size, num_tags)`` if ``batch_first`` is ``False``,\n                ``(batch_size, seq_length, num_tags)`` otherwise.\n            tags (`~torch.LongTensor`): Sequence of tags tensor of size\n                ``(seq_length, batch_size)`` if ``batch_first`` is ``False``,\n                ``(batch_size, seq_length)`` otherwise.\n            mask (`~torch.ByteTensor`): Mask tensor of size ``(seq_length, batch_size)``\n                if ``batch_first`` is ``False``, ``(batch_size, seq_length)`` otherwise.\n            reduction: Specifies  the reduction to apply to the output:\n                ``none|sum|mean|token_mean``. ``none``: no reduction will be applied.\n                ``sum``: the output will be summed over batches. ``mean``: the output will be\n                averaged over batches. ``token_mean``: the output will be averaged over tokens.\n\n        Returns:\n            `~torch.Tensor`: The log likelihood. This will have size ``(batch_size,)`` if\n            reduction is ``none``, ``()`` otherwise.\n        \"\"\"\n        self._validate(emissions, tags=tags, mask=mask)\n        if reduction not in ('none', 'sum', 'mean', 'token_mean'):\n            raise ValueError(f'invalid reduction: {reduction}')\n        if mask is None:\n            mask = torch.ones_like(tags, dtype=torch.uint8)\n\n        if self.batch_first:\n            emissions = emissions.transpose(0, 1)\n            tags = tags.transpose(0, 1)\n            mask = mask.transpose(0, 1)\n\n        # shape: (batch_size,)\n        numerator = self._compute_score(emissions, tags, mask)\n        # shape: (batch_size,)\n        denominator = self._compute_normalizer(emissions, mask)\n        # shape: (batch_size,)\n        llh = numerator - denominator\n\n        if reduction == 'none':\n            return llh\n        if reduction == 'sum':","sourceCodeStart":93,"sourceCodeEnd":129,"githubUrl":"https://github.com/hankcs/HanLP/blob/ddb1299bddff079e447af52ec12549c50636bfa8/hanlp/layers/crf/crf.py#L93-L129","documentation":"CRF.forward (used for compute_loss) reduces the total log-likelihood according to `reduction`, which must be one of 'none' (per-sequence), 'sum', 'mean' (batch mean), or 'token_mean' (mean per real token using the mask). Any other string raises ValueError.","triggerScenarios":"Calling crf(emissions, tags, mask, reduction=...) with an unsupported value like 'batch_mean', 'avg', or None; often from custom training loops choosing their own normalization.","commonSituations":"Writing a custom criterion around the CRF; copying reduction names from other loss APIs (e.g. 'mean'/'sum' from CrossEntropyLoss are fine but extras like 'batch' are not).","solutions":["Use 'none', 'sum', 'mean', or 'token_mean'","For per-token normalization with padding, 'token_mean' is what you want (not a custom 'token' string)"],"exampleFix":"# before\nloss = crf(emissions, tags, mask, reduction='batch')\n# after\nloss = crf(emissions, tags, mask, reduction='token_mean')","handlingStrategy":"validation","validationCode":"assert reduction in ('none', 'sum', 'mean', 'token_mean'), f'bad reduction {reduction!r}'","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Validate reduction strings at config load"],"tags":["python","pytorch","crf","argument-validation"],"backgroundTag":"invalid-argument-value","analyzedSha":"ddb1299bddff079e447af52ec12549c50636bfa8","analyzedAt":"2026-08-27T03:36:54.287Z","schemaVersion":2},"datasetVersion":"2026-08-27T08:17:20.692Z"}