hankcs/HanLP · error · ValueError

expected last dimension of emissions is {self.num_tags}, got

Error message

expected last dimension of emissions is {self.num_tags}, got {emissions.size(2)}

What it means

CRF._validate also checks that the last dimension of emissions equals the num_tags the CRF was constructed with (its transition matrices are num_tags x num_tags). A mismatch — e.g. the scoring head outputs a different number of classes than the CRF expects — raises ValueError with both sizes.

Source

Thrown at hanlp/layers/crf/crf.py:168

        self._validate(emissions, mask=mask)
        if mask is None:
            mask = emissions.new_ones(emissions.shape[:2], dtype=torch.uint8)

        if self.batch_first:
            emissions = emissions.transpose(0, 1)
            mask = mask.transpose(0, 1)

        return self._viterbi_decode(emissions, mask)

    def _validate(
            self,
            emissions: torch.Tensor,
            tags: Optional[torch.LongTensor] = None,
            mask: Optional[torch.ByteTensor] = None) -> None:
        if emissions.dim() != 3:
            raise ValueError(f'emissions must have dimension of 3, got {emissions.dim()}')
        if emissions.size(2) != self.num_tags:
            raise ValueError(
                f'expected last dimension of emissions is {self.num_tags}, '
                f'got {emissions.size(2)}')

        if tags is not None:
            if emissions.shape[:2] != tags.shape:
                raise ValueError(
                    'the first two dimensions of emissions and tags must match, '
                    f'got {tuple(emissions.shape[:2])} and {tuple(tags.shape)}')

        if mask is not None:
            if emissions.shape[:2] != mask.shape:
                raise ValueError(
                    'the first two dimensions of emissions and mask must match, '
                    f'got {tuple(emissions.shape[:2])} and {tuple(mask.shape)}')
            no_empty_seq = not self.batch_first and mask[0].all()
            no_empty_seq_bf = self.batch_first and mask[:, 0].all()
            if not no_empty_seq and not no_empty_seq_bf:
                raise ValueError('mask of the first timestep must all be on')

View on GitHub (pinned to ddb1299bdd)

Solutions

Rebuild the CRF with num_tags matching the head output: CRF(num_tags=out_features)
Re-create the vocab/tag set so model head and CRF are built from the same vocab in one place
When loading checkpoints, rebuild the whole model from the current vocab rather than patching layers

Example fix

# before
self.crf = CRF(num_tags=20)
out = self.linear(h)          # out.shape[-1] == 25 -> error
# after
self.crf = CRF(num_tags=self.linear.out_features)

Defensive patterns

Strategy: validation

Validate before calling

assert emissions.size(-1) == crf.num_tags, f'{emissions.size(-1)} != {crf.num_tags}; rebuild CRF with matching num_tags'

Type guard

def emissions_match_crf(emissions: torch.Tensor, crf) -> bool:
    return emissions.dim() == 3 and emissions.size(-1) == crf.num_tags

Prevention

Derive CRF num_tags from the same vocab object that sizes the output layer
Rebuild the entire model from the current vocab when label sets change

When it happens

Trigger: Loading/rebuilding a model where the linear output layer size differs from CRF.num_tags (vocab changed between runs); constructing CRF with num_tags=N but the encoder head outputs M features; fine-tuning with a reduced/extended label set.

Common situations: Changing the label vocabulary and restoring old checkpoints; manually wiring a CRF with a hardcoded tag count; multi-task heads sharing a CRF with mismatched sizes.

Related errors

AI-assisted analysis of hankcs/HanLP@ddb1299bdd (2026-08-27). Data as JSON: /api/errors/d6b05887fccb06ee. Report an issue: GitHub.