hankcs/HanLP · error · ValueError

the first two dimensions of emissions and tags must match, g

Error message

the first two dimensions of emissions and tags must match, got {tuple(emissions.shape[:2])} and {tuple(tags.shape)}

What it means

Raised by HanLP's TorchCRF._validate during forward/decode when the batch/sequence dimensions of emissions and tags disagree. emissions must be (seq_len, batch, num_tags) or (batch, seq_len, num_tags) per batch_first, and tags must match the first two dims exactly. This mirrors torchcrf's validation, ensuring score computation is well-defined.

Source

Thrown at hanlp/layers/crf/crf.py:174

            mask = mask.transpose(0, 1)

        return self._viterbi_decode(emissions, mask)

    def _validate(
            self,
            emissions: torch.Tensor,
            tags: Optional[torch.LongTensor] = None,
            mask: Optional[torch.ByteTensor] = None) -> None:
        if emissions.dim() != 3:
            raise ValueError(f'emissions must have dimension of 3, got {emissions.dim()}')
        if emissions.size(2) != self.num_tags:
            raise ValueError(
                f'expected last dimension of emissions is {self.num_tags}, '
                f'got {emissions.size(2)}')

        if tags is not None:
            if emissions.shape[:2] != tags.shape:
                raise ValueError(
                    'the first two dimensions of emissions and tags must match, '
                    f'got {tuple(emissions.shape[:2])} and {tuple(tags.shape)}')

        if mask is not None:
            if emissions.shape[:2] != mask.shape:
                raise ValueError(
                    'the first two dimensions of emissions and mask must match, '
                    f'got {tuple(emissions.shape[:2])} and {tuple(mask.shape)}')
            no_empty_seq = not self.batch_first and mask[0].all()
            no_empty_seq_bf = self.batch_first and mask[:, 0].all()
            if not no_empty_seq and not no_empty_seq_bf:
                raise ValueError('mask of the first timestep must all be on')

    def _compute_score(
            self, emissions: torch.Tensor, tags: torch.LongTensor,
            mask: torch.ByteTensor) -> torch.Tensor:
        # emissions: (seq_length, batch_size, num_tags)
        # tags: (seq_length, batch_size)

View on GitHub (pinned to ddb1299bdd)

Solutions

Ensure tags.shape == emissions.shape[:2] before calling forward/decode
Construct the CRF with batch_first=True if your tensors are (batch, seq, ...) and verify all tensors use that layout
Check your collate/padding function pads emissions and tags to the same max length

Example fix

# before
crf = CRF(num_tags, batch_first=False)
loss = crf(emissions, tags)  # emissions (B,T,C), tags (B,T) -> error
# after
crf = CRF(num_tags, batch_first=True)
loss = crf(emissions, tags)  # emissions (B,T,C), tags (B,T)

Defensive patterns

Strategy: validation

Validate before calling

assert emissions.dim() == 3 and emissions.shape[:2] == tags.shape, (emissions.shape, tags.shape)

Type guard

def tags_match(emissions: torch.Tensor, tags: torch.Tensor) -> bool:
    return emissions.dim() == 3 and tuple(emissions.shape[:2]) == tuple(tags.shape)

Try / catch

try:
    loss = crf(emissions, tags)
except ValueError as e:
    if 'first two dimensions' in str(e):
        raise ValueError(f'Padding mismatch: {emissions.shape} vs {tags.shape}') from e
    raise

Prevention

Derive tags and emissions from the same padded batch
Standardize batch_first across the whole model
Assert shapes in debug builds before crf.forward

When it happens

Trigger: Calling crf(emissions, tags, mask) where tags has a different batch size or sequence length than emissions, e.g. tags trimmed/padded differently, or passing batch-first tags with seq-first emissions (or vice versa) when batch_first=False (the default).

Common situations: Mismatched padding between the encoder output and the tag tensor; using a data loader that pads emissions and labels with different lengths; forgetting the CRF defaults to batch_first=False while the rest of the pipeline is batch_first=True.

Related errors

AI-assisted analysis of hankcs/HanLP@ddb1299bdd (2026-08-27). Data as JSON: /api/errors/fb5499282ff869e5. Report an issue: GitHub.