{"record":{"id":"760022d5b45cb58c","repo":"hankcs/HanLP","slug":"the-first-two-dimensions-of-emissions-and-mask-mus","errorCode":null,"errorMessage":"the first two dimensions of emissions and mask must match, got {tuple(emissions.shape[:2])} and {tuple(mask.shape)}","messagePattern":"the first two dimensions of emissions and mask must match, got (.+?) and (.+?)","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"hanlp/layers/crf/crf.py","lineNumber":180,"sourceCode":"            emissions: torch.Tensor,\n            tags: Optional[torch.LongTensor] = None,\n            mask: Optional[torch.ByteTensor] = None) -> None:\n        if emissions.dim() != 3:\n            raise ValueError(f'emissions must have dimension of 3, got {emissions.dim()}')\n        if emissions.size(2) != self.num_tags:\n            raise ValueError(\n                f'expected last dimension of emissions is {self.num_tags}, '\n                f'got {emissions.size(2)}')\n\n        if tags is not None:\n            if emissions.shape[:2] != tags.shape:\n                raise ValueError(\n                    'the first two dimensions of emissions and tags must match, '\n                    f'got {tuple(emissions.shape[:2])} and {tuple(tags.shape)}')\n\n        if mask is not None:\n            if emissions.shape[:2] != mask.shape:\n                raise ValueError(\n                    'the first two dimensions of emissions and mask must match, '\n                    f'got {tuple(emissions.shape[:2])} and {tuple(mask.shape)}')\n            no_empty_seq = not self.batch_first and mask[0].all()\n            no_empty_seq_bf = self.batch_first and mask[:, 0].all()\n            if not no_empty_seq and not no_empty_seq_bf:\n                raise ValueError('mask of the first timestep must all be on')\n\n    def _compute_score(\n            self, emissions: torch.Tensor, tags: torch.LongTensor,\n            mask: torch.ByteTensor) -> torch.Tensor:\n        # emissions: (seq_length, batch_size, num_tags)\n        # tags: (seq_length, batch_size)\n        # mask: (seq_length, batch_size)\n        assert emissions.dim() == 3 and tags.dim() == 2\n        assert emissions.shape[:2] == tags.shape\n        assert emissions.size(2) == self.num_tags\n        assert mask.shape == tags.shape\n        assert mask[0].all()","sourceCodeStart":162,"sourceCodeEnd":198,"githubUrl":"https://github.com/hankcs/HanLP/blob/ddb1299bddff079e447af52ec12549c50636bfa8/hanlp/layers/crf/crf.py#L162-L198","documentation":"Raised by HanLP's TorchCRF._validate during forward/decode when mask shape does not equal emissions.shape[:2]. The mask marks valid timesteps per sequence and must have exactly the same (seq, batch) or (batch, seq) dimensions as emissions. This matches torchcrf's contract so masking in Viterbi/score computations is valid.","triggerScenarios":"Passing a mask of different length/batch size than emissions, e.g. mask built from tags lengths while emissions were padded to another length, or a transposed mask when batch_first is False.","commonSituations":"Building mask from lengths with a different max_len than the padded emissions; batch_first layout mismatch between mask and CRF config; reusing a mask from a previous batch with different padding.","solutions":["Generate mask from the same lengths used for padding: mask = torch.arange(T)[None,:] < lengths[:,None] (batch_first) sized to emissions.shape[:2]","Confirm the CRF batch_first flag matches the layout of emissions and mask","Verify your padding collator produces emissions and mask from identical metadata"],"exampleFix":"# before\nmask = torch.ones(tags.shape, dtype=torch.uint8)  # wrong shape\n# after\nmask = (torch.arange(emissions.shape[1])[None, :] < lengths[:, None]).to(emissions.device)\n# mask.shape == emissions.shape[:2] when batch_first=True","handlingStrategy":"validation","validationCode":"assert tuple(emissions.shape[:2]) == tuple(mask.shape), (emissions.shape, mask.shape)","typeGuard":"def mask_ok(emissions: torch.Tensor, mask: torch.Tensor) -> bool:\n    return tuple(emissions.shape[:2]) == tuple(mask.shape)","tryCatchPattern":"try:\n    out = crf.decode(emissions, mask=mask)\nexcept ValueError as e:\n    if 'emissions and mask' in str(e):\n        mask = build_mask(lengths, emissions.shape[:2]); out = crf.decode(emissions, mask=mask)\n    else:\n        raise","preventionTips":["Build mask from the same lengths used by the collate function","Keep a single batch_first convention","Unit-test collator output shapes"],"tags":["hanlp","crf","pytorch","mask","shape-mismatch"],"backgroundTag":"tensor-shape-mismatch","analyzedSha":"ddb1299bddff079e447af52ec12549c50636bfa8","analyzedAt":"2026-08-27T03:36:54.287Z","schemaVersion":2},"datasetVersion":"2026-08-27T08:17:20.692Z"}