{"record":{"id":"53c348cfd5248db0","repo":"open-mmlab/mmdetection","slug":"byte-tensor-for-attn-mask-is-deprecated","errorCode":null,"errorMessage":"Byte tensor for attn_mask is deprecated.                     Use bool tensor instead.","messagePattern":"Byte tensor for attn_mask is deprecated\\.                     Use bool tensor instead\\.","errorType":"console","errorClass":null,"httpStatus":null,"severity":"info","filePath":"mmdet/models/layers/transformer/utils.py","lineNumber":564,"sourceCode":"        assert head_dims * self.num_heads == hidden_dims, \\\n            f'{\"hidden_dims must be divisible by num_heads\"}'\n        scaling = float(head_dims)**-0.5\n\n        q = query * scaling\n        k = key\n        v = value\n\n        if attn_mask is not None:\n            assert attn_mask.dtype == torch.float32 or \\\n                   attn_mask.dtype == torch.float64 or \\\n                   attn_mask.dtype == torch.float16 or \\\n                   attn_mask.dtype == torch.uint8 or \\\n                   attn_mask.dtype == torch.bool, \\\n                   'Only float, byte, and bool types are supported for \\\n                    attn_mask'\n\n            if attn_mask.dtype == torch.uint8:\n                warnings.warn('Byte tensor for attn_mask is deprecated.\\\n                     Use bool tensor instead.')\n                attn_mask = attn_mask.to(torch.bool)\n            if attn_mask.dim() == 2:\n                attn_mask = attn_mask.unsqueeze(0)\n                if list(attn_mask.size()) != [1, query.size(1), key.size(1)]:\n                    raise RuntimeError(\n                        'The size of the 2D attn_mask is not correct.')\n            elif attn_mask.dim() == 3:\n                if list(attn_mask.size()) != [\n                        bs * self.num_heads,\n                        query.size(1),\n                        key.size(1)\n                ]:\n                    raise RuntimeError(\n                        'The size of the 3D attn_mask is not correct.')\n            else:\n                raise RuntimeError(\n                    \"attn_mask's dimension {} is not supported\".format(","sourceCodeStart":546,"sourceCodeEnd":582,"githubUrl":"https://github.com/open-mmlab/mmdetection/blob/cfd5d3a985b0249de009b67d04f37263e11cdf3d/mmdet/models/layers/transformer/utils.py#L546-L582","documentation":"MultiheadAttention warns that a uint8 (byte) attention mask is deprecated and converts it to bool. PyTorch deprecated byte masks because nonzero bytes are interpreted inconsistently; bool masks are the supported format.","triggerScenarios":"Passing attn_mask with dtype torch.uint8 to mmdet's MultiheadAttention forward path (e.g. from custom DETR-style code or legacy checkpoints/pipelines).","commonSituations":"Code written for old PyTorch where byte masks were common; masks created via .byte() instead of .bool().","solutions":["Create masks with dtype torch.bool (mask.to(torch.bool) or direct bool construction)","Ensure padding/key_padding masks come from utils as bool tensors"],"exampleFix":"# before\nattn_mask = (positions == pad).byte()\n# after\nattn_mask = (positions == pad).bool()","handlingStrategy":"type-guard","validationCode":"assert attn_mask.dtype == torch.bool, f'expected bool mask, got {attn_mask.dtype}'","typeGuard":"def is_bool_mask(m):\n    return isinstance(m, torch.Tensor) and m.dtype == torch.bool","tryCatchPattern":null,"preventionTips":["Construct masks with .bool() everywhere","Ban .byte() masks in lint rules for transformer code"],"tags":["mmdet","attention","pytorch","deprecation","dtype"],"backgroundTag":"deprecated-dtype","analyzedSha":"cfd5d3a985b0249de009b67d04f37263e11cdf3d","analyzedAt":"2026-08-27T20:54:20.183Z","schemaVersion":2},"datasetVersion":"2026-08-28T00:17:15.603Z"}