{"record":{"id":"8666324969623bd0","repo":"sgl-project/sglang","slug":"t5-attention-bias-with-bucketed-positions-is-not-y","errorCode":null,"errorMessage":"T5 attention bias with bucketed positions is not yet tested","messagePattern":"T5 attention bias with bucketed positions is not yet tested","errorType":"validation","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/models/phi4mm_utils.py","lineNumber":732,"sourceCode":"            the maximum distance for logarithmic bucketing after which all\n            positions are in the same bucket.\n        symmetric: bool\n            Whether to use symmetric or asymmetric biases. symmetric=False uses\n            2x number of bias params to distinguish L->R from R->L. This was\n            found to be better for the encoder.\n    \"\"\"\n\n    def __init__(self, num_heads, num_buckets=-1, max_distance=1000, symmetric=False):\n        super().__init__()\n        self.num_heads = num_heads\n        self.num_buckets = num_buckets\n        self.max_distance = max_distance\n        self.symmetric = symmetric\n        self._skip_bucketing = self.num_buckets < 0\n        if self._skip_bucketing:\n            self.num_buckets = max_distance\n        else:\n            raise NotImplementedError(\n                \"T5 attention bias with bucketed positions is not yet tested\"\n            )\n        if not self.symmetric:\n            self.num_buckets *= 2\n        self.bias_values = nn.Embedding(self.num_buckets, self.num_heads)\n\n    def forward(self, x):\n        # instantiate bias compatible with shape of x\n        maxpos = x.size(1)\n        context_position = torch.arange(maxpos, device=x.device, dtype=torch.long)[\n            :, None\n        ]\n        memory_position = torch.arange(maxpos, device=x.device, dtype=torch.long)[\n            None, :\n        ]\n        relative_position = memory_position - context_position\n        # clipping to a maximum distance using ops that play well with ONNX\n        # export","sourceCodeStart":714,"sourceCodeEnd":750,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/models/phi4mm_utils.py#L714-L750","documentation":"This T5-style relative attention bias module only supports the non-bucketed setting (num_buckets < 0), in which case num_buckets defaults to max_distance. If a positive num_buckets is passed (bucketed relative positions), __init__ raises NotImplementedError because that path is untested in this port.","triggerScenarios":"Instantiating the relative position embedding with an explicit num_buckets >= 0, i.e. requesting classic T5 bucketed relative position bias.","commonSituations":"Copying T5/NeMo attention bias parameters (T5 uses num_buckets=32, max_distance=128) into a Phi-4-MM audio conformer config; porting models that rely on bucketing.","solutions":["Set num_buckets to -1 (or omit it) so bucketing is skipped and num_buckets=max_distance is used","If bucketing is required, port the tested bucketing logic from NeMo's RelativePositionEmbedding before using it"],"exampleFix":"# before\nRelativePositionEmbedding(num_buckets=32, max_distance=128, num_heads=h)  # NotImplementedError\n# after\nRelativePositionEmbedding(num_buckets=-1, max_distance=128, num_heads=h)","handlingStrategy":"validation","validationCode":"assert num_buckets < 0, \"bucketed T5 relative position bias is not implemented; use num_buckets=-1\"","typeGuard":"def supports_bucketing(num_buckets: int) -> bool:\n    return num_buckets < 0  # only non-bucketed path is implemented","tryCatchPattern":null,"preventionTips":["Never copy T5 defaults (num_buckets=32) into this module","Omit num_buckets so it defaults to the skip-bucketing sentinel"],"tags":["phi4","t5","attention-bias","not-implemented"],"backgroundTag":"unsupported-model-configuration","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}