{"record":{"id":"a1b063201f9a0966","repo":"keras-team/keras","slug":"returning-attention-scores-is-not-supported-when-f","errorCode":null,"errorMessage":"Returning attention scores is not supported when flash attention is enabled. Please disable flash attention to access attention scores.","messagePattern":"Returning attention scores is not supported when flash attention is enabled\\. Please disable flash attention to access attention scores\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"keras/src/layers/attention/grouped_query_attention.py","lineNumber":466,"sourceCode":"            ops.arange(q_seq_length, dtype=\"int32\"), (1, q_seq_length, 1)\n        )\n        col_index = ops.reshape(\n            ops.arange(v_seq_length, dtype=\"int32\"), (1, 1, v_seq_length)\n        )\n        return ops.less(ops.abs(row_index - col_index), self.sliding_window)\n\n    def _compute_attention(\n        self,\n        query,\n        key,\n        value,\n        attention_mask=None,\n        training=None,\n        use_causal_mask=False,\n    ):\n        # Check for flash attention constraints\n        if self._flash_attention and self._return_attention_scores:\n            raise ValueError(\n                \"Returning attention scores is not supported when flash \"\n                \"attention is enabled. Please disable flash attention to access\"\n                \" attention scores.\"\n            )\n\n        # Determine whether to use dot-product attention\n        use_dot_product_attention = not (\n            self.dropout > 0.0\n            or self._return_attention_scores\n            or (len(query.shape) != 4)\n        )\n\n        if use_dot_product_attention:\n            if use_causal_mask and attention_mask is None:\n                # Skip materializing the [T, S] mask and let the backend\n                # use its native causal kernel.\n                attention_output = ops.dot_product_attention(\n                    query=query,","sourceCodeStart":448,"sourceCodeEnd":484,"githubUrl":"https://github.com/keras-team/keras/blob/7a34a03db60bf60042242d6a556fc3be119046a5/keras/src/layers/attention/grouped_query_attention.py#L448-L484","documentation":"GroupedQueryAttention can use a fused flash-attention kernel, but flash attention does not materialize attention score matrices, so the layer refuses the combination of flash_attention enabled together with return_attention_scores=True. The check runs at the top of _compute_attention during call(), so the error appears on the first forward pass, not at construction.","triggerScenarios":"GroupedQueryAttention(..., flash_attention=True, return_attention_scores=True) then calling the layer; flash attention enabled by default in some configurations while interpretability code sets return_attention_scores=True.","commonSituations":"Interpretability code that inspects attention weights on a layer configured for speed; toggling flash_attention on for training while an attention-visualization path still exists.","solutions":["Disable flash attention on the layer: GroupedQueryAttention(..., flash_attention=False, return_attention_scores=True).","Or keep flash attention and drop return_attention_scores; scores are unavailable by design.","For visualization, run a separate cheap forward pass with a non-flash copy of the layer."],"exampleFix":"# before\nattn = GroupedQueryAttention(head_dim=64, flash_attention=True, return_attention_scores=True)\nout, scores = attn(x)  # -> ValueError\n\n# after\nattn = GroupedQueryAttention(head_dim=64, flash_attention=False, return_attention_scores=True)\nout, scores = attn(x)","handlingStrategy":"validation","validationCode":"if getattr(layer, '_flash_attention', False) and need_scores:\n    layer._flash_attention = False  # or rebuild the layer with flash_attention=False","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Decide up front: speed (flash) or interpretability (scores), not both.","Gate score-returning code paths on a config flag mirrored into the layer constructor.","Add a unit test asserting the layer builds and calls with your flag combination."],"tags":["keras","attention","flash-attention","grouped-query-attention","incompatibility"],"backgroundTag":"incompatible-feature-combination","analyzedSha":"7a34a03db60bf60042242d6a556fc3be119046a5","analyzedAt":"2026-08-25T21:25:25.994Z","schemaVersion":2},"datasetVersion":"2026-08-26T02:17:13.382Z"}