{"record":{"id":"f7c8109c49485778","repo":"keras-team/keras","slug":"the-last-dimension-of-query-shape-and-value-sha","errorCode":null,"errorMessage":"The last dimension of `query_shape` and `value_shape` must be equal, but are {query_shape[-1]}, {value_shape[-1]}. Received: query_shape={query_shape}, value_shape={value_shape}","messagePattern":"The last dimension of `query_shape` and `value_shape` must be equal, but are (.+?), (.+?)\\. Received: query_shape=(.+?), value_shape=(.+?)","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"keras/src/layers/attention/grouped_query_attention.py","lineNumber":577,"sourceCode":"            # key_attention_dims>)\n            mask_expansion_axis = -1 * 2 - 1\n            for _ in range(len(scores.shape) - len(attention_mask.shape)):\n                attention_mask = ops.expand_dims(\n                    attention_mask, axis=mask_expansion_axis\n                )\n        return self._softmax(scores, mask=attention_mask)\n\n    def compute_output_shape(\n        self,\n        query_shape,\n        value_shape,\n        key_shape=None,\n    ):\n        if key_shape is None:\n            key_shape = value_shape\n\n        if query_shape[-1] != value_shape[-1]:\n            raise ValueError(\n                \"The last dimension of `query_shape` and `value_shape` \"\n                f\"must be equal, but are {query_shape[-1]}, {value_shape[-1]}. \"\n                f\"Received: query_shape={query_shape}, \"\n                f\"value_shape={value_shape}\"\n            )\n\n        if value_shape[1:-1] != key_shape[1:-1]:\n            raise ValueError(\n                \"All dimensions of `value` and `key`, except the last one, \"\n                f\"must be equal. Received: value_shape={value_shape} and \"\n                f\"key_shape={key_shape}\"\n            )\n\n        return query_shape\n\n    def compute_output_spec(\n        self,\n        query,","sourceCodeStart":559,"sourceCodeEnd":595,"githubUrl":"https://github.com/keras-team/keras/blob/7a34a03db60bf60042242d6a556fc3be119046a5/keras/src/layers/attention/grouped_query_attention.py#L559-L595","documentation":"In GroupedQueryAttention.compute_output_shape, the model dimension of the query must equal that of the value (query_shape[-1] == value_shape[-1]); otherwise output projection shapes cannot be computed and a ValueError with both shapes is raised. The check fires during shape inference (called via compute_output_spec), typically at build time, first call, or when the layer is used inside a Functional model.","triggerScenarios":"Passing query of width 128 and value of width 256, e.g. q=Input((10,128)) and v=Input((10,256)) into GroupedQueryAttention; cross-attention where key/value come from an encoder with a different model dim and no projection to match it.","commonSituations":"Cross-attention between models of different widths (e.g. querying a 768-dim encoder with a 512-dim decoder) without a projection; config mistakes where query_shape or value_shape tuples are swapped or a dim is wrong.","solutions":["Project the query (or value) to a common width first, e.g. Dense(model_dim) on the smaller side before the attention layer.","Fix the inputs so query_shape[-1] equals value_shape[-1]; both encode the same model dimension.","Check for swapped or misspelled shape tuples when constructing the layer programmatically."],"exampleFix":"# before\nq = keras.Input((10, 128)); v = keras.Input((10, 256))\nout = GroupedQueryAttention(head_dim=64)(q, v, v)  # -> ValueError: 128 vs 256\n\n# after\nproj = keras.layers.Dense(256)\nout = GroupedQueryAttention(head_dim=64)(proj(q), v, v)","handlingStrategy":"validation","validationCode":"q, v = list(query_shape), list(value_shape)\nassert q[-1] == v[-1], f'query/value model dims differ: {q[-1]} vs {v[-1]}'","typeGuard":"def dims_compatible(q_shape, v_shape) -> bool:\n    q, v = list(q_shape), list(v_shape)\n    return len(q) == len(v) and q[-1] == v[-1]","tryCatchPattern":null,"preventionTips":["Keep one model_dim constant and use it for all Q/K/V producers.","Insert Dense(model_dim) projections before cross-attention between different-width models.","Assert shape contracts during config validation, before model build."],"tags":["keras","attention","shape-mismatch","validation","grouped-query-attention"],"backgroundTag":"tensor-shape-mismatch","analyzedSha":"7a34a03db60bf60042242d6a556fc3be119046a5","analyzedAt":"2026-08-25T21:25:25.994Z","schemaVersion":2},"datasetVersion":"2026-08-26T02:17:13.382Z"}