keras-team/keras · error · ValueError

All dimensions of `value` and `key`, except the last one, mu

Error message

All dimensions of `value` and `key`, except the last one, must be equal. Received: value_shape={value_shape} and key_shape={key_shape}

What it means

GroupedQueryAttention.compute_output_shape requires value and key shapes to agree on every dimension except the last one (value_shape[1:-1] == key_shape[1:-1]), because keys and values must be paired per token. A mismatch raises ValueError with both shapes during shape inference, typically at build or first call.

Source

Thrown at keras/src/layers/attention/grouped_query_attention.py:585

    def compute_output_shape(
        self,
        query_shape,
        value_shape,
        key_shape=None,
    ):
        if key_shape is None:
            key_shape = value_shape

        if query_shape[-1] != value_shape[-1]:
            raise ValueError(
                "The last dimension of `query_shape` and `value_shape` "
                f"must be equal, but are {query_shape[-1]}, {value_shape[-1]}. "
                f"Received: query_shape={query_shape}, "
                f"value_shape={value_shape}"
            )

        if value_shape[1:-1] != key_shape[1:-1]:
            raise ValueError(
                "All dimensions of `value` and `key`, except the last one, "
                f"must be equal. Received: value_shape={value_shape} and "
                f"key_shape={key_shape}"
            )

        return query_shape

    def compute_output_spec(
        self,
        query,
        value,
        key=None,
        query_mask=None,
        value_mask=None,
        key_mask=None,
        attention_mask=None,
        return_attention_scores=False,
        training=None,

View on GitHub (pinned to 7a34a03db6)

Solutions

Ensure key and value derive from the same tensor or have identical shapes except the last dimension.
Check for swapped positional args: the Keras attention call signature is (query, value, key); passing (query, key, value) is the classic cause.
Align preprocessing (same padding and truncation) for the K and V streams before the layer.

Example fix

# before
out = attn(query, key_tensor, value_tensor)  # K/V shapes mismatch -> ValueError

# after: Keras convention is call(query, value, key)
out = attn(query, value_tensor, key_tensor)

Defensive patterns

Strategy: validation

Validate before calling

assert list(value_shape)[1:-1] == list(key_shape)[1:-1], f'K/V mismatch: {key_shape} vs {value_shape}'

Type guard

def kv_aligned(value_shape, key_shape) -> bool:
    v, k = list(value_shape), list(key_shape)
    return len(v) == len(k) and v[1:-1] == k[1:-1]

Prevention

Remember the Keras attention call order is (query, value, key), not (query, key, value).
Derive K and V from the same tensor whenever possible.
Share padding and truncation between K and V preprocessing.

When it happens

Trigger: Passing key with a different sequence length or intermediate dims than value, e.g. attn(q, k=Input((5,64)), v=Input((10,64))); building a Functional model where the K and V inputs are declared with different shapes.

Common situations: Hand-built cross-attention where K and V are preprocessed independently (different pooling or windowing); accidental V/K argument swap so one of them receives the query tensor.

Related errors

AI-assisted analysis of keras-team/keras@7a34a03db6 (2026-08-25). Data as JSON: /api/errors/d7e6abd8e20d6ad2. Report an issue: GitHub.