keras-team/keras · error · ValueError

Received an invalid value for argument `num_heads`, expected

Error message

Received an invalid value for argument `num_heads`, expected a positive integer. Received: num_heads={num_heads}

What it means

MultiHeadAttention.__init__ validates num_heads up front: it must be a Python int (bools, floats, numpy scalars and None all fail the checks) and strictly positive, else ValueError. The same validation follows for key_dim and related args. The check exists because the head count splits the feature axis, which is impossible for zero, negative, or non-integer counts.

Source

Thrown at keras/src/layers/attention/multi_head_attention.py:138

        dropout=0.0,
        use_bias=True,
        output_shape=None,
        attention_axes=None,
        sliding_window=None,
        flash_attention=None,
        kernel_initializer="glorot_uniform",
        bias_initializer="zeros",
        kernel_regularizer=None,
        bias_regularizer=None,
        activity_regularizer=None,
        kernel_constraint=None,
        bias_constraint=None,
        use_gate=False,
        seed=None,
        **kwargs,
    ):
        if not isinstance(num_heads, int) or num_heads <= 0:
            raise ValueError(
                "Received an invalid value for argument `num_heads`, "
                f"expected a positive integer. Received: num_heads={num_heads}"
            )
        if not isinstance(key_dim, int) or key_dim <= 0:
            raise ValueError(
                "Received an invalid value for argument `key_dim`, expected "
                f"a positive integer. Received: key_dim={key_dim}"
            )
        if value_dim is not None and (
            not isinstance(value_dim, int) or value_dim <= 0
        ):
            raise ValueError(
                "Received an invalid value for argument `value_dim`, "
                "expected a positive integer or `None`. Received: "
                f"value_dim={value_dim}"
            )
        super().__init__(**kwargs)
        self.supports_masking = True

View on GitHub (pinned to 7a34a03db6)

Solutions

1. Pass a positive Python int: MultiHeadAttention(num_heads=8, key_dim=64).
2. Coerce config values at the boundary: num_heads = int(cfg['heads']) and assert num_heads > 0 before constructing.
3. Guard sweep search spaces to exclude non-positive head counts.

Example fix

# before
mha = MultiHeadAttention(num_heads=0, key_dim=64)  # -> ValueError

# after
heads = int(cfg.get('num_heads', 8))
assert heads > 0
mha = MultiHeadAttention(num_heads=heads, key_dim=64)

h = cfg.get('num_heads')
assert isinstance(h, int) and not isinstance(h, bool) and h > 0, 'num_heads must be a positive int'

def valid_num_heads(n) -> bool:
    return isinstance(n, int) and not isinstance(n, bool) and n > 0

When it happens

Trigger: MultiHeadAttention(num_heads=0), num_heads=-2, num_heads=8.0, num_heads=np.int64(8) (a numpy scalar is not a Python int), or num_heads coming out of a config as None.

Common situations: Hyperparameter sweeps that hit zero; YAML or JSON configs parsed as floats (8.0); numpy ints passed straight from array-based configs; refactors that leave the argument unset.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

• Unknown activation function '{activation}' cannot be seriali
• Could not interpret activation function identifier: {identif
• ConvNeXt does not support the `channels_first` image data fo
• If using `weights="imagenet"` with `include_top=True`, `clas
• The `weights` argument should be either `None` (random initi

AI-assisted analysis of keras-team/keras@7a34a03db6 (2026-08-25). Data as JSON: /api/errors/7fc713e6d743b4a0. Report an issue: GitHub.