google-research/timesfm · error · ValueError

Incompatible input dimension, got {input_in_features} but mo

Error message

Incompatible input dimension, got {input_in_features} but module expects {self.in_features}.

What it means

MultiHeadAttention.__call__ unpacks inputs_q.shape and verifies the last (feature) axis equals the in_features the projection layers were built with. A mismatch means the input tensor's hidden size differs from what the attention module was initialized for.

Source

Thrown at src/timesfm/flax/transformer.py:219

    if use_per_dim_scale:
      self.per_dim_scale = PerDimScale(num_dims=self.head_dim, rngs=rngs)
    else:
      self.per_dim_scale = None

  def __call__(
    self,
    inputs_q: Array,
    *,
    decode_cache: DecodeCache | None = None,
    patch_mask: Array | None = None,
    deterministic: bool | None = None,
    sow_weights: bool = False,
  ) -> tuple[Float[Array, "b ... o"], DecodeCache | None]:
    """Applies multi-head dot product attention on the input data."""
    _, n_patches, input_in_features = inputs_q.shape
    if input_in_features != self.in_features:
      raise ValueError(
        f"Incompatible input dimension, got {input_in_features} "
        f"but module expects {self.in_features}."
      )
    if patch_mask is None:
      patch_mask = jnp.zeros_like(inputs_q.shape[:-1], dtype=jnp.bool)

    # For query: rope -> ln -> per_dim_scale
    query = self.query(inputs_q)
    key = self.key(inputs_q)
    value = self.value(inputs_q)

    if decode_cache is None:
      num_masked = jnp.sum(patch_mask.astype(jnp.int32), axis=-1, keepdims=False)
      next_index = jnp.zeros_like(num_masked, dtype=jnp.int32)
    else:
      num_masked = (
        jnp.sum(patch_mask.astype(jnp.int32), axis=-1, keepdims=False)
        + decode_cache.num_masked

View on GitHub (pinned to 331c6d33cb)

Solutions

1. Make the input's last dimension match in_features (check the tensor just before the attention call).
2. Rebuild the attention module (and config) with in_features equal to the actual input width if the new size is intended.
3. Verify the checkpoint and config model_dims agree when loading pretrained weights.

Example fix

// before
attn = MultiHeadAttention(in_features=128, ...)
y = attn(x)  # x.shape[-1] == 64 -> ValueError
// after
x = nnx.Linear(64, 128)(x)  # project to 128 first
y = attn(x)

assert inputs.shape[-1] == attn.in_features, (
    f"input feature dim {inputs.shape[-1]} != attention in_features {attn.in_features}")

def fits_attention(attn, inputs) -> bool:
    return inputs.ndim >= 3 and inputs.shape[-1] == attn.in_features

try:
    out = attn(x)
except ValueError as e:
    if "Incompatible input dimension" in str(e):
        x = project_to(x, attn.in_features)  # add/fix projection layer
        out = attn(x)
    else:
        raise

When it happens

Trigger: Feeding a tensor whose last dimension differs from self.in_features into attention __call__ — e.g. passing patch embeddings of the wrong width, changing model_dims after layers were constructed, or wiring an intermediate tensor into the wrong attention layer.

Common situations: Pipeline wiring mistakes (feeding the wrong tensor), checkpoint/config mismatch where weights expect a different hidden size, or reshaping errors upstream that alter the feature axis.

Related errors

• Output dims must be a multiple of 4: {config.output_dims} %
• The embedding dims of the rotary position embeddingmust matc
• Inputs must be of rank 3 or 4.
• Memory dimension ({self.qkv_features}) must be divisible by
• Memory dimension ({self.in_features}) must be divisible by '

AI-assisted analysis of google-research/timesfm@331c6d33cb (2026-08-29). Data as JSON: /api/errors/c6ca544dab9e201c. Report an issue: GitHub.