What causes "Inputs must be of rank 3 or 4." in google-research/timesfm?

Ensure inputs is 3-D (batch, seq, dim) or 4-D (batch, heads, seq, dim).

google-research/timesfm · error · ValueError

Inputs must be of rank 3 or 4.

Error message

Inputs must be of rank 3 or 4.

What it means

The sinusoid position embedding supports 4-D inputs (batch, heads, seq, dim) and 3-D inputs (batch, seq, dim); any other rank has ambiguous broadcasting semantics for position and timescale, so forward() raises ValueError. Rank determines how position/timescale are broadcast.

Source

Thrown at src/timesfm/torch/transformer.py:103

      / self.embedding_dims
    )
    timescale = (
      self.min_timescale * (self.max_timescale / self.min_timescale) ** fraction
    ).to(inputs.device)
    if position is None:
      seq_length = inputs.shape[1]
      position = torch.arange(seq_length, dtype=torch.float32, device=inputs.device)[
        None, :
      ]

    if len(inputs.shape) == 4:
      position = position[..., None, None]
      timescale = timescale[None, None, None, :]
    elif len(inputs.shape) == 3:
      position = position[..., None]
      timescale = timescale[None, None, :]
    else:
      raise ValueError("Inputs must be of rank 3 or 4.")

    sinusoid_inp = position / timescale
    sin = torch.sin(sinusoid_inp)
    cos = torch.cos(sinusoid_inp)
    first_half, second_half = torch.chunk(inputs, 2, dim=-1)
    first_part = first_half * cos - second_half * sin
    second_part = second_half * cos + first_half * sin
    return torch.cat([first_part, second_part], dim=-1)


def _dot_product_attention(
  query,
  key,
  value,
  mask=None,
):
  """Computes dot-product attention given query, key, and value."""
  attn_weights = torch.einsum("...qhd,...khd->...hqk", query, key)

View on GitHub (pinned to 331c6d33cb)

Solutions

Ensure inputs is 3-D (batch, seq, dim) or 4-D (batch, heads, seq, dim).
Add a batch dim if missing: inputs = inputs[None, ...].
Remove unintended leading dims: inputs = inputs.squeeze(0) or flatten extra axes.
Pass a position tensor whose shape broadcasts against the input rank.

Example fix

// before
pos = sinusoid_position_embedding(inputs)  # inputs is (seq, dim), rank 2
// after
pos = sinusoid_position_embedding(inputs[None, ...])  # add batch dim -> rank 3

Defensive patterns

Strategy: validation

Validate before calling

if inputs.ndim not in (3, 4):
    if inputs.ndim == 2:
        inputs = inputs[None, ...]  # add batch dim
    else:
        raise ValueError(f"expected rank 3 or 4, got {inputs.ndim}")

Type guard

def is_embedding_input(x) -> bool:
    return x.ndim in (3, 4)

Try / catch

try:
    out = pos_emb(inputs)
except ValueError:
    inputs = inputs[None, ...]  # add missing batch dim
    out = pos_emb(inputs)

Prevention

Never squeeze the batch dimension before position embeddings.
Keep tensors at rank 3 (b,t,d) or 4 (b,h,t,d) through the embedding.
Squeeze extra dims only after the embedding layer.
Add ndim asserts in custom call paths.

When it happens

Trigger: Calling the position-embedding forward with a tensor of ndim other than 3 or 4 — e.g. a batch-less (seq, dim) 2-D tensor, a flattened 1-D tensor, or a 5-D tensor from an extra leading dimension.

Common situations: Unit-testing the embedding module with a hand-made tensor lacking a batch dim; squeezing a batch of size 1 before the layer; stacking an extra ensemble/data axis upstream.

Related errors

AI-assisted analysis of google-research/timesfm@331c6d33cb (2026-08-29). Data as JSON: /api/errors/fd0f4f5e612122d6. Report an issue: GitHub.