{"record":{"id":"7d5848622653c714","repo":"jax-ml/jax","slug":"scaled-matmul-requires-all-inputs-to-be-3-dimensio","errorCode":null,"errorMessage":"scaled_matmul requires all inputs to be 3-dimensional arrays","messagePattern":"scaled_matmul requires all inputs to be 3-dimensional arrays","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"jax/_src/nn/functions.py","lineNumber":1355,"sourceCode":"      >>> a = jnp.array([1, 2, 3]).reshape((1, 1, 3))\n      >>> b = jnp.array([4, 5, 6]).reshape((1, 1, 3))\n      >>> a_scales = jnp.array([0.5]).reshape((1, 1, 1))\n      >>> b_scales = jnp.array([0.5]).reshape((1, 1, 1))\n      >>> scaled_matmul(a, b, a_scales, b_scales)  # doctest: +SKIP\n      Array([[[8.]]], dtype=float32)\n\n      Using fused cuDNN call on Blackwell GPUs:\n\n      >>> dtype = jnp.float8_e4m3fn\n      >>> a = jax.random.normal(jax.random.PRNGKey(1), (3, 128, 64), dtype=dtype)\n      >>> b = jax.random.normal(jax.random.PRNGKey(2), (3, 128, 64), dtype=dtype)\n      >>> a_scales = jnp.ones((3, 128, 4), dtype=jnp.float8_e8m0fnu)\n      >>> b_scales = jnp.ones((3, 128, 4), dtype=jnp.float8_e8m0fnu)\n      >>> scaled_matmul(a, b, a_scales, b_scales)  # doctest: +SKIP\n    \"\"\"\n    a, b, a_scales, b_scales = lhs, rhs, lhs_scales, rhs_scales\n    if not all(x.ndim == 3 for x in (a, b, a_scales, b_scales)):\n        raise ValueError(\n            \"scaled_matmul requires all inputs to be 3-dimensional arrays\"\n        )\n\n    B_a, M_a, K_a = a.shape\n    B_b, N_b, K_b = b.shape\n    if K_a != K_b or B_a != B_b:\n        raise ValueError(\n            \"scaled_matmul requires inputs a and b to have matching batch (B) \"\n            f\"and contract (K) dimensions, but got shapes {a.shape} and \"\n            f\"{b.shape}\"\n        )\n\n    B_as, M_as, K_as = a_scales.shape\n    B_bs, N_bs, K_bs = b_scales.shape\n    if K_as != K_bs or B_as != B_bs:\n        raise ValueError(\n            \"scaled_matmul requires scales to have matching batch (B) and \"\n            f\"contract (K) dimensions, but got shapes {a_scales.shape} and \"","sourceCodeStart":1337,"sourceCodeEnd":1373,"githubUrl":"https://github.com/jax-ml/jax/blob/1e1c6a8fc06dfcd1247076ec5cae4640cea5d7bb/jax/_src/nn/functions.py#L1337-L1373","documentation":"jax.nn.scaled_matmul implements block-scaled (MX-format float8) matrix multiplication and requires lhs, rhs, lhs_scales, rhs_scales all to be exactly 3D (B, M, K) / (B, N, K) blocks. Passing any 2D or 4D tensor raises this.","triggerScenarios":"Passing plain 2D matrices without an outer batch dim, or 4D attention tensors, to scaled_matmul; forgetting the scales arrays or passing scalars.","commonSituations":"Adapting a normal jnp.matmul call to MXFP8 scaled matmul and forgetting to add a batch dimension; passing per-tensor scalar scales instead of 3D block scales.","solutions":["Add a leading batch dimension: a[None], b[None], a_scales[None], b_scales[None]","Reshape 4D tensors to 3D (fold batch*heads) if appropriate","Build scales with the block layout (B, ceil(M/32), ceil(K/32)) as float8_e8m0fnu"],"exampleFix":"// before\nout = jax.nn.scaled_matmul(a2d, b2d, a_s2d, b_s2d)\n\n// after\nout = jax.nn.scaled_matmul(a2d[None], b2d[None], a_s2d[None], b_s2d[None])[0]","handlingStrategy":"validation","validationCode":"assert all(x.ndim == 3 for x in (a, b, a_s, b_s)), 'scaled_matmul needs 3D (B,M,K)/(B,N,K) inputs'\nif a.ndim == 2: a, b, a_s, b_s = (t[None] for t in (a, b, a_s, b_s))","typeGuard":"def is_3d(*ts) -> bool: return all(getattr(t, 'ndim', -1) == 3 for t in ts)","tryCatchPattern":null,"preventionTips":["Always materialize an explicit batch dim before scaled matmul","Write a helper mx_shapes(a,b) that asserts and returns expected scale shapes"],"tags":["jax","nn","matmul","float8","mxfp8","shape-validation"],"backgroundTag":"rank-mismatch","analyzedSha":"1e1c6a8fc06dfcd1247076ec5cae4640cea5d7bb","analyzedAt":"2026-08-27T09:53:25.647Z","schemaVersion":2},"datasetVersion":"2026-08-27T13:17:12.746Z"}