{"record":{"id":"08d44819ecbf054a","repo":"jax-ml/jax","slug":"orthogonal-initializer-requires-at-least-a-2d-shap","errorCode":null,"errorMessage":"orthogonal initializer requires at least a 2D shape","messagePattern":"orthogonal initializer requires at least a 2D shape","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"jax/_src/nn/initializers.py","lineNumber":632,"sourceCode":"    An orthogonal initializer.\n\n  Examples:\n\n  >>> import jax, jax.numpy as jnp\n  >>> initializer = jax.nn.initializers.orthogonal()\n  >>> initializer(jax.random.key(42), (2, 3), jnp.float32)  # doctest: +SKIP\n  Array([[ 3.9026976e-01,  7.2495741e-01, -5.6756169e-01],\n         [ 8.8047469e-01, -4.7409311e-01, -1.3157725e-04]],            dtype=float32)\n  \"\"\"\n  def init(key: Array,\n           shape: core.Shape,\n           dtype: DTypeLikeInexact | None = dtype,\n           out_sharding: OutShardingType = None) -> Array:\n    if out_sharding is not None:\n      raise NotImplementedError\n    dtype = dtypes.default_float_dtype() if dtype is None else dtype\n    if len(shape) < 2:\n      raise ValueError(\"orthogonal initializer requires at least a 2D shape\")\n\n    if any(dim == 0 for dim in shape):\n      # empty shape\n      return jnp.zeros(shape, dtype=dtype, out_sharding=out_sharding)\n\n    n_rows, n_cols = math.prod(shape) // shape[column_axis], shape[column_axis]\n    Q = random.orthogonal(key, n_rows, (), dtype, n_cols)\n    Q = jnp.reshape(Q, tuple(np.delete(shape, column_axis)) + (shape[column_axis],))\n    Q = jnp.moveaxis(Q, -1, column_axis)\n    return jnp.array(scale, dtype) * Q\n  return init\n\n@export\ndef delta_orthogonal(\n  scale: RealNumeric = 1.0,\n  column_axis: int = -1,\n  dtype: DTypeLikeInexact | None = None) -> Initializer:\n  \"\"\"","sourceCodeStart":614,"sourceCodeEnd":650,"githubUrl":"https://github.com/jax-ml/jax/blob/1e1c6a8fc06dfcd1247076ec5cae4640cea5d7bb/jax/_src/nn/initializers.py#L614-L650","documentation":"jax.nn.initializers.orthogonal produces a (semi-)orthogonal matrix via QR/SVD, which requires at least 2 dimensions. Passing a 0D or 1D shape raises this ValueError.","triggerScenarios":"Calling orthogonal()(key, ()) or orthogonal()(key, (128,)); transposing a Linear layer's shape incorrectly so only one dim reaches the initializer.","commonSituations":"Initializing biases or embeddings-of-rank-1 with orthogonal; generic layer code reusing one initializer for both weights (2D) and biases (1D).","solutions":["Use delta_initializer() or zeros for 1D parameters","Reshape 1D params to (n, 1) only if mathematically appropriate (a single orthogonal column is just a unit vector)"],"exampleFix":"// before\nb = jax.nn.initializers.orthogonal()(key, (128,))\n\n// after\nb = jax.nn.initializers.zeros(key, (128,))","handlingStrategy":"type-guard","validationCode":"def pick_initializer(shape):\n    if len(shape) < 2:\n        return jax.nn.initializers.zeros  # or normal\n    return jax.nn.initializers.orthogonal()","typeGuard":"def supports_orthogonal(shape) -> bool: return len(shape) >= 2","tryCatchPattern":null,"preventionTips":["Branch initializer choice on parameter rank in generic layer code","Never reuse matrix-only initializers for bias/vector params"],"tags":["jax","nn","initializers","shape-validation"],"backgroundTag":"rank-mismatch","analyzedSha":"1e1c6a8fc06dfcd1247076ec5cae4640cea5d7bb","analyzedAt":"2026-08-27T09:53:25.647Z","schemaVersion":2},"datasetVersion":"2026-08-27T13:17:12.746Z"}