{"record":{"id":"366510463ac70bf2","repo":"jax-ml/jax","slug":"fan-in-must-be-less-or-equal-than-fan-out","errorCode":null,"errorMessage":"`fan_in` must be less or equal than `fan_out`. ","messagePattern":"`fan_in` must be less or equal than `fan_out`\\. ","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"jax/_src/nn/initializers.py","lineNumber":693,"sourceCode":"         [[ 0.        ,  0.        ,  0.        ],\n          [ 0.        ,  0.        ,  0.        ],\n          [ 0.        ,  0.        ,  0.        ]]], dtype=float32)\n\n\n  .. _delta orthogonal initializer: https://arxiv.org/abs/1806.05393\n  \"\"\"\n  def init(key: Array,\n           shape: core.Shape,\n           dtype: DTypeLikeInexact | None = dtype,\n           out_sharding: OutShardingType = None) -> Array:\n    if out_sharding is not None:\n      raise NotImplementedError\n    dtype = dtypes.default_float_dtype() if dtype is None else dtype\n    if len(shape) not in [3, 4, 5]:\n      raise ValueError(\"Delta orthogonal initializer requires a 3D, 4D or 5D \"\n                       \"shape.\")\n    if shape[-1] < shape[-2]:\n      raise ValueError(\"`fan_in` must be less or equal than `fan_out`. \")\n    ortho_init = orthogonal(scale=scale, column_axis=column_axis, dtype=dtype)\n    ortho_matrix = ortho_init(key, shape[-2:])\n    W = jnp.zeros(shape, dtype=dtype)\n    if len(shape) == 3:\n      k = shape[0]\n      return W.at[(k-1)//2, ...].set(ortho_matrix)\n    elif len(shape) == 4:\n      k1, k2 = shape[:2]\n      return W.at[(k1-1)//2, (k2-1)//2, ...].set(ortho_matrix)\n    else:\n      k1, k2, k3 = shape[:3]\n      return W.at[(k1-1)//2, (k2-1)//2, (k3-1)//2, ...].set(ortho_matrix)\n  return init\n","sourceCodeStart":675,"sourceCodeEnd":707,"githubUrl":"https://github.com/jax-ml/jax/blob/1e1c6a8fc06dfcd1247076ec5cae4640cea5d7bb/jax/_src/nn/initializers.py#L675-L707","documentation":"delta_orthogonal builds an orthogonal matrix of shape (fan_in, fan_out) = shape[-2:] and requires fan_in <= fan_out (JAX convention: columns are outputs). If the last dimension is smaller than the second-to-last, the QR-based construction cannot be centered in the kernel as required.","triggerScenarios":"Calling delta_orthogonal()(key, shape) where shape[-1] < shape[-2], e.g. (3, 64, 32) for a channel-increasing convolution.","commonSituations":"Conv layers that reduce channels (e.g. 64 input channels to 32 output channels), or using a different fan_in/fan_out axis convention than JAX expects.","solutions":["Transpose the last two dims of your kernel layout so fan_out >= fan_in (swap channel order in the layer definition)","Use orthogonal() or a variance-scaling initializer for channel-reducing layers","If channel reduction is required, initialize on the transposed shape and transpose the result back"],"exampleFix":"// before\nw = jax.nn.initializers.delta_orthogonal()(key, (3, 64, 32))  # fan_out < fan_in\n// after\nw = jax.nn.initializers.delta_orthogonal()(key, (3, 32, 64)).transpose(0, 2, 1)","handlingStrategy":"validation","validationCode":"def delta_ortho_checked(init, key, shape):\n    assert shape[-1] >= shape[-2], 'fan_out (shape[-1]) must be >= fan_in (shape[-2])'\n    return init(key, shape)","typeGuard":"null","tryCatchPattern":"null","preventionTips":["Remember JAX kernel convention: last dim is fan_out","Transpose kernel layout for channel-reducing convs"],"tags":["jax","initializer","shape-validation","fan-in-out"],"backgroundTag":"invalid-shape-argument","analyzedSha":"1e1c6a8fc06dfcd1247076ec5cae4640cea5d7bb","analyzedAt":"2026-08-27T09:53:25.647Z","schemaVersion":2},"datasetVersion":"2026-08-27T13:17:12.746Z"}