{"record":{"id":"1c304903ab464bed","repo":"xai-org/x-algorithm","slug":"rmsnorm-when-weight-decay-mask-0-the-layer-is-r","errorCode":null,"errorMessage":"RMSNorm: when weight_decay_mask > 0 the layer is re-parameterized as (1 + scale) * x and `scale_init` must be jnp.zeros (got {scale_init!r}). Pass scale_init=jnp.zeros or leave it as None to use the default.","messagePattern":"RMSNorm: when weight_decay_mask > 0 the layer is re-parameterized as \\(1 \\+ scale\\) \\* x and `scale_init` must be jnp\\.zeros \\(got (.+?)\\)\\. Pass scale_init=jnp\\.zeros or leave it as None to use the default\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"phoenix/xrex/models/normalization.py","lineNumber":87,"sourceCode":"        weight_decay_mask: float = 0.0,\n    ):\n        super().__init__(name=name)\n        if isinstance(axis, slice):\n            self.axis = axis\n        elif isinstance(axis, int):\n            self.axis = (axis,)\n        elif isinstance(axis, abc.Iterable) and all(isinstance(ax, int) for ax in axis):\n            self.axis = tuple(axis)\n        else:\n            raise ValueError(\"`axis` should be an int, slice or iterable of ints.\")\n\n        self.eps = eps\n        self.create_scale = create_scale\n        self.reparameterize = weight_decay_mask > 0\n        if scale_init is None:\n            scale_init = jnp.zeros if self.reparameterize else jnp.ones\n        if self.reparameterize and scale_init is not jnp.zeros:\n            raise ValueError(\n                \"RMSNorm: when weight_decay_mask > 0 the layer is \"\n                \"re-parameterized as (1 + scale) * x and `scale_init` must be \"\n                f\"jnp.zeros (got {scale_init!r}). Pass scale_init=jnp.zeros or \"\n                \"leave it as None to use the default.\"\n            )\n        self.scale_init = scale_init\n        self.pspec = pspec\n        self.lr_multiplier = lr_multiplier\n        self.weight_decay_mask = weight_decay_mask\n\n    def __call__(self, inputs: jax.Array):\n        fprop_dtype = inputs.dtype\n        param_shape = (inputs.shape[-1],)\n        if self.create_scale:\n            scale = get_parameter(\n                \"scale\",\n                param_shape,\n                dtype=jnp.float32,","sourceCodeStart":69,"sourceCodeEnd":105,"githubUrl":"https://github.com/xai-org/x-algorithm/blob/24c60942c5c5fdad3a6addffb4c6e6d2f228f04f/phoenix/xrex/models/normalization.py#L69-L105","documentation":"When weight_decay_mask > 0, RMSNorm re-parameterizes the scale as (1 + scale) * x, which requires the scale to start at zero; hence scale_init must be exactly jnp.zeros (or None to get that default). Passing any other initializer (e.g. jnp.ones) breaks the re-parameterization invariant and is rejected.","triggerScenarios":"Constructing the norm with weight_decay_mask > 0 while explicitly passing scale_init=jnp.ones or a custom initializer; copying constructor args from a non-reparameterized layer.","commonSituations":"Enabling weight decay on norms mid-project and reusing old constructor kwargs; defaults changed in shared builder code.","solutions":["Drop the scale_init argument (let it default to jnp.zeros when reparameterizing).","Or explicitly pass scale_init=jnp.zeros.","If you need nonzero init scale, set weight_decay_mask=0 for that layer."],"exampleFix":"# before\nRMSNorm(..., weight_decay_mask=1, scale_init=jnp.ones)\n\n# after\nRMSNorm(..., weight_decay_mask=1)  # scale_init defaults to jnp.zeros","handlingStrategy":"validation","validationCode":"if weight_decay_mask > 0:\n    scale_init = None  # force default jnp.zeros reparameterization","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Never pass scale_init when reparameterization is on; let the default apply.","Encapsulate norm construction in one builder that enforces the constraint."],"tags":["normalization","weight-decay","initializer","config-constraint"],"backgroundTag":"invalid-config-combination","analyzedSha":"24c60942c5c5fdad3a6addffb4c6e6d2f228f04f","analyzedAt":"2026-08-28T11:40:14.686Z","schemaVersion":2},"datasetVersion":"2026-08-28T16:17:29.566Z"}