{"record":{"id":"1b038d0c04980f80","repo":"xai-org/x-algorithm","slug":"the-fused-rowwise-adagrad-update-needs-a-row-shard","errorCode":null,"errorMessage":"the fused rowwise Adagrad update needs a row shard that divides a warp, got shard_width={context.shard_width} (emb_width={context.emb_width})","messagePattern":"the fused rowwise Adagrad update needs a row shard that divides a warp, got shard_width=(.+?) \\(emb_width=(.+?)\\)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"phoenix/xrex/optimizers/recsys/rowwise_adagrad.py","lineNumber":106,"sourceCode":"        return embeddings, None\n\n    @property\n    def decay_factor(self) -> float:\n        return math.exp(-self._decay_rate) if self._decay_rate is not None else 1.0\n\n    def gradient_update_start(\n        self,\n        context: async_emb.AsyncEmbContextHandle,\n        update: AsyncEmbGradientUpdate,\n        table: jax.Array,\n        state: RecsysRowwiseAdagradState,\n        gate: jax.Array,\n    ) -> tuple[tuple[jax.Array, ...], jax.Array, RecsysRowwiseAdagradState]:\n        if self._lazy_decay and (state.step is None or state.last_step is None):\n            raise ValueError(\"fused lazy decay needs timestamped state (step/last_step)\")\n\n        if 32 % context.shard_width != 0:\n            raise ValueError(\n                f\"the fused rowwise Adagrad update needs a row shard that divides a warp, \"\n                f\"got shard_width={context.shard_width} (emb_width={context.emb_width})\"\n            )\n\n        from xrex.cuda.async_emb import async_emb\n\n        metrics: dict[str, jax.Array] = {}\n\n        if self._lazy_decay:\n\n            @shard_map(\n                mesh=context.mesh,\n                in_specs=(\n                    P(context.data_axis, None),\n                    P(context.data_axis),\n                    P(),\n                    P(None, context.table_axis),\n                    P(),","sourceCodeStart":88,"sourceCodeEnd":124,"githubUrl":"https://github.com/xai-org/x-algorithm/blob/24c60942c5c5fdad3a6addffb4c6e6d2f228f04f/phoenix/xrex/optimizers/recsys/rowwise_adagrad.py#L88-L124","documentation":"The fused rowwise Adagrad CUDA kernel processes one embedding row across a warp (32 threads), so each row shard must evenly divide 32. This error is raised when shard_width (derived from emb_width/sharding context) does not divide 32, making the warp-cooperative update impossible.","triggerScenarios":"Calling gradient_update_start with an embedding width (or shard width) like 48, 96, or any value where 32 % shard_width != 0, e.g. emb_width producing shard_width=3 or 6.","commonSituations":"Switching an embedding table to a non-power-of-two width (e.g. 96 or 768/7=... odd widths), changing sharding so each shard is narrower, or porting a config from a different optimizer without the warp-divisibility constraint.","solutions":["Change emb_width/sharding so shard_width divides 32 (1, 2, 4, 8, 16, or 32) — e.g. pad emb_width to 64/128/256","Check context.shard_width vs context.emb_width to confirm how many shards the row is split into","If the width cannot change, use a non-fused optimizer path instead of the fused rowwise Adagrad update"],"exampleFix":"# before\nemb = Embedding(num_embeddings=N, emb_width=96)  # shard_width=3 -> 32 % 3 != 0\n\n# after\nemb = Embedding(num_embeddings=N, emb_width=128)  # shard_width=4 -> ok","handlingStrategy":"validation","validationCode":"assert 32 % shard_width == 0, f\"shard_width={shard_width} must divide 32 (warp size)\"","typeGuard":"def warp_compatible(emb_width: int, shards: int) -> bool:\n    return 32 % (emb_width // shards) == 0","tryCatchPattern":null,"preventionTips":["Restrict emb_width to multiples of 32","Validate shard geometry in config parsing before building the optimizer"],"tags":["cuda","warp","shard-width","embedding","adagrad","pallas"],"backgroundTag":"invalid-kernel-configuration","analyzedSha":"24c60942c5c5fdad3a6addffb4c6e6d2f228f04f","analyzedAt":"2026-08-28T11:40:14.686Z","schemaVersion":2},"datasetVersion":"2026-08-28T16:17:29.566Z"}