{"record":{"id":"d6556f5e7b68050c","repo":"xai-org/x-algorithm","slug":"emb-size-was-divided-into-output-vocab-size","errorCode":null,"errorMessage":"{emb_size=} was divided into {output_vocab_size=} equal parts of length {D_per_V} each. But received an input with {V=}.","messagePattern":"(.+?) was divided into (.+?) equal parts of length (.+?) each\\. But received an input with (.+?)\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"phoenix/xrex/models/recsys_model.py","lineNumber":1909,"sourceCode":"        embedding_table = get_parameter(\n            name,\n            shape=[\n                1,\n                emb_size,\n            ],\n            init=embed_init,\n            dtype=jnp.float32,\n            pspec=P(),\n            rms_clip_axes=(-2, -1),\n        )\n        table_reshaped = embedding_table.reshape(output_vocab_size, emb_size // output_vocab_size)\n\n        B, S, V = input.shape\n        D = emb_size\n        D_per_V = table_reshaped.shape[1]\n\n        if V * D_per_V != D:\n            raise ValueError(\n                f\"{emb_size=} was divided into {output_vocab_size=} equal parts of length {D_per_V} each. But received an input with {V=}.\"\n            )\n\n        input_reshaped = (2 * input - 1)[:, :, :, None]\n        table_reshaped = table_reshaped[None, None, :, :]\n\n        selected_embeddings = input_reshaped * table_reshaped\n\n        output = selected_embeddings.reshape(B, S, D)\n\n        mask = jnp.any(input, axis=-1)\n        output = output * mask[..., None]\n        output = output.astype(DTYPE_BY_NAME[self.config.fprop_dtype])\n\n        return output, embedding_table\n\n    @hk.transparent\n    def single_hot_to_embeddings(","sourceCodeStart":1891,"sourceCodeEnd":1927,"githubUrl":"https://github.com/xai-org/x-algorithm/blob/24c60942c5c5fdad3a6addffb4c6e6d2f228f04f/phoenix/xrex/models/recsys_model.py#L1891-L1927","documentation":"multi_hot_to_embeddings splits the embedding table row of size emb_size into output_vocab_size equal chunks (D_per_V each) and requires the incoming multi-hot input's vocab dimension V to satisfy V * D_per_V == D (the emb_size). A mismatch means the input vocabulary size differs from the one the table was reshaped for, and the error reports all four numbers.","triggerScenarios":"Feeding a [B, S, V] multi-hot tensor whose V differs from output_vocab_size used when the table was reshaped; changing the vocab size in data preprocessing without regenerating the embedding table; emb_size not evenly divisible by output_vocab_size plus a stale V.","commonSituations":"Vocabulary grown/shrunk between training and serving; using an old checkpoint's embeddings with a new tokenizer/vocab; mismatched output_vocab_size config vs. input pipeline.","solutions":["Align V in the input batch with output_vocab_size (re-tokenize / re-vocab the data).","Update output_vocab_size in the config to the actual input V.","Ensure emb_size is divisible by output_vocab_size and rebuild the table if either changed."],"exampleFix":"# before\noutput_vocab_size = 1000   # input has V=1200\n\n# after\noutput_vocab_size = 1200     # matches input.shape[-1]","handlingStrategy":"validation","validationCode":"B, S, V = multi_hot.shape\nassert V == output_vocab_size and emb_size % output_vocab_size == 0, (V, output_vocab_size, emb_size)","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Pin vocab size in one config shared by data pipeline and model.","Assert V against output_vocab_size in data-loader unit tests."],"tags":["embeddings","vocab-size","shape-validation","multi-hot"],"backgroundTag":"tensor-shape-mismatch","analyzedSha":"24c60942c5c5fdad3a6addffb4c6e6d2f228f04f","analyzedAt":"2026-08-28T11:40:14.686Z","schemaVersion":2},"datasetVersion":"2026-08-28T16:17:29.566Z"}