{"record":{"id":"3bed9985b8edf376","repo":"keras-team/keras","slug":"awq-quantization-only-supports-2d-or-3d-kernels","errorCode":null,"errorMessage":"AWQ quantization only supports 2D or 3D kernels.","messagePattern":"AWQ quantization only supports 2D or 3D kernels\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"keras/src/layers/core/einsum_dense.py","lineNumber":817,"sourceCode":"            shape = list(self.original_kernel_shape)\n            d_model_dim_index = shape.index(max(shape))\n\n            if d_model_dim_index == 0:  # QKV projection case\n                in_features, heads, head_dim = shape\n                rows, columns = (\n                    in_features,\n                    heads * head_dim,\n                )\n            elif d_model_dim_index in [1, 2]:  # Attention Output case\n                heads, head_dim, out_features = shape\n                rows, columns = (\n                    heads * head_dim,\n                    out_features,\n                )\n            else:\n                raise ValueError(\"Could not determine row/column split.\")\n        else:\n            raise ValueError(\"AWQ quantization only supports 2D or 3D kernels.\")\n\n        group_size = awq_core.get_group_size_for_layer(self, config)\n        num_groups = 1 if group_size == -1 else math.ceil(rows / group_size)\n\n        self.awq_unpacked_column_size = columns\n\n        # For 4-bit weights, we pack two values per byte.\n        kernel_columns = (columns + 1) // 2\n\n        self._set_quantization_info()\n\n        self.quantized_kernel = self.add_weight(\n            name=\"kernel\",\n            shape=(kernel_columns, rows),\n            initializer=\"zeros\",\n            dtype=\"uint8\",\n            trainable=False,\n        )","sourceCodeStart":799,"sourceCodeEnd":835,"githubUrl":"https://github.com/keras-team/keras/blob/7a34a03db60bf60042242d6a556fc3be119046a5/keras/src/layers/core/einsum_dense.py#L799-L835","documentation":"EinsumDense._awq_build only supports 2D and 3D kernels for AWQ quantization: group-wise scaling is defined over a rows-by-columns view of the kernel. A kernel of any other rank (e.g. 4D from a nested output_dim) makes the AWQ math undefined, so it raises this ValueError.","triggerScenarios":"model.quantize(...) with mode 'awq' including an EinsumDense whose output_dim/equation yields a 4D+ kernel (e.g. output_dim=(heads, head_dim, features, extra)).","commonSituations":"Fused or exotic projections with higher-rank kernels caught by a broad AWQ layer filter; models ported from other frameworks where EinsumDense is used as a general tensor contraction.","solutions":["Exclude this layer from AWQ quantization with filters.","Reduce the kernel to rank 2 or 3 by flattening dimensions in the equation or splitting the layer into simpler EinsumDense ops.","Pick a different quantization mode for this layer."],"exampleFix":"# before\nlayer = keras.layers.EinsumDense('abcd,cde->abe', output_dim=(4, 8, 16))  # 4D kernel\nmodel.quantize(awq_config)  # ValueError\n\n# after\nmodel.quantize(awq_config, filters=[l.name for l in model.layers if l is not layer])","handlingStrategy":"validation","validationCode":"rank = len(tuple(layer.kernel.shape))\nif rank not in (2, 3):\n    raise RuntimeError(f'AWQ unsupported for rank-{rank} kernel: {layer.name}')","typeGuard":"def awq_compatible(layer) -> bool:\n    return layer.built and len(tuple(layer.kernel.shape)) in (2, 3)","tryCatchPattern":null,"preventionTips":["Avoid 4D+ EinsumDense kernels in models you plan to AWQ-quantize.","Split high-rank contractions into 2D matmuls."],"tags":["keras","awq","quantization","einsum-dense","kernel-rank"],"backgroundTag":"quantization-unsupported-shape","analyzedSha":"7a34a03db60bf60042242d6a556fc3be119046a5","analyzedAt":"2026-08-25T21:25:25.994Z","schemaVersion":2},"datasetVersion":"2026-08-26T02:17:13.382Z"}