keras-team/keras · error · ValueError

Unsupported quantization mode: {self.quantization_mode}

Error message

Unsupported quantization mode: {self.quantization_mode}

What it means

While saving, EinsumDense._get_kernel_with_merged_lora dequantizes the kernel and merges the LoRA update; it only knows how to dequantize 'int8' and 'int4' (plus a preceding float8 branch). If quantization_mode is anything else, it raises 'Unsupported quantization mode'. This usually signals an internal state mismatch — GPTQ/AWQ states are normally rejected earlier by the calibration guard — rather than a supported user configuration.

Source

Thrown at keras/src/layers/core/einsum_dense.py:1571

                    unpacked_kernel,
                    self.kernel_scale,
                    self.kernel_zero,
                    self.g_idx,
                    group_axis=0,
                )
            else:
                # Per-channel dequantization:
                # kernel [rows, columns], scale [columns]
                kernel_fp = ops.divide(
                    ops.cast(unpacked_kernel, self.compute_dtype),
                    self.kernel_scale,
                )
            kernel_fp = ops.reshape(kernel_fp, self.original_kernel_shape)
        elif self.quantization_mode == "int8":
            adjusted_scale = self._adjust_scale_for_dequant(self.kernel_scale)
            kernel_fp = ops.divide(self._kernel, adjusted_scale)
        else:
            raise ValueError(
                f"Unsupported quantization mode: {self.quantization_mode}"
            )

        # 2. Merge the LoRA update in the float domain
        lora_update = (self.lora_alpha / self.lora_rank) * ops.matmul(
            self.lora_kernel_a, self.lora_kernel_b
        )
        merged_kernel = ops.add(kernel_fp, lora_update)

        # 3. Re-quantize the merged float kernel back to the target format
        if self.quantization_mode == "int4":
            block_size = getattr(self, "_int4_block_size", None)
            rows = self._int4_rows
            columns = self._int4_unpacked_column_size

            # Flatten to 2D [rows, columns]
            flat_kernel = ops.reshape(merged_kernel, (rows, columns))

View on GitHub (pinned to 7a34a03db6)

Solutions

  1. Verify layer.quantization_mode before saving; if it is gptq/awq, run calibration so the earlier guard handles it, or avoid LoRA on that layer.
  2. Keep the layer on a supported mode: int8, int4, or float8 (float8 excludes LoRA).
  3. If subclassing EinsumDense with a custom mode, override _get_kernel_with_merged_lora to handle it.

Example fix

# before
assert layer.lora_enabled and layer.quantization_mode == 'custom_q'
model.save('m.keras')  # ValueError: Unsupported quantization mode

# after
supported = {'float8', 'int8', 'int4'}
assert layer.quantization_mode in supported or not layer.lora_enabled
model.save('m.keras')
Defensive patterns

Strategy: validation

Validate before calling

supported = {'float8', 'int8', 'int4'}
for l in model.layers:
    if getattr(l, 'lora_enabled', False):
        mode = getattr(l, 'quantization_mode', None)
        if mode is not None and mode not in supported:
            raise RuntimeError(f'cannot save {l.name} with mode {mode} + LoRA')

Try / catch

try:
    model.save(path)
except ValueError as e:
    if 'Unsupported quantization mode' in str(e):
        inspect_and_normalize_quantization_modes(model)
    else:
        raise

Prevention

When it happens

Trigger: Saving a model whose EinsumDense has lora_enabled=True and a quantization_mode outside {float8, int8, int4}; reaching save_own_variables with an unexpected mode string (custom or corrupted quantization state).

Common situations: Custom quantization modes injected by subclassing; state corruption after partially applied quantization; version mismatches where a checkpoint carries an unknown quantization_mode value.

Related errors

AI-assisted analysis of keras-team/keras@7a34a03db6 (2026-08-25). Data as JSON: /api/errors/adb26ec55f39e83d. Report an issue: GitHub.