keras-team/keras · error · ValueError

Lora is incompatible with kernel constraints. In order to en

Error message

Lora is incompatible with kernel constraints. In order to enable lora on this layer, remove the `kernel_constraint` argument.

What it means

EinsumDense.enable_lora() refuses to enable LoRA when the layer was constructed with a kernel_constraint argument. Constraints (e.g. UnitNorm, MaxNorm) are applied to the kernel variable and are incompatible with the LoRA A/B factorization Keras maintains. The check runs before any LoRA state is created, so the layer is left unchanged.

Source

Thrown at keras/src/layers/core/einsum_dense.py:335

        return tuple(full_output_shape)

    def call(self, inputs, training=None):
        x = ops.einsum(self.equation, inputs, self.kernel)
        if self.bias is not None:
            x = ops.add(x, self.bias)
        if self.activation is not None:
            x = self.activation(x)
        return x

    def enable_lora(
        self,
        rank,
        lora_alpha=None,
        a_initializer="he_uniform",
        b_initializer="zeros",
    ):
        if self.kernel_constraint:
            raise ValueError(
                "Lora is incompatible with kernel constraints. "
                "In order to enable lora on this layer, remove the "
                "`kernel_constraint` argument."
            )
        if not self.built:
            raise ValueError(
                "Cannot enable lora on a layer that isn't yet built."
            )
        if self.lora_enabled:
            raise ValueError(
                "lora is already enabled. This can only be done once per layer."
            )
        if self.quantization_mode == "gptq":
            raise NotImplementedError(
                "lora is not currently supported with GPTQ quantization."
            )
        self._tracker.unlock()
        # Determine the appropriate (unpacked) kernel shape for LoRA.

View on GitHub (pinned to 7a34a03db6)

Solutions

Remove the kernel_constraint argument from the EinsumDense constructor (and from any config that sets it), then call enable_lora again.
If a norm-bounded kernel is required, apply normalization outside the layer (e.g. a separate normalization layer) so the kernel itself is unconstrained.
If you loaded the layer from a saved model, rebuild it without the constraint and transfer weights before enabling LoRA.

Example fix

# before
layer = keras.layers.EinsumDense('ab,bc->ac', output_dim=64, kernel_constraint='unit_norm')
layer.enable_lora(rank=8)  # ValueError

# after
layer = keras.layers.EinsumDense('ab,bc->ac', output_dim=64)
layer.enable_lora(rank=8)

Defensive patterns

Strategy: validation

Validate before calling

if getattr(layer, 'kernel_constraint', None) is not None:
    raise RuntimeError(f'{layer.name} has kernel_constraint; remove before enable_lora')
layer.enable_lora(rank)

Try / catch

try:
    layer.enable_lora(rank)
except ValueError as e:
    if 'kernel_constraint' in str(e):
        # rebuild layer without constraint
        ...

Prevention

Do not set kernel_constraint on layers you plan to LoRA-tune.
Assert layer.kernel_constraint is None before calling enable_lora.

When it happens

Trigger: Calling layer.enable_lora(rank=...) on an EinsumDense (or a subclass such as an attention projection) that was created with kernel_constraint=... (e.g. keras.constraints.UnitNorm()).

Common situations: Copy-pasting a layer definition that included a norm constraint while adding PEFT/LoRA fine-tuning; loading a legacy model with constrained projections and trying to LoRA-ize it; config files that set kernel_constraint globally.

Related errors

AI-assisted analysis of keras-team/keras@7a34a03db6 (2026-08-25). Data as JSON: /api/errors/278ddfa4f47b25cd. Report an issue: GitHub.