Stability-AI/generative-models · error · RuntimeError

Initializing ActNorm in reverse direction is disabled by def

Error message

Initializing ActNorm in reverse direction is disabled by default. Use allow_reverse_init=True to enable.

What it means

ActNorm layers can data-dependently initialize their scale/bias on the first forward pass. Initialization in the reverse (generative) direction is considered unsafe by default, so ActNorm2D.reverse raises this RuntimeError during training when the layer is uninitialized and allow_reverse_init is False (the default).

Source

Thrown at sgm/modules/autoencoding/lpips/util.py:110

            self.initialized.fill_(1)

        h = self.scale * (input + self.loc)

        if squeeze:
            h = h.squeeze(-1).squeeze(-1)

        if self.logdet:
            log_abs = torch.log(torch.abs(self.scale))
            logdet = height * width * torch.sum(log_abs)
            logdet = logdet * torch.ones(input.shape[0]).to(input)
            return h, logdet

        return h

    def reverse(self, output):
        if self.training and self.initialized.item() == 0:
            if not self.allow_reverse_init:
                raise RuntimeError(
                    "Initializing ActNorm in reverse direction is "
                    "disabled by default. Use allow_reverse_init=True to enable."
                )
            else:
                self.initialize(output)
                self.initialized.fill_(1)

        if len(output.shape) == 2:
            output = output[:, :, None, None]
            squeeze = True
        else:
            squeeze = False

        h = output / self.scale - self.loc

        if squeeze:
            h = h.squeeze(-1).squeeze(-1)
        return h

View on GitHub (pinned to e8cd657656)

Solutions

1. Construct ActNorm2D with allow_reverse_init=True so reverse-direction initialization is permitted.
2. Run one forward pass (or call initialize manually) before invoking reverse in training.
3. If reverse should never initialize, restructure the code so normalization is applied in the forward direction first.

Example fix

// before
actnorm = ActNorm2D(num_features)
loss = model.reverse(...)  # RuntimeError on first training step
// after
actnorm = ActNorm2D(num_features, allow_reverse_init=True)

if flow_layer.training and flow_layer.initialized.item() == 0 and not flow_layer.allow_reverse_init:
    flow_layer.initialize(output)  # or run one forward pass before reverse

def can_reverse_init(layer) -> bool:
    return (not layer.training) or layer.initialized.item() == 1 or layer.allow_reverse_init

try:
    out = actnorm.reverse(x)
except RuntimeError as e:
    if "reverse direction" in str(e):
        actnorm.allow_reverse_init = True
        out = actnorm.reverse(x)

When it happens

Trigger: Running a model flow that applies ActNorm in reverse mode while model.training is True and the layer's data has not yet been initialized (initialized==0), with ActNorm2D created without allow_reverse_init=True — e.g. computing a loss that requires the inverse pass first during training.

Common situations: Using LPIPS/flow utilities in a training loop that evaluates the reverse pass before any forward pass initialized the layers; swapping inference code (which initializes in forward) into training where the first call is reverse.

Related errors

• Sampler and loss function need to be set for training.

AI-assisted analysis of Stability-AI/generative-models@e8cd657656 (2026-08-29). Data as JSON: /api/errors/a2b09b956982e7d6. Report an issue: GitHub.