lllyasviel/ControlNet · error · NotImplementedError

encoder_posterior of type '{type(encoder_posterior)}' not ye

Error message

encoder_posterior of type '{type(encoder_posterior)}' not yet implemented

What it means

get_first_stage_encoding accepts either a DiagonalGaussianDistribution (VAE posterior, sampled to a latent) or a plain torch.Tensor latent. Anything else — e.g. a numpy array, list, or a different distribution object — raises NotImplementedError before applying scale_factor.

Source

Thrown at ldm/models/diffusion/ddpm.py:661

    def _get_denoise_row_from_list(self, samples, desc='', force_no_decoder_quantization=False):
        denoise_row = []
        for zd in tqdm(samples, desc=desc):
            denoise_row.append(self.decode_first_stage(zd.to(self.device),
                                                       force_not_quantize=force_no_decoder_quantization))
        n_imgs_per_row = len(denoise_row)
        denoise_row = torch.stack(denoise_row)  # n_log_step, n_row, C, H, W
        denoise_grid = rearrange(denoise_row, 'n b c h w -> b n c h w')
        denoise_grid = rearrange(denoise_grid, 'b n c h w -> (b n) c h w')
        denoise_grid = make_grid(denoise_grid, nrow=n_imgs_per_row)
        return denoise_grid

    def get_first_stage_encoding(self, encoder_posterior):
        if isinstance(encoder_posterior, DiagonalGaussianDistribution):
            z = encoder_posterior.sample()
        elif isinstance(encoder_posterior, torch.Tensor):
            z = encoder_posterior
        else:
            raise NotImplementedError(f"encoder_posterior of type '{type(encoder_posterior)}' not yet implemented")
        return self.scale_factor * z

    def get_learned_conditioning(self, c):
        if self.cond_stage_forward is None:
            if hasattr(self.cond_stage_model, 'encode') and callable(self.cond_stage_model.encode):
                c = self.cond_stage_model.encode(c)
                if isinstance(c, DiagonalGaussianDistribution):
                    c = c.mode()
            else:
                c = self.cond_stage_model(c)
        else:
            assert hasattr(self.cond_stage_model, self.cond_stage_forward)
            c = getattr(self.cond_stage_model, self.cond_stage_forward)(c)
        return c

    def meshgrid(self, h, w):
        y = torch.arange(0, h).view(h, 1, 1).repeat(1, w, 1)
        x = torch.arange(0, w).view(1, w, 1).repeat(h, 1, 1)

View on GitHub (pinned to ed85cd1e25)

Solutions

1. Ensure the first-stage encoder returns DiagonalGaussianDistribution (use ldm.modules.distributions.DiagonalGaussianDistribution) or a torch tensor
2. Convert precomputed latents to torch tensors: torch.from_numpy(latent).to(device)
3. Check any custom encode override returns one of the two supported types

Example fix

# before
z = np.load('latent.npy')
# after
z = torch.from_numpy(np.load('latent.npy')).to(device, torch.float32)

from ldm.modules.distributions import DiagonalGaussianDistribution
assert isinstance(encoder_posterior, (DiagonalGaussianDistribution, torch.Tensor)), type(encoder_posterior)

import torch
from ldm.modules.distributions import DiagonalGaussianDistribution

def is_valid_posterior(p) -> bool:
    return isinstance(p, (DiagonalGaussianDistribution, torch.Tensor))

try:
    z = model.get_first_stage_encoding(post)
except NotImplementedError:
    z = model.scale_factor * torch.as_tensor(post, dtype=torch.float32, device=model.device)

When it happens

Trigger: Calling on_train_batch_start/get_input where the VAE encode step returned a non-tensor/non-DiagonalGaussianDistribution object: a numpy latent, a tuple of (mean, logvar), or None from a custom first_stage_model.

Common situations: Swapping in a custom VAE whose encode returns a different type; preprocessing pipelines that precompute latents to numpy .npy files; monkey-patches that bypass the standard encode path.

Related errors

• unknown loss type '{loss_type}'
• Parameterization {self.parameterization} not yet supported
• Unsupported noise schedule {}. The schedule needs to be 'dis

AI-assisted analysis of lllyasviel/ControlNet@ed85cd1e25 (2026-08-27). Data as JSON: /api/errors/e4719032baf63952. Report an issue: GitHub.