Stability-AI/generative-models · warning

{self.class.name}: Found context dims {context_dim}

Error message

{self.__class__.__name__}: Found context dims {context_dim} of depth {len(context_dim)}, which does not match the specified 'depth' of {depth}. Setting context_dim to {depth * [context_dim[0]]} now.

What it means

This is a warning logged by the SpatialTransformer in Stability AI's generative-models codebase when the length of the context_dim list does not match the transformer 'depth' (number of transformer blocks). The library does not raise a fatal exception; it prints a warning and auto-corrects context_dim to a list of length 'depth' where every entry is the first element of the provided list, so all blocks share the same cross-attention context dimension. It signals that the model config was not internally consistent, and the effective architecture differs from what the config literally described.

Source

Thrown at sgm/modules/attention.py:654

        context_dim=None,
        disable_self_attn=False,
        use_linear=False,
        attn_type="softmax",
        use_checkpoint=True,
        # sdp_backend=SDPBackend.FLASH_ATTENTION
        sdp_backend=None,
    ):
        super().__init__()
        logpy.debug(
            f"constructing {self.__class__.__name__} of depth {depth} w/ "
            f"{in_channels} channels and {n_heads} heads."
        )

        if exists(context_dim) and not isinstance(context_dim, list):
            context_dim = [context_dim]
        if exists(context_dim) and isinstance(context_dim, list):
            if depth != len(context_dim):
                logpy.warn(
                    f"{self.__class__.__name__}: Found context dims "
                    f"{context_dim} of depth {len(context_dim)}, which does not "
                    f"match the specified 'depth' of {depth}. Setting context_dim "
                    f"to {depth * [context_dim[0]]} now."
                )
                # depth does not match context dims.
                assert all(
                    map(lambda x: x == context_dim[0], context_dim)
                ), "need homogenous context_dim to match depth automatically"
                context_dim = depth * [context_dim[0]]
        elif context_dim is None:
            context_dim = [None] * depth
        self.in_channels = in_channels
        inner_dim = n_heads * d_head
        self.norm = Normalize(in_channels)
        if not use_linear:
            self.proj_in = nn.Conv2d(
                in_channels, inner_dim, kernel_size=1, stride=1, padding=0

View on GitHub (pinned to e8cd657656)

Solutions

Pass context_dim as a list whose length equals depth, e.g. context_dim=[768, 768, 768, 768] for depth=4, instead of a scalar or single-element list.
If a single shared dim is intended, do nothing — the warning is benign and the library already substitutes depth * [context_dim[0]]; silence/ignore it after confirming the architecture is as intended.
Check the model config (YAML or dict) for a mismatch between depth and the number of context dims and fix the config source.
If loading a checkpoint, verify the checkpoint's expected UNet depth/context_dim values match your config to avoid silently mismatched weights.

Example fix

// before
transformer = SpatialTransformer(
    in_channels=320, n_heads=8, d_head=40, depth=4,
    context_dim=768,  # coerced to [768], len 1 != depth 4
)
// after
transformer = SpatialTransformer(
    in_channels=320, n_heads=8, d_head=40, depth=4,
    context_dim=[768, 768, 768, 768],  # one dim per depth level
)

Defensive patterns

Strategy: validation

Validate before calling

def validate_context_dim(context_dim, depth):
    if context_dim is not None and not isinstance(context_dim, list):
        context_dim = [context_dim]
    if context_dim is not None and len(context_dim) != depth:
        raise ValueError(
            f"context_dim has {len(context_dim)} entries but depth={depth}; "
            f"expected a list of exactly {depth} dims"
        )
    return context_dim

# call before constructing SpatialTransformer
context_dim = validate_context_dim(cfg.get("context_dim"), cfg["depth"])

Type guard

from typing import Optional, Union, List

def is_valid_context_dim(
    context_dim: Optional[Union[int, List[int]]], depth: int
) -> bool:
    if context_dim is None:
        return True
    dims = context_dim if isinstance(context_dim, list) else [context_dim]
    return len(dims) == depth

Prevention

Always define context_dim as a list with one entry per transformer depth level in model configs.
Add a config schema/lint check that asserts len(context_dim) == depth before model instantiation.
When porting configs across model versions, re-check scalar-to-list parameters like context_dim, model_channels, and num_res_blocks.
When a warning appears, read it and verify depth * [context_dim[0]] is the intended architecture before training to avoid silently mis-configured models.

When it happens

Trigger: Constructing SpatialTransformer (directly or via UNet model config, e.g. in sgm UNetModel attn precision or a YAML model config) with a context_dim that was wrapped into a single-element list [dim] while depth > 1, e.g. SpatialTransformer(in_channels, n_heads, d_head, depth=4, context_dim=768) with context_dim coerced to [768], so len([768]) != 4.

Common situations: Hand-edited Stable Diffusion / SDXL model YAML configs where context_dim was specified as a scalar but depth is multi-block; porting configs between versions where context_dim used to be a scalar; building UNets programmatically and passing one context dim instead of one per level.

Related errors

AI-assisted analysis of Stability-AI/generative-models@e8cd657656 (2026-08-29). Data as JSON: /api/errors/5e49de6d2fca5fba. Report an issue: GitHub.