{"record":{"id":"5e49de6d2fca5fba","repo":"Stability-AI/generative-models","slug":"self-class-name-found-context-dims-con","errorCode":null,"errorMessage":"{self.__class__.__name__}: Found context dims {context_dim} of depth {len(context_dim)}, which does not match the specified 'depth' of {depth}. Setting context_dim to {depth * [context_dim[0]]} now.","messagePattern":"(.+?): Found context dims (.+?) of depth (.+?), which does not match the specified 'depth' of (.+?)\\. Setting context_dim to (.+?) now\\.","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"sgm/modules/attention.py","lineNumber":654,"sourceCode":"        context_dim=None,\n        disable_self_attn=False,\n        use_linear=False,\n        attn_type=\"softmax\",\n        use_checkpoint=True,\n        # sdp_backend=SDPBackend.FLASH_ATTENTION\n        sdp_backend=None,\n    ):\n        super().__init__()\n        logpy.debug(\n            f\"constructing {self.__class__.__name__} of depth {depth} w/ \"\n            f\"{in_channels} channels and {n_heads} heads.\"\n        )\n\n        if exists(context_dim) and not isinstance(context_dim, list):\n            context_dim = [context_dim]\n        if exists(context_dim) and isinstance(context_dim, list):\n            if depth != len(context_dim):\n                logpy.warn(\n                    f\"{self.__class__.__name__}: Found context dims \"\n                    f\"{context_dim} of depth {len(context_dim)}, which does not \"\n                    f\"match the specified 'depth' of {depth}. Setting context_dim \"\n                    f\"to {depth * [context_dim[0]]} now.\"\n                )\n                # depth does not match context dims.\n                assert all(\n                    map(lambda x: x == context_dim[0], context_dim)\n                ), \"need homogenous context_dim to match depth automatically\"\n                context_dim = depth * [context_dim[0]]\n        elif context_dim is None:\n            context_dim = [None] * depth\n        self.in_channels = in_channels\n        inner_dim = n_heads * d_head\n        self.norm = Normalize(in_channels)\n        if not use_linear:\n            self.proj_in = nn.Conv2d(\n                in_channels, inner_dim, kernel_size=1, stride=1, padding=0","sourceCodeStart":636,"sourceCodeEnd":672,"githubUrl":"https://github.com/Stability-AI/generative-models/blob/e8cd657656fa5d61688191730d0e03242bf4ed44/sgm/modules/attention.py#L636-L672","documentation":"This is a warning logged by the SpatialTransformer in Stability AI's generative-models codebase when the length of the context_dim list does not match the transformer 'depth' (number of transformer blocks). The library does not raise a fatal exception; it prints a warning and auto-corrects context_dim to a list of length 'depth' where every entry is the first element of the provided list, so all blocks share the same cross-attention context dimension. It signals that the model config was not internally consistent, and the effective architecture differs from what the config literally described.","triggerScenarios":"Constructing SpatialTransformer (directly or via UNet model config, e.g. in sgm UNetModel attn precision or a YAML model config) with a context_dim that was wrapped into a single-element list [dim] while depth > 1, e.g. SpatialTransformer(in_channels, n_heads, d_head, depth=4, context_dim=768) with context_dim coerced to [768], so len([768]) != 4.","commonSituations":"Hand-edited Stable Diffusion / SDXL model YAML configs where context_dim was specified as a scalar but depth is multi-block; porting configs between versions where context_dim used to be a scalar; building UNets programmatically and passing one context dim instead of one per level.","solutions":["Pass context_dim as a list whose length equals depth, e.g. context_dim=[768, 768, 768, 768] for depth=4, instead of a scalar or single-element list.","If a single shared dim is intended, do nothing — the warning is benign and the library already substitutes depth * [context_dim[0]]; silence/ignore it after confirming the architecture is as intended.","Check the model config (YAML or dict) for a mismatch between depth and the number of context dims and fix the config source.","If loading a checkpoint, verify the checkpoint's expected UNet depth/context_dim values match your config to avoid silently mismatched weights."],"exampleFix":"// before\ntransformer = SpatialTransformer(\n    in_channels=320, n_heads=8, d_head=40, depth=4,\n    context_dim=768,  # coerced to [768], len 1 != depth 4\n)\n// after\ntransformer = SpatialTransformer(\n    in_channels=320, n_heads=8, d_head=40, depth=4,\n    context_dim=[768, 768, 768, 768],  # one dim per depth level\n)","handlingStrategy":"validation","validationCode":"def validate_context_dim(context_dim, depth):\n    if context_dim is not None and not isinstance(context_dim, list):\n        context_dim = [context_dim]\n    if context_dim is not None and len(context_dim) != depth:\n        raise ValueError(\n            f\"context_dim has {len(context_dim)} entries but depth={depth}; \"\n            f\"expected a list of exactly {depth} dims\"\n        )\n    return context_dim\n\n# call before constructing SpatialTransformer\ncontext_dim = validate_context_dim(cfg.get(\"context_dim\"), cfg[\"depth\"])","typeGuard":"from typing import Optional, Union, List\n\ndef is_valid_context_dim(\n    context_dim: Optional[Union[int, List[int]]], depth: int\n) -> bool:\n    if context_dim is None:\n        return True\n    dims = context_dim if isinstance(context_dim, list) else [context_dim]\n    return len(dims) == depth","tryCatchPattern":null,"preventionTips":["Always define context_dim as a list with one entry per transformer depth level in model configs.","Add a config schema/lint check that asserts len(context_dim) == depth before model instantiation.","When porting configs across model versions, re-check scalar-to-list parameters like context_dim, model_channels, and num_res_blocks.","When a warning appears, read it and verify depth * [context_dim[0]] is the intended architecture before training to avoid silently mis-configured models."],"tags":["python","configuration","deep-learning","dimension-mismatch"],"backgroundTag":"dimension-mismatch-warning","analyzedSha":"e8cd657656fa5d61688191730d0e03242bf4ed44","analyzedAt":"2026-08-29T11:23:43.234Z","schemaVersion":2},"datasetVersion":"2026-08-29T12:17:43.993Z"}