invoke-ai/InvokeAI · error · ValueError

ed_hidden_size {self.ed_hidden_size} must be divisible by ed

Error message

ed_hidden_size {self.ed_hidden_size} must be divisible by ed_num_heads {self.ed_num_heads}

What it means

PixDiT_T2I multi-head attention on the encoder-decoder (ED) path requires ed_hidden_size to be evenly divisible by ed_num_heads so the hidden dimension can be split into per-head slices. During __init__, when enable_ed is on and the computed stage count is > 0, the constructor validates this and raises ValueError if the head dimension would not be an integer. It is a configuration error caught at model construction time, before any weights are loaded or forwards run.

Source

Thrown at invokeai/backend/pid/_src/networks/pixeldit_official.py:1292

        self.enable_ed = bool(enable_ed)
        self.ed_compress_ratio = int(ed_compress_ratio)
        self.ed_depth_per_stage = int(ed_depth_per_stage)
        self.ed_window_size = int(ed_window_size)
        self.ed_num_heads = int(ed_num_heads) if ed_num_heads is not None else self.num_groups
        self.ed_hidden_size = int(ed_hidden_size) if ed_hidden_size is not None else self.hidden_size
        self.ed_use_token_shuffle = bool(ed_use_token_shuffle)
        self.encoder_ed: Optional[_EncoderED] = None
        self.decoder_ed: Optional[_DecoderED] = None
        self.s_ed_proj_in: Optional[nn.Module] = None
        self.s_ed_proj_out: Optional[nn.Module] = None
        self.s_ed_cond_proj: Optional[nn.Module] = None
        self.s_ed_in_norm: Optional[RMSNorm] = None
        self.s_ed_out_norm: Optional[RMSNorm] = None
        num_stages = _compute_num_stages_from_ratio(self.ed_compress_ratio) if self.enable_ed else 0
        self.use_ed = self.enable_ed and num_stages > 0
        if self.use_ed:
            if self.ed_hidden_size % self.ed_num_heads != 0:
                raise ValueError(
                    f"ed_hidden_size {self.ed_hidden_size} must be divisible by ed_num_heads {self.ed_num_heads}"
                )
            self.s_ed_proj_in = (
                nn.Identity()
                if self.ed_hidden_size == self.hidden_size
                else nn.Linear(self.hidden_size, self.ed_hidden_size, bias=True)
            )
            self.s_ed_proj_out = (
                nn.Identity()
                if self.ed_hidden_size == self.hidden_size
                else nn.Linear(self.ed_hidden_size, self.hidden_size, bias=True)
            )
            self.s_ed_cond_proj = (
                nn.Identity()
                if self.ed_hidden_size == self.hidden_size
                else nn.Linear(self.hidden_size, self.ed_hidden_size, bias=True)
            )
            self.s_ed_in_norm = RMSNorm(self.ed_hidden_size, eps=1e-6)

View on GitHub (pinned to 0b6a024f2f)

Solutions

  1. Choose ed_num_heads that divides ed_hidden_size evenly (e.g. 1024 with 8, 16, or 32 heads)
  2. Or adjust ed_hidden_size to the nearest value divisible by the desired head count
  3. Disable the ED path with enable_ed=False if the encoder-decoder stages are not needed

Example fix

// before
PixDiT_T2I(ed_hidden_size=1024, ed_num_heads=48, enable_ed=True)
// after
PixDiT_T2I(ed_hidden_size=1024, ed_num_heads=16, enable_ed=True)
Defensive patterns

Strategy: validation

Validate before calling

if enable_ed and ed_hidden_size % ed_num_heads != 0:
    raise ValueError("ed_hidden_size must be divisible by ed_num_heads")

Try / catch

try:
    net = PixDiT_T2I(**cfg)
except ValueError as e:
    if "divisible" in str(e):
        cfg["ed_num_heads"] = next(h for h in (32,16,8) if cfg["ed_hidden_size"] % h == 0)
        net = PixDiT_T2I(**cfg)
    else:
        raise

Prevention

When it happens

Trigger: Constructing PixDiT_T2I (directly or via a config) with enable_ed=True while ed_hidden_size % ed_num_heads != 0, e.g. ed_hidden_size=1024 with ed_num_heads=48 or a prime/odd head count.

Common situations: Hand-edited YAML/JSON model configs, porting hyperparameters from a differently sized variant, or changing ed_hidden_size for memory reasons without re-deriving a compatible head count.

Related errors

AI-assisted analysis of invoke-ai/InvokeAI@0b6a024f2f (2026-08-29). Data as JSON: /api/errors/0c76ef0ad5c4b6da. Report an issue: GitHub.