lllyasviel/Fooocus · error · ValueError

normalize should be True if scale is passed

Error message

normalize should be True if scale is passed

What it means

PositionEmbeddingSine (DETR-style, used inside CodeFormer's transformer) validates its constructor arguments: passing a scale while normalize=False is contradictory - scale defines the normalization range for the cumulative position embeddings, so it is meaningless without normalization. ValueError fires immediately at construction.

Source

Thrown at ldm_patched/pfn/architecture/face/codeformer.py:448

    )
    return normalized_feat * style_std.expand(size) + style_mean.expand(size)


class PositionEmbeddingSine(nn.Module):
    """
    This is a more standard version of the position embedding, very similar to the one
    used by the Attention is all you need paper, generalized to work on images.
    """

    def __init__(
        self, num_pos_feats=64, temperature=10000, normalize=False, scale=None
    ):
        super().__init__()
        self.num_pos_feats = num_pos_feats
        self.temperature = temperature
        self.normalize = normalize
        if scale is not None and normalize is False:
            raise ValueError("normalize should be True if scale is passed")
        if scale is None:
            scale = 2 * math.pi
        self.scale = scale

    def forward(self, x, mask=None):
        if mask is None:
            mask = torch.zeros(
                (x.size(0), x.size(2), x.size(3)), device=x.device, dtype=torch.bool
            )
        not_mask = ~mask  # pylint: disable=invalid-unary-operand-type
        y_embed = not_mask.cumsum(1, dtype=torch.float32)
        x_embed = not_mask.cumsum(2, dtype=torch.float32)
        if self.normalize:
            eps = 1e-6
            y_embed = y_embed / (y_embed[:, -1:, :] + eps) * self.scale
            x_embed = x_embed / (x_embed[:, :, -1:] + eps) * self.scale

        dim_t = torch.arange(self.num_pos_feats, dtype=torch.float32, device=x.device)

View on GitHub (pinned to ae05379cc9)

Solutions

1. Set normalize=True when you pass scale
2. Or omit scale entirely (it defaults to 2*pi) and keep normalize=False

Example fix

# before
emb = PositionEmbeddingSine(num_pos_feats=64, scale=2 * math.pi, normalize=False)
# -> ValueError: normalize should be True if scale is passed

# after
emb = PositionEmbeddingSine(num_pos_feats=64, scale=2 * math.pi, normalize=True)

def make_pos_embed(num_pos_feats=64, scale=None, normalize=False):
    if scale is not None:
        normalize = True  # scale requires normalized embeddings
    from ldm_patched.pfn.architecture.face.codeformer import PositionEmbeddingSine
    return PositionEmbeddingSine(num_pos_feats=num_pos_feats, normalize=normalize, scale=scale)

When it happens

Trigger: Instantiating PositionEmbeddingSine(num_pos_feats=..., scale=2*math.pi, normalize=False) - i.e. copying a config that sets scale but leaves normalize at its False default. Not a runtime/data error; purely a constructor argument inconsistency.

Common situations: Porting DETR/CodeFormer config dicts where scale was added experimentally; programmatic sweeps that vary scale but forget the normalize flag.

Related errors

• activation should be relu/gelu, not {activation}.
• checkpoint url or path is invalid
• The hidden size (%d) is not a multiple of the number of atte
• The hidden size (%d) is not a multiple of the number of atte
• scale {scale} is not supported. Supported scales: 2^n and 3.

AI-assisted analysis of lllyasviel/Fooocus@ae05379cc9 (2026-08-15). Data as JSON: /api/errors/3e387b458b8afccb. Report an issue: GitHub.