lllyasviel/Fooocus · error · ValueError

invalid distribution {distribution}

Error message

invalid distribution {distribution}

What it means

variance_scaling_ in the vendored timm weight-init helper initializes a tensor with fan-based variance using one of three distributions: 'truncated_normal', 'normal', or 'uniform'. An unrecognized distribution string falls through the if/elif chain and raises ValueError(f"invalid distribution {distribution}"). This runs at model weight-initialization time, before any training/inference.

Source

Thrown at ldm_patched/pfn/architecture/timm/weight_init.py:124

        denom = fan_in
    elif mode == "fan_out":
        denom = fan_out
    elif mode == "fan_avg":
        denom = (fan_in + fan_out) / 2

    variance = scale / denom  # type: ignore

    if distribution == "truncated_normal":
        # constant is stddev of standard normal truncated to (-2, 2)
        trunc_normal_tf_(tensor, std=math.sqrt(variance) / 0.87962566103423978)
    elif distribution == "normal":
        tensor.normal_(std=math.sqrt(variance))
    elif distribution == "uniform":
        bound = math.sqrt(3 * variance)
        # pylint: disable=invalid-unary-operand-type
        tensor.uniform_(-bound, bound)
    else:
        raise ValueError(f"invalid distribution {distribution}")


def lecun_normal_(tensor):
    variance_scaling_(tensor, mode="fan_in", distribution="truncated_normal")

View on GitHub (pinned to ae05379cc9)

Solutions

1. Use one of the exact strings: 'truncated_normal', 'normal', or 'uniform'.
2. For truncated normal you can also call trunc_normal_(tensor, std=...) or lecun_normal_ directly instead of variance_scaling_.
3. Print/validate the distribution value before init when it comes from a config file.

Example fix

# before
variance_scaling_(w, mode='fan_in', distribution='trunc_normal')
# after
variance_scaling_(w, mode='fan_in', distribution='truncated_normal')

DISTRIBUTIONS = ('truncated_normal', 'normal', 'uniform')
if distribution not in DISTRIBUTIONS:
    raise ValueError(f'distribution must be one of {DISTRIBUTIONS}, got {distribution!r}')

def is_valid_distribution(v) -> bool:
    return v in ('truncated_normal', 'normal', 'uniform')

When it happens

Trigger: Calling variance_scaling_(w, mode=..., distribution='trunc_normal') (missing final 'ed'), 'gaussian', 'xavier', or any non-listed string; or calling the related _init_vit_weights / init helpers with a bad distribution argument.

Common situations: Copying init code from another timm version where distribution names differ, or hand-rolling a custom init for a CodeFormer/pfn face model that routes into this helper.

Related errors

• Wrong activation value in EqualLinear: {activation}Supported
• Wrong sample mode {self.sample_mode}, supported ones are ['u
• Wrong activation value in EqualLinear: {activation}Supported
• Max depth of recursive function `tie_encoder_to_decoder` rea
• Invalid value for parameter `type`: {type}. Please choose fr

AI-assisted analysis of lllyasviel/Fooocus@ae05379cc9 (2026-08-15). Data as JSON: /api/errors/44bc59d953dbd2f7. Report an issue: GitHub.