invoke-ai/InvokeAI · error · ValueError

Cannot load embedding for {trigger}. It was trained on a mod

Error message

Cannot load embedding for {trigger}. It was trained on a model with token dimension {embedding.shape[0]}, but the current model has token dimension {model_embeddings.weight.data[token_id].shape[0]}.

What it means

apply_ti() checks that each embedding vector's dimension matches the model's token embedding width before the in-place copy. A mismatch means the textual inversion was trained against a model with a different embedding dimension, and ValueError is raised naming both dimensions.

Source

Thrown at invokeai/backend/model_patcher.py:123

            with skip_torch_weight_init():
                text_encoder.resize_token_embeddings(init_tokens_count + new_tokens_added, pad_to_multiple_of)
            model_embeddings = text_encoder.get_input_embeddings()

            for ti_name, ti in ti_list:
                assert isinstance(ti, TextualInversionModelRaw)
                ti_embedding = _get_ti_embedding(text_encoder.get_input_embeddings(), ti)

                ti_tokens = []
                for i in range(ti_embedding.shape[0]):
                    embedding = ti_embedding[i]
                    trigger = _get_trigger(ti_name, i)

                    token_id = ti_tokenizer.convert_tokens_to_ids(trigger)
                    if token_id == ti_tokenizer.unk_token_id:
                        raise RuntimeError(f"Unable to find token id for token '{trigger}'")

                    if model_embeddings.weight.data[token_id].shape != embedding.shape:
                        raise ValueError(
                            f"Cannot load embedding for {trigger}. It was trained on a model with token dimension"
                            f" {embedding.shape[0]}, but the current model has token dimension"
                            f" {model_embeddings.weight.data[token_id].shape[0]}."
                        )

                    model_embeddings.weight.data[token_id] = embedding.to(
                        device=TorchDevice.choose_torch_device(), dtype=text_encoder.dtype
                    )
                    ti_tokens.append(token_id)

                if len(ti_tokens) > 1:
                    ti_manager.pad_tokens[ti_tokens[0]] = ti_tokens[1:]

            yield ti_tokenizer, ti_manager

        finally:
            if init_tokens_count and new_tokens_added:
                text_encoder.resize_token_embeddings(init_tokens_count, pad_to_multiple_of)

View on GitHub (pinned to 0b6a024f2f)

Solutions

Use an embedding trained for the same base model architecture as the currently loaded checkpoint.
If only the trigger name collides, rename the embedding's trigger to avoid confusion.
Retrain or convert the embedding for your base model (dimension conversion is not automatic).
Check the embedding's base model metadata in the Model Manager before enabling it.

Example fix

// before: SD1.5 embedding applied to SDXL pipeline
pipeline = InvokePipelinite(model='sdxl-base', loras=[sd15_embedding])
// after: match embedding to base model
pipeline = InvokePipelinite(model='sdxl-base', loras=[sdxl_compatible_embedding])

Defensive patterns

Strategy: validation

Validate before calling

ti_vec = ti_embedding[0]
token_width = model_embeddings.weight.data[0].shape[0]
if ti_vec.shape[0] != token_width:
    print(f'Embedding dim {ti_vec.shape[0]} != model token dim {token_width}: wrong base model')

Type guard

def embedding_compatible(ti_embedding, model_embeddings) -> bool:
    return ti_embedding.shape[-1] == model_embeddings.weight.data.shape[-1]

Try / catch

try:
    patcher.apply_ti(...)
except ValueError as e:
    if 'Cannot load embedding' in str(e) and 'token dimension' in str(e):
        print('Use an embedding trained for this base model architecture')
    else:
        raise

Prevention

Store/label embeddings with their base model (SD1.5=768, SD2.x=1024, SDXL=2048).
Filter enabled embeddings by base-model compatibility in the Model Manager.
Don't reuse trigger names across models trained for different architectures.

When it happens

Trigger: Applying a TI embedding whose vector length (embedding.shape[0]) differs from the target CLIP token embedding width (model_embeddings.weight.data[token_id].shape[0]) — e.g. a 768-dim SD1.5 embedding applied to an SDXL (2048-dim) text encoder or vice versa.

Common situations: Cross-base-model embedding usage (SD1.5 ↔ SD2.x ↔ SDXL); SD2.x's 1024-dim vs SD1.5's 768-dim embeddings; accidentally selecting the wrong embedding for the active pipeline.

Related errors

AI-assisted analysis of invoke-ai/InvokeAI@0b6a024f2f (2026-08-29). Data as JSON: /api/errors/9272f1261b0364aa. Report an issue: GitHub.