invoke-ai/InvokeAI · error · RuntimeError

Unable to find token id for token '{trigger}'

Error message

Unable to find token id for token '{trigger}'

What it means

apply_ti() injects textual-inversion embedding vectors into the CLIP token embedding matrix. It maps each trigger token via the tokenizer; if the tokenizer returns unk_token_id the trigger does not exist in the model's vocabulary, so patching cannot proceed and RuntimeError is raised.

Source

Thrown at invokeai/backend/model_patcher.py:120

            # resize_token_embeddings(...) constructs a new torch.nn.Embedding internally. Initializing the weights of
            # this embedding is slow and unnecessary, so we wrap this step in skip_torch_weight_init() to save some
            # time.
            with skip_torch_weight_init():
                text_encoder.resize_token_embeddings(init_tokens_count + new_tokens_added, pad_to_multiple_of)
            model_embeddings = text_encoder.get_input_embeddings()

            for ti_name, ti in ti_list:
                assert isinstance(ti, TextualInversionModelRaw)
                ti_embedding = _get_ti_embedding(text_encoder.get_input_embeddings(), ti)

                ti_tokens = []
                for i in range(ti_embedding.shape[0]):
                    embedding = ti_embedding[i]
                    trigger = _get_trigger(ti_name, i)

                    token_id = ti_tokenizer.convert_tokens_to_ids(trigger)
                    if token_id == ti_tokenizer.unk_token_id:
                        raise RuntimeError(f"Unable to find token id for token '{trigger}'")

                    if model_embeddings.weight.data[token_id].shape != embedding.shape:
                        raise ValueError(
                            f"Cannot load embedding for {trigger}. It was trained on a model with token dimension"
                            f" {embedding.shape[0]}, but the current model has token dimension"
                            f" {model_embeddings.weight.data[token_id].shape[0]}."
                        )

                    model_embeddings.weight.data[token_id] = embedding.to(
                        device=TorchDevice.choose_torch_device(), dtype=text_encoder.dtype
                    )
                    ti_tokens.append(token_id)

                if len(ti_tokens) > 1:
                    ti_manager.pad_tokens[ti_tokens[0]] = ti_tokens[1:]

            yield ti_tokenizer, ti_manager

View on GitHub (pinned to 0b6a024f2f)

Solutions

1. Verify the embedding's trigger string matches what you typed in the prompt and that the embedding is loaded (check InvokeAI's embedding list).
2. Use an embedding trained for the same base model/tokenizer (e.g. SD1.5 embedding on SD1.5, not SDXL).
3. Re-import the embedding so its trigger tokens are registered; ensure the embedding file metadata contains valid trigger names.
4. Update InvokeAI — token registration for embeddings has been fixed in several releases.

Example fix

// before: trigger unknown to tokenizer
token_id = ti_tokenizer.convert_tokens_to_ids(trigger)
// after: register the trigger as a special token before patching
num_added = ti_tokenizer.add_tokens(trigger)
text_encoder.resize_token_embeddings(len(ti_tokenizer))

triggers = ti_tokenizer.convert_tokens_to_ids(trigger)
if triggers == ti_tokenizer.unk_token_id:
    print(f'Trigger {trigger!r} not in vocabulary; embedding incompatible with this tokenizer')
    # register it: ti_tokenizer.add_tokens(trigger) + resize embeddings

def trigger_exists(tokenizer, trigger: str) -> bool:
    return tokenizer.convert_tokens_to_ids(trigger) != tokenizer.unk_token_id

try:
    patcher.apply_ti(...)
except RuntimeError as e:
    if str(e).startswith('Unable to find token id'):
        missing = str(e).split("'")[1]
        print(f'Embedding trigger {missing} not registered for this model/tokenizer')
    else:
        raise

When it happens

Trigger: Loading a textual inversion embedding whose trigger token (from _get_trigger, e.g. '<my-embedding>' or per-vector triggers) is not resolvable to a token id by ti_tokenizer — typically because the embedding's trigger strings were never added as special tokens to the tokenizer before this call.

Common situations: Embeddings trained for a different tokenizer/CLIP variant than the loaded model; mismatched embedding naming so trigger lookup falls back to the raw name; corrupted embedding metadata lacking trained_tokens.

Related errors

• Cannot load embedding for {trigger}. It was trained on a mod
• Expected PreTrainedTokenizerBase for tokenizer, got {type(to
• Tokenizer returned unexpected types.
• Blend is not supported here - you need to get tokens for eac
• Expected PreTrainedTokenizerBase for tokenizer, got {type(to

AI-assisted analysis of invoke-ai/InvokeAI@0b6a024f2f (2026-08-29). Data as JSON: /api/errors/8481a4935291bbf5. Report an issue: GitHub.