invoke-ai/InvokeAI · error · NotAMatchError

missing tokenizer/ subfolder

Error message

missing tokenizer/ subfolder

What it means

`NotAMatchError` from `QwenVLTextEncoderConfig.from_model_on_disk` means the model directory lacks a `tokenizer/` subfolder. This config loader expects a diffusers-style Qwen VL text encoder layout with sibling `text_encoder/` and `tokenizer/` directories; without the tokenizer the model cannot be instantiated correctly, so the class refuses to match and lets the model manager try other configs.

Source

Thrown at invokeai/backend/model_manager/configs/qwen_vl_encoder.py:92

    def from_model_on_disk(cls, mod: ModelOnDisk, override_fields: dict[str, Any]) -> Self:
        raise_if_not_dir(mod)

        raise_for_override_fields(cls, override_fields)

        # Reject anything that looks like a full pipeline (those are matched as Main models).
        if (mod.path / "model_index.json").exists() or (mod.path / "transformer").exists():
            raise NotAMatchError(
                "directory looks like a full diffusers pipeline (has model_index.json or transformer folder), "
                "not a standalone Qwen VL encoder"
            )

        text_encoder_dir = mod.path / "text_encoder"
        tokenizer_dir = mod.path / "tokenizer"

        if not text_encoder_dir.is_dir():
            raise NotAMatchError("missing text_encoder/ subfolder")
        if not tokenizer_dir.is_dir():
            raise NotAMatchError("missing tokenizer/ subfolder")

        config_path = text_encoder_dir / "config.json"
        if not config_path.is_file():
            raise NotAMatchError(f"missing {config_path}")

        try:
            with open(config_path, "r", encoding="utf-8") as f:
                cfg = json.load(f)
        except (OSError, json.JSONDecodeError) as e:
            raise NotAMatchError(f"could not read text_encoder/config.json: {e}") from e

        class_name = cfg.get("_class_name")
        architectures = cfg.get("architectures") or []
        candidates = {class_name, *architectures} - {None}

        if not candidates & _RECOGNIZED_TEXT_ENCODER_CLASSES:
            raise NotAMatchError(
                f"text_encoder class is {sorted(candidates) or 'unknown'}, "

View on GitHub (pinned to 0b6a024f2f)

Solutions

1. Download or copy the tokenizer from the matching Qwen2.5-VL/Qwen2-VL HF repo into a `tokenizer/` subfolder next to `text_encoder/`
2. Verify the layout with `ls <model_dir>`: it must contain both `text_encoder/` and `tokenizer/` directories
3. Re-install the model through the InvokeAI model manager instead of copying files manually so the full repo structure is fetched
4. If the tokenizer exists under another name (e.g. `tokenizer_2/`), rename it to `tokenizer/`

Example fix

// before
models/qwen-vl/text_encoder/config.json
// after
models/qwen-vl/text_encoder/config.json
models/qwen-vl/tokenizer/tokenizer_config.json

from pathlib import Path

def is_valid_qwen_vl_dir(path: Path) -> bool:
    return (path / "text_encoder").is_dir() and (path / "tokenizer").is_dir()

def has_qwen_vl_layout(path: Path) -> bool:
    return path.is_dir() and (path / "text_encoder").is_dir() and (path / "tokenizer").is_dir()

from invokeai.backend.model_manager.configs.qwen_vl_encoder import QwenVLTextEncoderConfig
from invokeai.backend.model_manager import NotAMatchError

try:
    cfg = QwenVLTextEncoderConfig.from_model_on_disk(mod, override_fields)
except NotAMatchError:
    # fall through to other config classes / surface a clear install error
    ...

When it happens

Trigger: Calling model-install/probe APIs (which invoke `from_model_on_disk`) on a directory that contains `text_encoder/` but no `tokenizer/` subfolder, e.g. when someone manually downloads only the text_encoder portion of a Qwen2.5-VL/Qwen2-VL repo or moves/renames the tokenizer directory.

Common situations: Partial `huggingface-cli download` runs, hand-copied model folders that omit the tokenizer to save space, checkpoints converted from single-file format without regenerating a tokenizer, or a tokenizer stored under a nonstandard name like `tokenizer_2/`.

Related errors

• missing {config_path}
• Unsupported model source: '{url}'
• model does not look like a Qwen Image Edit LoRA
• model does not match Krea-2 LoRA heuristics (no complete lor
• Krea-2 LoRA has an incomplete lora_A/B (or lora_down/up) wei

AI-assisted analysis of invoke-ai/InvokeAI@0b6a024f2f (2026-08-29). Data as JSON: /api/errors/eb0988fd3b74a1be. Report an issue: GitHub.