{"record":{"id":"eb0988fd3b74a1be","repo":"invoke-ai/InvokeAI","slug":"missing-tokenizer-subfolder","errorCode":null,"errorMessage":"missing tokenizer/ subfolder","messagePattern":"missing tokenizer/ subfolder","errorType":"validation","errorClass":"NotAMatchError","httpStatus":null,"severity":"error","filePath":"invokeai/backend/model_manager/configs/qwen_vl_encoder.py","lineNumber":92,"sourceCode":"    def from_model_on_disk(cls, mod: ModelOnDisk, override_fields: dict[str, Any]) -> Self:\n        raise_if_not_dir(mod)\n\n        raise_for_override_fields(cls, override_fields)\n\n        # Reject anything that looks like a full pipeline (those are matched as Main models).\n        if (mod.path / \"model_index.json\").exists() or (mod.path / \"transformer\").exists():\n            raise NotAMatchError(\n                \"directory looks like a full diffusers pipeline (has model_index.json or transformer folder), \"\n                \"not a standalone Qwen VL encoder\"\n            )\n\n        text_encoder_dir = mod.path / \"text_encoder\"\n        tokenizer_dir = mod.path / \"tokenizer\"\n\n        if not text_encoder_dir.is_dir():\n            raise NotAMatchError(\"missing text_encoder/ subfolder\")\n        if not tokenizer_dir.is_dir():\n            raise NotAMatchError(\"missing tokenizer/ subfolder\")\n\n        config_path = text_encoder_dir / \"config.json\"\n        if not config_path.is_file():\n            raise NotAMatchError(f\"missing {config_path}\")\n\n        try:\n            with open(config_path, \"r\", encoding=\"utf-8\") as f:\n                cfg = json.load(f)\n        except (OSError, json.JSONDecodeError) as e:\n            raise NotAMatchError(f\"could not read text_encoder/config.json: {e}\") from e\n\n        class_name = cfg.get(\"_class_name\")\n        architectures = cfg.get(\"architectures\") or []\n        candidates = {class_name, *architectures} - {None}\n\n        if not candidates & _RECOGNIZED_TEXT_ENCODER_CLASSES:\n            raise NotAMatchError(\n                f\"text_encoder class is {sorted(candidates) or 'unknown'}, \"","sourceCodeStart":74,"sourceCodeEnd":110,"githubUrl":"https://github.com/invoke-ai/InvokeAI/blob/0b6a024f2ff6a86bfb953dcdb9cc504ef7397a06/invokeai/backend/model_manager/configs/qwen_vl_encoder.py#L74-L110","documentation":"`NotAMatchError` from `QwenVLTextEncoderConfig.from_model_on_disk` means the model directory lacks a `tokenizer/` subfolder. This config loader expects a diffusers-style Qwen VL text encoder layout with sibling `text_encoder/` and `tokenizer/` directories; without the tokenizer the model cannot be instantiated correctly, so the class refuses to match and lets the model manager try other configs.","triggerScenarios":"Calling model-install/probe APIs (which invoke `from_model_on_disk`) on a directory that contains `text_encoder/` but no `tokenizer/` subfolder, e.g. when someone manually downloads only the text_encoder portion of a Qwen2.5-VL/Qwen2-VL repo or moves/renames the tokenizer directory.","commonSituations":"Partial `huggingface-cli download` runs, hand-copied model folders that omit the tokenizer to save space, checkpoints converted from single-file format without regenerating a tokenizer, or a tokenizer stored under a nonstandard name like `tokenizer_2/`.","solutions":["Download or copy the tokenizer from the matching Qwen2.5-VL/Qwen2-VL HF repo into a `tokenizer/` subfolder next to `text_encoder/`","Verify the layout with `ls <model_dir>`: it must contain both `text_encoder/` and `tokenizer/` directories","Re-install the model through the InvokeAI model manager instead of copying files manually so the full repo structure is fetched","If the tokenizer exists under another name (e.g. `tokenizer_2/`), rename it to `tokenizer/`"],"exampleFix":"// before\nmodels/qwen-vl/text_encoder/config.json\n// after\nmodels/qwen-vl/text_encoder/config.json\nmodels/qwen-vl/tokenizer/tokenizer_config.json","handlingStrategy":"validation","validationCode":"from pathlib import Path\n\ndef is_valid_qwen_vl_dir(path: Path) -> bool:\n    return (path / \"text_encoder\").is_dir() and (path / \"tokenizer\").is_dir()","typeGuard":"def has_qwen_vl_layout(path: Path) -> bool:\n    return path.is_dir() and (path / \"text_encoder\").is_dir() and (path / \"tokenizer\").is_dir()","tryCatchPattern":"from invokeai.backend.model_manager.configs.qwen_vl_encoder import QwenVLTextEncoderConfig\nfrom invokeai.backend.model_manager import NotAMatchError\n\ntry:\n    cfg = QwenVLTextEncoderConfig.from_model_on_disk(mod, override_fields)\nexcept NotAMatchError:\n    # fall through to other config classes / surface a clear install error\n    ...","preventionTips":["Always download the entire HF repo folder (tokenizer included), never weights only","Use `--include \"text_encoder/*\" \"tokenizer/*\"` when doing selective downloads","Check directory layout with ls before installing manually","Let the model manager perform installs instead of hand-copying files"],"tags":["model-install","qwen-vl","directory-layout"],"backgroundTag":"missing-model-subfolder","analyzedSha":"0b6a024f2ff6a86bfb953dcdb9cc504ef7397a06","analyzedAt":"2026-08-29T04:46:49.967Z","schemaVersion":2},"datasetVersion":"2026-08-29T07:17:48.351Z"}