{"record":{"id":"b9489cbfd23aa20e","repo":"zylon-ai/private-gpt","slug":"found-len-matched-files-files-matching-the-patt-b9489c","errorCode":null,"errorMessage":"Found {len(matched_files)} files matching the pattern: {file_pattern}. Make sure that a Mistral tokenizer is present in {files}.","messagePattern":"Found (.+?) files matching the pattern: (.+?)\\. Make sure that a Mistral tokenizer is present in (.+?)\\.","errorType":"exception","errorClass":"OSError","httpStatus":null,"severity":"error","filePath":"private_gpt/components/llm/tokenizers/mistral.py","lineNumber":196,"sourceCode":"\n    return []\n\n\ndef find_tokenizer_file(files: list[str]) -> str:\n    \"\"\"Find the Mistral tokenizer file from a list of repository files.\"\"\"\n    file_pattern = re.compile(\n        r\"^tokenizer\\.model\\.v.*$|^tekken\\.json$|^tokenizer\\.mm\\.model\\.v.*$\"\n    )\n\n    matched_files = [file for file in files if file_pattern.match(file)]\n    if len(matched_files) > 1:\n        raise OSError(\n            f\"Found {len(matched_files)} files matching the \"\n            f\"pattern: {file_pattern}. Make sure only one Mistral \"\n            f\"tokenizer is present in {files}.\"\n        )\n    elif len(matched_files) == 0:\n        raise OSError(\n            f\"Found {len(matched_files)} files matching the \"\n            f\"pattern: {file_pattern}. Make sure that a Mistral \"\n            f\"tokenizer is present in {files}.\"\n        )\n\n    return matched_files[0]\n\n\ndef _prepare_apply_chat_template_tools_and_messages(\n    messages: list[dict[str, Any]],\n    tools: list[dict[str, Any]] | None = None,\n    continue_final_message: bool = False,\n    add_generation_prompt: bool = False,\n) -> tuple[list[dict[str, Any]], list[dict[str, Any]] | None]:\n    \"\"\"Prepare messages and tools for Mistral's chat template format.\n\n    Handles validation and formatting of messages and tools to ensure\n    compatibility with Mistral's requirements.","sourceCodeStart":178,"sourceCodeEnd":214,"githubUrl":"https://github.com/zylon-ai/private-gpt/blob/4a030776a31a901ad80b1bf4d7faa2c1a367efbb/private_gpt/components/llm/tokenizers/mistral.py#L178-L214","documentation":"The companion case of the ambiguous-file error: find_tokenizer_file matched zero files against ^tokenizer\\.model\\.v.*$|^tekken\\.json$|^tokenizer\\.mm\\.model\\.v.*$ in the given directory list, so there is no Mistral tokenizer to load and it raises OSError.","triggerScenarios":"MistralTokenizer.from_pretrained(<local dir>) on a directory containing only HF-style files (tokenizer.json / tokenizer_config.json) and no Mistral-format file; also triggered on HF Hub repos (list_repo_files) that lack the expected file.","commonSituations":"Pointing the mistral tokenizer mode at a generic HF-converted model; incomplete downloads; directory listing that includes files under a subfolder (pattern is anchored to the bare filename).","solutions":["Check the directory/repo file list in the error and confirm which tokenizer file format the model actually ships.","Use a model repo that contains a Mistral tokenizer file (tekken.json or tokenizer.model.vN), e.g. official mistralai repos.","If the model only has HF tokenizer.json, use tokenizer_mode='huggingface' instead of 'mistral'."],"exampleFix":"# before: model_dir only has tokenizer.json (HF format)\ntok = MistralTokenizer.from_pretrained('model_dir')\n\n# after\ntok = TokenizerRegistry.get_tokenizer('huggingface', model_id='model_dir')","handlingStrategy":"validation","validationCode":"def dir_has_mistral_tokenizer(path: str) -> bool:\n    import os, re\n    pat = re.compile(r'^tokenizer\\.model\\.v.*$|^tekken\\.json$|^tokenizer\\.mm\\.model\\.v.*$')\n    return any(pat.match(f) for f in os.listdir(path))","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Prefer official mistralai repos when using tokenizer_mode='mistral'.","Check the model card/file list for tekken.json or tokenizer.model.vN before selecting the mode.","Use tokenizer_mode='huggingface' for repos that only ship HF-format tokenizer.json."],"tags":["mistral","tokenizer","local-model","model-format"],"backgroundTag":null,"analyzedSha":"4a030776a31a901ad80b1bf4d7faa2c1a367efbb","analyzedAt":"2026-08-15T03:51:26.951Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}