oobabooga/textgen · error · FileNotFoundError

No .gguf models found in directory: {path}

Error message

No .gguf models found in directory: {path}

What it means

Error "No .gguf models found in directory: {path}" thrown in oobabooga/textgen.

Source

Thrown at modules/models_settings.py:59

            model_settings,
            hf_quant_method=quant_method
        )

    # Default bos/eos tokens (may be overridden by GGUF metadata or tokenizer_config.json)
    shared.bos_token = '<s>'
    shared.eos_token = '</s>'

    # GGUF metadata
    if model_settings['loader'] == 'llama.cpp':
        path = model_path
        if path.is_file():
            model_file = path
        else:
            gguf_files = list(path.glob('*.gguf'))
            if not gguf_files:
                error_msg = f"No .gguf models found in directory: {path}"
                logger.error(error_msg)
                raise FileNotFoundError(error_msg)

            model_file = gguf_files[0]

        metadata = load_gguf_metadata_with_cache(model_file)

        for k in metadata:
            if k.endswith('.context_length'):
                model_settings['ctx_size'] = 0
                model_settings['truncation_length_info'] = metadata[k]
            elif k.endswith('.block_count'):
                model_settings['gpu_layers'] = -1
                model_settings['max_gpu_layers'] = metadata[k] + 1
            elif k.endswith('.nextn_predict_layers') and metadata[k] > 0:
                model_settings['spec_type'] = 'draft-mtp'

        if 'tokenizer.chat_template' in metadata:
            template = metadata['tokenizer.chat_template']
            if 'tokenizer.ggml.eos_token_id' in metadata:

View on GitHub (pinned to ed888c71f2)

Solutions

1. Place at least one .gguf model file in the reported directory.
2. Point the model directory setting at the folder that actually contains your .gguf files.

When it happens

Trigger: Raised when scanning a model directory for llama.cpp models and no .gguf files are found. Triggers when the model folder path is wrong, the model was not downloaded, or the folder only contains sharded/non-GGUF weights.

Common situations: See trigger scenarios.

AI-assisted analysis of oobabooga/textgen@ed888c71f2 (2026-08-15). Data as JSON: /api/errors/d77a1c6e643b2e80. Report an issue: GitHub.