huggingface/transformers · error · OSError

`{pretrained_model_name_or_path}` does not contain a `custom

Error message

`{pretrained_model_name_or_path}` does not contain a `custom_generate` subdirectory with a `generate.py` file, can't load the custom generate function.

What it means

OSError raised while loading a custom generate function: the hub/local repo at pretrained_model_name_or_path must contain custom_generate/generate.py, and get_cached_module_file failed to fetch it (a .no_exist cache entry is written to avoid re-requesting). The error is re-raised with a wrapper message, so the underlying network/404 cause is swallowed into this generic text.

Source

Thrown at src/transformers/generation/utils.py:489

                should only be set to `True` for repositories you trust and in which you have read the code, as it will
                execute code present on the Hub on your local machine.
            **kwargs:
                Additional keyword arguments for remote code loading.

        Raises:
            OSError: If `pretrained_model_name_or_path` does not contain a `custom_generate` subdirectory.

        Returns:
            A callable that can be used to generate text.
        """
        # Fetches the generate.py file from the model repo. If it doesn't exist, a file in `.no_exist` cache directory
        # is created (preventing future hub requests), and an OSError is raised.
        try:
            module = get_cached_module_file(
                pretrained_model_name_or_path, module_file="custom_generate/generate.py", **kwargs
            )
        except OSError:
            raise OSError(
                f"`{pretrained_model_name_or_path}` does not contain a `custom_generate` subdirectory with a "
                "`generate.py` file, can't load the custom generate function."
            )

        # Loading a custom generate function executes the repository's `custom_generate/generate.py`
        # (via `get_class_in_module` below), so it is remote code and must be gated by
        # `trust_remote_code` -- including when it is loaded from a local directory. `from_pretrained`
        # likewise requires `trust_remote_code` for custom modeling code that lives in a local repo;
        # treating a local path as trusted here previously let `custom_generate/generate.py` run with
        # no opt-in.
        error_message = (
            f"The repository `{pretrained_model_name_or_path}` contains custom generation code that will override "
            "the default `generate` method."
        )
        resolve_trust_remote_code(
            trust_remote_code,
            pretrained_model_name_or_path,
            has_local_code=False,

View on GitHub (pinned to a597f97485)

Solutions

1. Check the repo on the Hub for custom_generate/generate.py; if missing, the model has no custom generate function — don't request it.
2. If it should exist, verify the exact repo id/revision and re-fetch (delete the .no_exist cache entry in the local hub cache for that file).
3. For local dirs, ensure <dir>/custom_generate/generate.py exists and pass trust_remote_code=True.
4. Retry with network access if the initial fetch failed offline.

Example fix

# before
fn = load_custom_generate("user/model-x", trust_remote_code=True)  # repo has no custom_generate/

# after
# upload custom_generate/generate.py to the repo first, then:
fn = load_custom_generate("user/model-x", trust_remote_code=True)

from huggingface_hub import file_exists

if not file_exists(repo_id, "custom_generate/generate.py", revision=revision):
    raise FileNotFoundError(f"{repo_id} has no custom_generate/generate.py")

try:
    fn = load_custom_generate(repo_id, trust_remote_code=True)
except OSError as e:
    if "custom_generate" in str(e):
        logger.info("no custom generate code on %s; using default generate()", repo_id)
        fn = None
    else:
        raise

When it happens

Trigger: Calling load_custom_generate(model_id) (or a generate() path that resolves custom generation code) on a repo that lacks the custom_generate/ subdirectory; typo in the repo id; offline/hub outage causing the file fetch to fail; the file exists only on a different revision/branch.

Common situations: Using a community repo that never shipped custom_generate/; pointing at a fine-tune saved locally without copying the custom_generate folder; HF_HUB_OFFLINE=1 with a cold cache.

Related errors

• Can't load the configuration of '{pretrained_model_name_or_p
• `inputs`: {inputs}` were passed alongside {input_name} which
• You passed `inputs_embeds` to `.generate()`, but the model c
• You passed `inputs_embeds` and `input_ids` to `.generate()`.
• `bos_token_id` has to be defined when no `input_ids` are pro

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/c78888013727cdfa. Report an issue: GitHub.