huggingface/transformers · error · ValueError

Invalid checkpoint path: '{checkpoint}' attempts to escape `

Error message

Invalid checkpoint path: '{checkpoint}' attempts to escape `dump_path`: {dump_path}

What it means

Security ValueError from convert_slow_checkpoint_to_fast: when a checkpoint name contains '/', the script joins it under dump_path and then verifies with Path.resolve().relative_to() that the resolved destination stays inside dump_path. A checkpoint whose organization component resolves outside dump_path (e.g. '../..', absolute paths, or symlink tricks) trips this guard.

Source

Thrown at src/transformers/convert_slow_tokenizers_checkpoints_to_fast.py:88

        for checkpoint in checkpoint_names:
            logger.info(f"Loading {tokenizer_class.__class__.__name__} {checkpoint}")

            # Load tokenizer
            tokenizer = tokenizer_class.from_pretrained(checkpoint, force_download=force_download)

            # Save fast tokenizer
            logger.info(f"Save fast tokenizer to {dump_path} with prefix {checkpoint} add_prefix {add_prefix}")

            # For organization names we create sub-directories
            if "/" in checkpoint:
                checkpoint_directory, checkpoint_prefix_name = checkpoint.split("/")
                dump_path_full = os.path.join(dump_path, checkpoint_directory)

                # Security check
                try:
                    Path(dump_path_full).resolve().relative_to(Path(dump_path).resolve())
                except ValueError:
                    raise ValueError(
                        f"Invalid checkpoint path: '{checkpoint}' attempts to escape `dump_path`: {dump_path}"
                    )

            elif add_prefix:
                checkpoint_prefix_name = checkpoint
                dump_path_full = dump_path
            else:
                checkpoint_prefix_name = None
                dump_path_full = dump_path

            logger.info(f"=> {dump_path_full} with prefix {checkpoint_prefix_name}, add_prefix {add_prefix}")

            if checkpoint in list(tokenizer.pretrained_vocab_files_map.values())[0]:
                file_path = list(tokenizer.pretrained_vocab_files_map.values())[0][checkpoint]
                next_char = file_path.split(checkpoint)[-1][0]
                if next_char == "/":
                    dump_path_full = os.path.join(dump_path_full, checkpoint_prefix_name)
                    checkpoint_prefix_name = None

View on GitHub (pinned to a597f97485)

Solutions

1. Use plain 'org/model' style checkpoint names with no '..' or absolute-path components.
2. Remove or repoint symlinks inside dump_path before running.
3. If orchestrating untrusted inputs, normalize and reject names containing '..' or starting with '/' upstream.

Example fix

// before
python utils/convert_slow_tokenizers_checkpoints_to_fast.py --checkpoints ../escape/model --dump_path /data/out  # ValueError

// after
python utils/convert_slow_tokenizers_checkpoints_to_fast.py --checkpoints valid_org/model --dump_path /data/out

from pathlib import Path
name = checkpoint_name
parts = name.split("/") if "/" in name else [name]
if name.startswith(("/", "\\")) or ".." in parts or Path(name).is_absolute():
    raise ValueError(f"checkpoint name {name!r} must be a relative org/model style path")

def checkpoint_name_is_safe(checkpoint: str, dump_path: str) -> bool:
    from pathlib import Path
    full = Path(dump_path) / checkpoint.split("/")[0]
    try:
        full.resolve().relative_to(Path(dump_path).resolve())
        return True
    except ValueError:
        return False

When it happens

Trigger: Passing a checkpoint name like "../evil/model" so dump_path_full resolves above dump_path; a symlinked organization directory inside dump_path that points elsewhere and changes the resolved path.

Common situations: Feeding untrusted checkpoint lists to the batch conversion tool; paths with '..' segments; symlinked caches under the dump directory.

Related errors

• Unrecognized tokenizer name, should be one of {list(TOKENIZE
• {error_message} requires the protobuf library but it was not
• You're trying to run a `Unigram` model but you're file was t
• `tiktoken` is required to read a `tiktoken` file. Install it
• Converting from SentencePiece and Tiktoken failed, if a conv

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/cbd6aefbcd02864d. Report an issue: GitHub.