huggingface/transformers · error · RuntimeError

Unknown type for trial {trial.__class__}

Error message

Unknown type for trial {trial.__class__}

What it means

hp_params(trial) converts a hyperparameter-search trial object into a params dict. It tries optuna (BaseTrial), then Ray Tune and W&B (both accept plain dicts); if the trial is none of these, it raises RuntimeError('Unknown type for trial ...'). In practice the most common cause is not an exotic trial type but a missing backend: if optuna is not installed, an optuna trial falls through every isinstance check and reaches the raise.

Source

Thrown at src/transformers/integrations/integration_utils.py:232

    return os.getenv("KUBEFLOW_TRAINER_SERVER_URL") is not None


def hp_params(trial):
    if is_optuna_available():
        import optuna

        if isinstance(trial, optuna.trial.BaseTrial):
            return trial.params

    if is_ray_tune_available():
        if isinstance(trial, dict):
            return trial

    if is_wandb_available():
        if isinstance(trial, dict):
            return trial

    raise RuntimeError(f"Unknown type for trial {trial.__class__}")


def run_hp_search_optuna(trainer, n_trials: int, direction: str, **kwargs) -> BestRun:
    import optuna
    from accelerate.utils.memory import release_memory

    if trainer.args.process_index == 0:

        def _objective(trial: optuna.Trial, checkpoint_dir=None):
            checkpoint = None
            if checkpoint_dir:
                for subdir in os.listdir(checkpoint_dir):
                    if subdir.startswith(PREFIX_CHECKPOINT_DIR):
                        checkpoint = os.path.join(checkpoint_dir, subdir)
            trainer.objective = None
            if trainer.args.world_size > 1:
                if trainer.args.parallel_mode != ParallelMode.DISTRIBUTED:
                    raise RuntimeError("only support DDP optuna HPO for ParallelMode.DISTRIBUTED currently.")

View on GitHub (pinned to a597f97485)

Solutions

1. pip install optuna (or ray[tune] / wandb) in the environment that executes the training loop and restart the run.
2. Verify the trial object you pass matches the installed backend: use optuna Trial objects only when optuna is importable.
3. For custom HPO frameworks, convert your trial to a plain dict of params before passing it to trainer.train(trial=...).
4. In multi-node setups, confirm identical package sets on all ranks (pip freeze diff) so worker ranks can recognize the trial.

Example fix

# before
trainer.train(trial=optuna_trial)  # optuna not installed -> RuntimeError

# after
subprocess.run([sys.executable, "-m", "pip", "install", "optuna"])
# restart training, then:
trainer.train(trial=optuna_trial)

import importlib.util

def backend_for_trial(trial):
    if importlib.util.find_spec("optuna") and type(trial).__module__.startswith("optuna"):
        return "optuna"
    if isinstance(trial, dict) and (importlib.util.find_spec("ray") or importlib.util.find_spec("wandb")):
        return "dict-compatible"
    raise RuntimeError(f"install the backend matching trial {type(trial)}")

import importlib.util

def trial_is_supported(trial) -> bool:
    if importlib.util.find_spec("optuna"):
        import optuna
        if isinstance(trial, optuna.trial.BaseTrial):
            return True
    if isinstance(trial, dict):
        return bool(importlib.util.find_spec("ray")) or bool(importlib.util.find_spec("wandb"))
    return False

try:
    params = hp_params(trial)
except RuntimeError as e:
    if "Unknown type for trial" in str(e):
        params = dict(trial) if hasattr(trial, "keys") else trial.params  # convert to plain dict
    else:
        raise

When it happens

Trigger: trainer.train(..., trial=<optuna trial>) or Trainer hyperparameter_search with a backend whose package is not importable in the current process (optuna/ray missing or a broken install), so the matching is_..._available() guard skips the only branch that would accept the trial; also genuinely unsupported trial objects (e.g. a SIGOPT or custom sweep trial passed by mistake).

Common situations: Running distributed training where the trial is broadcast to worker ranks whose environment lacks optuna; running the Trainer in a subprocess without the training extras; passing a W&B sweep config while neither wandb nor ray is installed.

Related errors

• only support DDP optuna HPO for ParallelMode.DISTRIBUTED cur
• Invalid return_format: {return_format}. Must be 'base64', 'd
• Error loading audio: {e}
• You need to install optimum-quanto in order to use KV cache
• You need to install `HQQ` in order to use KV cache quantizat

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/94609316d1f19e55. Report an issue: GitHub.