huggingface/open-r1 · error

Callback {callback_name} not found in CALLBACKS.

Error message

Callback {callback_name} not found in CALLBACKS.

What it means

get_callbacks resolves each callback name listed in train_config.callbacks against the module-level CALLBACKS registry dict and instantiates it with model_config. An unknown name is not silently skipped — it raises this ValueError so misconfigured callback names fail fast at startup.

Source

Thrown at src/open_r1/utils/callbacks.py:89

                dummy_config.benchmarks = args.benchmarks

                def run_benchmark_callback(_):
                    print(f"Checkpoint {global_step} pushed to hub.")
                    run_benchmark_jobs(dummy_config, self.model_config)

                future.add_done_callback(run_benchmark_callback)


CALLBACKS = {
    "push_to_hub_revision": PushToHubRevisionCallback,
}


def get_callbacks(train_config, model_config) -> List[TrainerCallback]:
    callbacks = []
    for callback_name in train_config.callbacks:
        if callback_name not in CALLBACKS:
            raise ValueError(f"Callback {callback_name} not found in CALLBACKS.")
        callbacks.append(CALLBACKS[callback_name](model_config))

    return callbacks

View on GitHub (pinned to 1416fa0cf2)

Solutions

1. Correct the name in your config to one present in CALLBACKS (check src/open_r1/utils/callbacks.py).
2. If it's a custom callback, import it in callbacks.py and register it in the CALLBACKS dict.
3. Remove the entry from train_config.callbacks if you don't need it.

Example fix

// before
# config.yaml
callbacks: [tensorboard_callback]   # not registered
// after
callbacks: [rich_progress_callback]  # a key that exists in CALLBACKS

from open_r1.utils.callbacks import CALLBACKS
unknown = [c for c in train_config.callbacks if c not in CALLBACKS]
if unknown:
    raise SystemExit(f"Unknown callbacks {unknown}; available: {sorted(CALLBACKS)}")

def callbacks_registered(names, registry) -> bool:
    return all(isinstance(n, str) and n in registry for n in names)

try:
    cbs = get_callbacks(train_config, model_config)
except ValueError as e:
    if "not found in CALLBACKS" in str(e):
        sys.exit(f"Fix config callbacks; valid options: {sorted(CALLBACKS)}")
    raise

When it happens

Trigger: train_config.callbacks contains a name (e.g. from a YAML 'callbacks: [my_callback]') that is not a key in CALLBACKS (e.g. not one of the built-ins like 'rich_progress_callback').

Common situations: Typo in the callback name in a training YAML; assuming arbitrary custom TrainerCallback classes can be referenced by name without registering them; copying configs between repos with different registries.

Related errors

• Either `dataset_name` or `dataset_mixture` must be provided
• dataset_mixture must be a dictionary with a 'datasets' key.
• 'datasets' must be a list of dataset configurations
• Column names must be consistent across all dataset configura
• MorphCloud API key not found. Please set the MORPH_API_KEY e

AI-assisted analysis of huggingface/open-r1@1416fa0cf2 (2026-08-30). Data as JSON: /api/errors/3a24350ecf5de56f. Report an issue: GitHub.