karpathy/nanoGPT · error · ValueError

Unknown config key: {key}

Error message

Unknown config key: {key}

What it means

This error comes from nanoGPT's 'Poor Man's Configurator' (configurator.py), which is exec'd at the top of train.py/sample.py and mutates the script's globals(). For every --key=value command-line argument, it checks whether key already exists in globals(); if not, there is no config variable to override, so it raises ValueError('Unknown config key: ...'). In short: you passed a --flag whose name does not match any variable defined in the exec'd script or in any config file that ran before it.

Source

Thrown at configurator.py:47

    else:
        # assume it's a --key=value argument
        assert arg.startswith('--')
        key, val = arg.split('=')
        key = key[2:]
        if key in globals():
            try:
                # attempt to eval it it (e.g. if bool, number, or etc)
                attempt = literal_eval(val)
            except (SyntaxError, ValueError):
                # if that goes wrong, just use the string
                attempt = val
            # ensure the types match ok
            assert type(attempt) == type(globals()[key])
            # cross fingers
            print(f"Overriding: {key} = {attempt}")
            globals()[key] = attempt
        else:
            raise ValueError(f"Unknown config key: {key}")

View on GitHub (pinned to 3adf61e154)

Solutions

1. Check the exact variable name: open the config file you passed (or train.py's top-level assignments) and confirm the global exists, e.g. it must be `--batch_size=32` matching `batch_size = 64`.
2. Make sure you passed the base config file before the override, e.g. `python train.py config/train_shakespeare_char.py --batch_size=32` — without the config file the key may never be defined.
3. Fix typos, casing, and dashes: the parser splits on '=' and strips a leading '--', so `--batch-size=32` yields key `batch-size`, which will never match `batch_size`.
4. If you genuinely need a new knob, define it in the config file (or at the top of train.py) with a default value first, then override it on the command line.
5. Keep flags belonging to launchers (torchrun, accelerate, WandB env-style args) out of the positional/override section that configurator.py scans; set them as environment variables instead.

Example fix

# before (train.py never defines a global named `dtype`)
$ python train.py config/train_shakespeare_char.py --dtype=bfloat16
ValueError: Unknown config key: dtype

# after (dtype is a global that train.py/config defines)
$ python train.py config/train_shakespeare_char.py --dtype=bf16
# or add to the config file:
#   dtype = 'bf16'
# then run: python train.py config/train_shakespeare_char.py --dtype='bf16'

import sys
from ast import literal_eval

# run before launching the training script
config_globals = set()
for a in sys.argv[1:]:
    if a.endswith('.py') and '=' not in a:
        src = open(a).read()
        ns = {}
        exec(compile(src, a, 'exec'), ns)
        config_globals.update(k for k in ns if not k.startswith('_'))

bad = [a for a in sys.argv[1:]
       if a.startswith('--') and '=' in a and a[2:].split('=')[0] not in config_globals]
assert not bad, f'Unknown config keys: {bad}'

def is_known_config_key(key: str, known: set[str]) -> bool:
    """True if `key` matches a global defined by the script or its config files."""
    return key in known

# usage:
# key = '--batch_size=32'[2:].split('=')[0]
# if not is_known_config_key(key, known_globals): sys.exit(f'unknown key {key}')

# only if you exec configurator.py yourself / wrap script startup
try:
    exec(open('configurator.py').read())
except ValueError as e:
    if str(e).startswith('Unknown config key:'):
        key = str(e).split(':')[-1].strip()
        sys.exit(f'config error: {key!r} is not defined in the script or config file; add it to the config or fix the flag name')
    raise

When it happens

Trigger: Running e.g. `python train.py config/train_shakespeare_char.py --dtype=bfloat16` when the exec'd script (train.py plus the config file) never defines a global named `dtype`. Also triggered by typos (`--batch_size` vs `--batch-size`, which instead parses as key `batch-size`), by flags that only exist in a different config file (e.g. using a GPT-2 fine-tune flag against a base train script), or by arguments intended for another program (e.g. a launcher like torchrun) placed after the script's config args and containing '='.

Common situations: Copy-pasting a training command from a different nanoGPT version or model recipe whose variable names changed; assuming every variable in a config file is overridable when train.py only overridable if it also declares it in its own top-level code; passing WandB/SLURM/torchrun flags with an '=' to a script that exec's configurator.py; typos or case mismatches in flag names; forgetting to pass the base config file first (e.g. `python train.py --batch_size=32` with no config file, so almost nothing is in globals()).

AI-assisted analysis of karpathy/nanoGPT@3adf61e154 (2026-08-15). Data as JSON: /api/errors/d2db27cdf3f2e75e. Report an issue: GitHub.