karpathy/nanochat · error · ValueError

Unknown dataset tag: {dataset_tag}

Error message

Unknown dataset tag: {dataset_tag}

What it means

dev/repackage_data_reference.py validates the module-level `dataset_tag` variable against the only two supported values: 'fineweb_edu' and 'climbmix'. Any other string falls through the if/elif chain to the else branch and raises ValueError. The script is documentation-only (it describes how the two hosted datasets were prepared), so this is a guard against editing the tag to a dataset that has no preparation recipe.

Source

Thrown at dev/repackage_data_reference.py:63

    }
    output_dirname = "fineweb_edu"
    data_column_name = "text"
    tokenizer = None
    upload_tag = "fineweb-edu-100b-shuffle"

elif dataset_tag == "climbmix":
    import tiktoken # the ClimbMix data is stored tokenized with GPT-2 tokenizer
    dataset_kwargs = {
        "path": "nvidia/Nemotron-ClimbMix",
        "split": "train",
    }
    output_dirname = "climbmix"
    data_column_name = "tokens"
    tokenizer = tiktoken.encoding_for_model("gpt-2")
    upload_tag = "climbmix-400b-shuffle"

else:
    raise ValueError(f"Unknown dataset tag: {dataset_tag}")

# Source dataset
ds = load_dataset(**dataset_kwargs)

# Shuffle to scramble the order
ds = ds.shuffle(seed=42)
ndocs = len(ds) # total number of documents to process
print(f"Total number of documents: {ndocs}")

# Repackage into parquet files
output_dir = f"/home/ubuntu/.cache/nanochat/base_data_{output_dirname}"
os.makedirs(output_dir, exist_ok=True)

# Write to parquet files
chars_per_shard = 250_000_000
row_group_size = 1024 # HF uses 1000 but we use multiple of 2, nicer for distributed data loader later
shard_docs = []
shard_index = 0

View on GitHub (pinned to 92d63d4e8b)

Solutions

1. Set dataset_tag to one of the supported literals: 'fineweb_edu' or 'climbmix'.
2. If you need a new dataset, add a new `elif dataset_tag == "<your_tag>":` block that fills in dataset_kwargs, output_dirname, data_column_name, tokenizer, and upload_tag before the else branch.
3. Check for typos/casing — the comparison is exact and case-sensitive.

Example fix

// before
dataset_tag = "fineweb"

// after
dataset_tag = "fineweb_edu"  # or "climbmix"

SUPPORTED_DATASET_TAGS = {"fineweb_edu", "climbmix"}
assert dataset_tag in SUPPORTED_DATASET_TAGS, f"dataset_tag must be one of {SUPPORTED_DATASET_TAGS}, got {dataset_tag!r}"

try:
    run_repackage(dataset_tag)
except ValueError as e:
    if "Unknown dataset tag" in str(e):
        raise SystemExit(f"Unsupported dataset tag {dataset_tag!r}; supported: fineweb_edu, climbmix")
    raise

When it happens

Trigger: Setting `dataset_tag` at line 36 to anything other than the literal strings 'fineweb_edu' or 'climbmix' (e.g. 'fineweb', 'ClimbMix' with different casing, or a new dataset name) before running the script.

Common situations: Developers copying this reference script to prepare a new dataset and changing only the tag without adding a matching elif branch; typos or casing mismatches ('FineWebEdu', 'climb_mix').

Related errors

• Unsupported task type: {task_type}
• Only 'tensorwise' recipe is supported, got '{recipe_name}'.
• Unknown optimizer kind: {group['kind']}
• Invalid input type: {type(text)}
• Unknown part type: {part['type']}

AI-assisted analysis of karpathy/nanochat@92d63d4e8b (2026-08-15). Data as JSON: /api/errors/b185c6602c5a5e89. Report an issue: GitHub.