TheAlgorithms/Python · error · ValueError

Invalid magic number {magic} in MNIST label file: {f.name}

Error message

Invalid magic number {magic} in MNIST label file: {f.name}

What it means

Raised by _extract_labels while parsing a gzipped MNIST label file when the first 32-bit field (the magic number) is not 2049. The MNIST IDX format prefixes every file with a magic number that identifies its content type, so this check distinguishes label files (2049) from image files (2051). A mismatch means the byte stream being parsed is not a valid MNIST label file.

Source

Thrown at neural_network/input_data.py:105

    """Extract the labels into a 1D uint8 numpy array [index].

    Args:
      f: A file object that can be passed into a gzip reader.
      one_hot: Does one hot encoding for the result.
      num_classes: Number of classes for the one hot encoding.

    Returns:
      labels: a 1D uint8 numpy array.

    Raises:
      ValueError: If the bystream doesn't start with 2049.
    """
    print("Extracting", f.name)
    with gzip.GzipFile(fileobj=f) as bytestream:
        magic = _read32(bytestream)
        if magic != 2049:
            msg = f"Invalid magic number {magic} in MNIST label file: {f.name}"
            raise ValueError(msg)
        num_items = _read32(bytestream)
        buf = bytestream.read(num_items)
        labels = np.frombuffer(buf, dtype=np.uint8)
        if one_hot:
            return _dense_to_one_hot(labels, num_classes)
        return labels


class _DataSet:
    """Container class for a _DataSet (deprecated).

    THIS CLASS IS DEPRECATED.
    """

    @deprecated(
        None,
        "Please use alternatives such as official/mnist/_DataSet.py"
        " from tensorflow/models.",

View on GitHub (pinned to f5988cc097)

Solutions

1. Delete the MNIST files in train_dir and let _maybe_download re-fetch them from a known-good mirror, then retry
2. Verify the file is valid gzip: gunzip -t train-labels-idx1-ubyte.gz
3. Check the first bytes after decompression: the 4-byte big-endian value must be 2049 (0x00000801); if it is 2051 you have an image file where a label file was expected
4. If using a custom SOURCE_URL, confirm the label-file URL actually points to the idx1 (labels) archive, not the idx3 (images) archive

Example fix

# before: wrong file was placed where labels are expected
with gfile.Open('train-images-idx3-ubyte.gz', 'rb') as f:
    labels = _extract_labels(f)  # ValueError: magic is 2051

# after: open the labels archive
with gfile.Open('train-labels-idx1-ubyte.gz', 'rb') as f:
    labels = _extract_labels(f)

import gzip, struct

def is_valid_mnist_label_file(path) -> bool:
    try:
        with gzip.open(path, 'rb') as f:
            return struct.unpack('>I', f.read(4))[0] == 2049
    except (OSError, EOFError):
        return False

try:
    labels = _extract_labels(f, one_hot=True)
except ValueError as e:
    if 'magic number' in str(e):
        raise RuntimeError(f'Corrupt MNIST label file {f.name}; re-download it') from e
    raise

When it happens

Trigger: Calling read_data_sets (which opens the downloaded train-labels/t10k-labels files) when the label file is corrupted, truncated, empty, or is actually an image file (magic 2051) opened by mistake. Also triggered by pointing SOURCE_URL/train_dir at a directory containing wrong or partial files.

Common situations: Interrupted or partial downloads, a mirror that serves an HTML error page instead of the .gz file, swapped image/label filenames in a custom mirror, or hand-crafted IDX files with an incorrect magic header.

Related errors

• Invalid magic number {magic} in MNIST image file: {f.name}
• Invalid image dtype {dtype!r}, expected uint8 or float32
• Validation size should be between 0 and {len(train_images)}.

AI-assisted analysis of TheAlgorithms/Python@f5988cc097 (2026-08-14). Data as JSON: /api/errors/a56123963e3d6290. Report an issue: GitHub.