TheAlgorithms/Python · critical · ValueError

Invalid magic number {magic} in MNIST image file: {f.name}

Error message

Invalid magic number {magic} in MNIST image file: {f.name}

What it means

Raised by neural_network/input_data when the first 32-bit word of a gzipped MNIST image file is not the magic number 2051. Every IDX-format MNIST file starts with a magic number identifying its content: 2051 for images, 2049 for labels. Seeing anything else means the bytestream is not a valid MNIST image file — most commonly because the labels file was passed to the image reader, or the download is corrupt/truncated.

Source

Thrown at neural_network/input_data.py:65

def _extract_images(f):
    """Extract the images into a 4D uint8 numpy array [index, y, x, depth].

    Args:
      f: A file object that can be passed into a gzip reader.

    Returns:
      data: A 4D uint8 numpy array [index, y, x, depth].

    Raises:
      ValueError: If the bytestream does not start with 2051.

    """
    print("Extracting", f.name)
    with gzip.GzipFile(fileobj=f) as bytestream:
        magic = _read32(bytestream)
        if magic != 2051:
            msg = f"Invalid magic number {magic} in MNIST image file: {f.name}"
            raise ValueError(msg)
        num_images = _read32(bytestream)
        rows = _read32(bytestream)
        cols = _read32(bytestream)
        buf = bytestream.read(rows * cols * num_images)
        data = np.frombuffer(buf, dtype=np.uint8)
        data = data.reshape(num_images, rows, cols, 1)
        return data


@deprecated(None, "Please use tf.one_hot on tensors.")
def _dense_to_one_hot(labels_dense, num_classes):
    """Convert class labels from scalars to one-hot vectors."""
    num_labels = labels_dense.shape[0]
    index_offset = np.arange(num_labels) * num_classes
    labels_one_hot = np.zeros((num_labels, num_classes))
    labels_one_hot.flat[index_offset + labels_dense.ravel()] = 1
    return labels_one_hot

View on GitHub (pinned to f5988cc097)

Solutions

1. Check the reported magic number in the message: 2049 means you passed a labels file to the image reader — swap the arguments/files.
2. Verify the file integrity: gzip -t file.gz should pass; compare byte size against the official MNIST sizes (~9.9 MB images, ~29 KB labels).
3. Re-download from a known-good source (e.g. the original LeCun mirror list or a reputable mirror) and confirm the file starts with 00 00 08 03 in hex.
4. If the file is another IDX type, use the matching reader (extract_labels for magic 2049) or a general IDX parser.

Example fix

# before
with open("train-labels-idx1-ubyte.gz", "rb") as f:
    images = extract_images(f)  # magic 2049 != 2051 -> ValueError

# after
with open("train-images-idx3-ubyte.gz", "rb") as f:
    images = extract_images(f)  # correct file, magic 2051
with open("train-labels-idx1-ubyte.gz", "rb") as f:
    labels = extract_labels(f)

import gzip, struct

MAGIC_IMAGES = 2051

def looks_like_idx_images(path: str) -> bool:
    try:
        with gzip.open(path, "rb") as f:
            return struct.unpack(">I", f.read(4))[0] == MAGIC_IMAGES
    except (OSError, struct.error):
        return False

if not looks_like_idx_images(path):
    raise ValueError(f"{path} is not a valid MNIST image file; re-download it")
with open(path, "rb") as f:
    images = extract_images(f)

def is_idx_image_file(path: str) -> bool:
    """Guard: gzip file whose first big-endian u32 is the 2051 image magic."""
    try:
        with gzip.open(path, "rb") as f:
            return struct.unpack(">I", f.read(4))[0] == 2051
    except (OSError, struct.error):
        return False

try:
    with open(path, "rb") as f:
        images = extract_images(f)
except ValueError as e:
    if "Invalid magic number" in str(e):
        # 2049 in the message means a labels file was passed; anything else suggests corruption
        raise ValueError(f"{path} is not the image file (wrong type or corrupt); re-download") from e
    raise

When it happens

Trigger: Calling extract_images() on train-labels-idx1-ubyte.gz (magic 2049), on an HTML error page saved by a failed mirror download, on a truncated gzip file, or with a byte-swapped/unsupported IDX variant. The error message includes both the found magic value and the filename, which usually settles the diagnosis.

Common situations: Hard-coded MNIST URLs pointing at dead mirrors that now return HTML; mixing up the images and labels arguments; manually downloaded files renamed incorrectly; partial downloads from an interrupted transfer; newer alternative datasets (Fashion-MNIST is fine at 2051, but arbitrary IDX files are not).

Related errors

• Expected a_coeffs to have {self.order + 1} elements for {sel
• n must not be negative
• Candidates list should not be empty
• Depth cannot be less than 0
• Invalid input needed_sum must be between 1 and 1000, power b

AI-assisted analysis of TheAlgorithms/Python@f5988cc097 (2026-08-14). Data as JSON: /api/errors/753438e5f88460cc. Report an issue: GitHub.