{"record":{"id":"753438e5f88460cc","repo":"TheAlgorithms/Python","slug":"invalid-magic-number-magic-in-mnist-image-file","errorCode":null,"errorMessage":"Invalid magic number {magic} in MNIST image file: {f.name}","messagePattern":"Invalid magic number (.+?) in MNIST image file: (.+?)","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"critical","filePath":"neural_network/input_data.py","lineNumber":65,"sourceCode":"def _extract_images(f):\n    \"\"\"Extract the images into a 4D uint8 numpy array [index, y, x, depth].\n\n    Args:\n      f: A file object that can be passed into a gzip reader.\n\n    Returns:\n      data: A 4D uint8 numpy array [index, y, x, depth].\n\n    Raises:\n      ValueError: If the bytestream does not start with 2051.\n\n    \"\"\"\n    print(\"Extracting\", f.name)\n    with gzip.GzipFile(fileobj=f) as bytestream:\n        magic = _read32(bytestream)\n        if magic != 2051:\n            msg = f\"Invalid magic number {magic} in MNIST image file: {f.name}\"\n            raise ValueError(msg)\n        num_images = _read32(bytestream)\n        rows = _read32(bytestream)\n        cols = _read32(bytestream)\n        buf = bytestream.read(rows * cols * num_images)\n        data = np.frombuffer(buf, dtype=np.uint8)\n        data = data.reshape(num_images, rows, cols, 1)\n        return data\n\n\n@deprecated(None, \"Please use tf.one_hot on tensors.\")\ndef _dense_to_one_hot(labels_dense, num_classes):\n    \"\"\"Convert class labels from scalars to one-hot vectors.\"\"\"\n    num_labels = labels_dense.shape[0]\n    index_offset = np.arange(num_labels) * num_classes\n    labels_one_hot = np.zeros((num_labels, num_classes))\n    labels_one_hot.flat[index_offset + labels_dense.ravel()] = 1\n    return labels_one_hot\n","sourceCodeStart":47,"sourceCodeEnd":83,"githubUrl":"https://github.com/TheAlgorithms/Python/blob/f5988cc09713315817df6a7e327e258013a94440/neural_network/input_data.py#L47-L83","documentation":"Raised by neural_network/input_data when the first 32-bit word of a gzipped MNIST image file is not the magic number 2051. Every IDX-format MNIST file starts with a magic number identifying its content: 2051 for images, 2049 for labels. Seeing anything else means the bytestream is not a valid MNIST image file — most commonly because the labels file was passed to the image reader, or the download is corrupt/truncated.","triggerScenarios":"Calling extract_images() on train-labels-idx1-ubyte.gz (magic 2049), on an HTML error page saved by a failed mirror download, on a truncated gzip file, or with a byte-swapped/unsupported IDX variant. The error message includes both the found magic value and the filename, which usually settles the diagnosis.","commonSituations":"Hard-coded MNIST URLs pointing at dead mirrors that now return HTML; mixing up the images and labels arguments; manually downloaded files renamed incorrectly; partial downloads from an interrupted transfer; newer alternative datasets (Fashion-MNIST is fine at 2051, but arbitrary IDX files are not).","solutions":["Check the reported magic number in the message: 2049 means you passed a labels file to the image reader — swap the arguments/files.","Verify the file integrity: gzip -t file.gz should pass; compare byte size against the official MNIST sizes (~9.9 MB images, ~29 KB labels).","Re-download from a known-good source (e.g. the original LeCun mirror list or a reputable mirror) and confirm the file starts with 00 00 08 03 in hex.","If the file is another IDX type, use the matching reader (extract_labels for magic 2049) or a general IDX parser."],"exampleFix":"# before\nwith open(\"train-labels-idx1-ubyte.gz\", \"rb\") as f:\n    images = extract_images(f)  # magic 2049 != 2051 -> ValueError\n\n# after\nwith open(\"train-images-idx3-ubyte.gz\", \"rb\") as f:\n    images = extract_images(f)  # correct file, magic 2051\nwith open(\"train-labels-idx1-ubyte.gz\", \"rb\") as f:\n    labels = extract_labels(f)","handlingStrategy":"validation","validationCode":"import gzip, struct\n\nMAGIC_IMAGES = 2051\n\ndef looks_like_idx_images(path: str) -> bool:\n    try:\n        with gzip.open(path, \"rb\") as f:\n            return struct.unpack(\">I\", f.read(4))[0] == MAGIC_IMAGES\n    except (OSError, struct.error):\n        return False\n\nif not looks_like_idx_images(path):\n    raise ValueError(f\"{path} is not a valid MNIST image file; re-download it\")\nwith open(path, \"rb\") as f:\n    images = extract_images(f)","typeGuard":"def is_idx_image_file(path: str) -> bool:\n    \"\"\"Guard: gzip file whose first big-endian u32 is the 2051 image magic.\"\"\"\n    try:\n        with gzip.open(path, \"rb\") as f:\n            return struct.unpack(\">I\", f.read(4))[0] == 2051\n    except (OSError, struct.error):\n        return False","tryCatchPattern":"try:\n    with open(path, \"rb\") as f:\n        images = extract_images(f)\nexcept ValueError as e:\n    if \"Invalid magic number\" in str(e):\n        # 2049 in the message means a labels file was passed; anything else suggests corruption\n        raise ValueError(f\"{path} is not the image file (wrong type or corrupt); re-download\") from e\n    raise","preventionTips":["Name and route files explicitly: images -> extract_images, labels -> extract_labels; never guess by position.","Verify downloads with checksums or at least gzip -t before first use.","Sniff the magic number (first 4 bytes, big-endian) when loading IDX files from untrusted paths.","Watch for mirrors returning HTML error pages with a 200 status — check content type and size."],"tags":["neural-network","mnist","data-loading","corrupt-file","valueerror"],"backgroundTag":null,"analyzedSha":"f5988cc09713315817df6a7e327e258013a94440","analyzedAt":"2026-08-14T17:30:07.041Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}