prestodb/presto · critical · PrestoException

STORAGE_ERROR

STORAGE_ERROR

Error message

Disk page checksum does not match. Data seems to be corrupted on disk for file

What it means

Thrown from PrestoSparkDiskPageInput.loadBroadcastTable() when the checksum computed after deserializing a broadcast table page from temp storage does not match the checksum recorded in the PrestoSparkStorageHandle. This means the spilled file's bytes changed between write and read — data corruption on the storage layer.

Source

Thrown at presto-spark-base/src/main/java/com/facebook/presto/spark/execution/PrestoSparkDiskPageInput.java:170

                checksum.reset();
                PrestoSparkStorageHandle prestoSparkStorageHandle = (PrestoSparkStorageHandle) taskFileInfo;
                TempStorageHandle tempStorageHandle = tempStorage.deserialize(prestoSparkStorageHandle.getSerializedStorageHandle());
                log.info("Reading path: " + tempStorageHandle.toString());
                try (InputStream inputStream = tempStorage.open(tempDataOperationContext, tempStorageHandle);
                        InputStreamSliceInput inputStreamSliceInput = new InputStreamSliceInput(inputStream)) {
                    Iterator<SerializedPage> pagesIterator = readSerializedPages(inputStreamSliceInput);
                    while (pagesIterator.hasNext()) {
                        SerializedPage serializedPage = pagesIterator.next();
                        checksum.update(serializedPage.getSlice().byteArray(), serializedPage.getSlice().byteArrayOffset(), serializedPage.getSlice().length());
                        Page deserializedPage = pagesSerde.deserialize(serializedPage);
                        pages.add(deserializedPage);
                        stagingBroadcastTableSizeInBytes += deserializedPage.getRetainedSizeInBytes();
                    }
                    updateMemory.update();
                }

                if (checksum.getValue() != prestoSparkStorageHandle.getChecksum()) {
                    throw new PrestoException(STORAGE_ERROR, "Disk page checksum does not match. " +
                            "Data seems to be corrupted on disk for file " + tempStorageHandle.toString());
                }
            }
            return pages.build();
        }
        catch (UncheckedIOException | IOException e) {
            throw new PrestoException(STORAGE_ERROR, "Unable to read data from disk: ", e);
        }
    }

    public long getRetainedSizeInBytes()
    {
        return prestoSparkBroadcastTableCacheManager.getBroadcastTableSizeInBytes(stageId, planNodeId);
    }

    public long getStagingBroadcastTableSizeInBytes()
    {
        return stagingBroadcastTableSizeInBytes;

View on GitHub (pinned to 55bb57d202)

Solutions

Re-run the query — the spill file may need regeneration
Investigate the underlying temp storage health (disk SMART, HDFS/S3 integrity) for the specific file in the message
Ensure nothing external reads/writes or cleans the temp storage directory while queries run
Verify checksums/stability of the storage backend and consider a different storage medium

Example fix

// no caller-side fix; diagnose via the storage handle in the message
// check the file at the temp storage location referenced by tempStorageHandle
// before retrying: clear stale broadcast spill files for the stage
// then re-run the affected query

Defensive patterns

Strategy: retry

Validate before calling

// pre-flight storage integrity check
Checksum c = computeChecksum(tempStorage.get(ctx, handle));
if (c.getValue() != prestoSparkStorageHandle.getChecksum()) {
    throw new IllegalStateException("spill file corrupted before read: " + handle);
}

Type guard

null

Try / catch

try {
    return diskPageInput.getPages();
}
catch (PrestoException e) {
    if (e.getMessage() != null && e.getMessage().contains("checksum does not match")) {
        // do NOT retry blindly: corruption is deterministic; invalidate spill data and re-broadcast
        throw new IllegalStateException("re-run query to regenerate broadcast spill", e);
    }
    throw e;
}

Prevention

Monitor storage media health (disk errors, ECC, S3/HDFS integrity reports)
Never let external processes modify temp storage files
Use storage backends with end-to-end integrity verification
Treat a checksum error as deterministic corruption: regenerate data, don't loop retries

When it happens

Trigger: getPages() -> loadBroadcastTable() reads a broadcast spill file, computes a checksum, and compares it against prestoSparkStorageHandle.getChecksum(); any mismatch throws immediately.

Common situations: Bit rot or truncated files on disk/HDFS/S3; external modification of temp storage files; faulty disks or network corruption during remote read; files overwritten by a colliding temp path.

Related errors

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/42d2493eed3ba5d3. Report an issue: GitHub.